Module for ETL & Predictive Modeling across Multiple Platform (DW) Endterm Presentation

Community GSoC
1

Greetings Everyone,

This GSoC session with OpenMRS is like a wonderful journey for me and for all of us. We learned a lot from the platform which is provided for us. I am very thankful to OpenMRS, Google Summer of Code and God for providing me such a wonderful opportunity to learn something new and build something big.

Here is a short presentation video about my work that I have done in last three months. Please feel free to add your comments and review about my work. Enjoy watching this video.

https://www.youtube.com/watch?v=mQ3VMST8rVA

Problem Definition Healthcare provider can check upon the predictive modeling result based on historical data they are having/loading.

Problem Solution

1. Extract, Transform and Loading

  • Login to the database.
  • Select and Choose your data.
  • Login to Datawarehouse.
  • Load data in the DW.

2. Predictive Analysis

  • Login to DW
  • Run the analytics query on the DW
  • Fetch Back the Results.
  • Preview Results in Tabular, Graphs and Charts.

Timeline Accomplishments

  • ETL Module Wizard : User Friendly wizard for gathering information from user.
  • Access to MySQL and Hive database in module Web Interface.
  • Drag-Drop support for selecting data columns.
  • Loading data into Hive-Hadoop.
  • Running Queries over Hive.
  • Fetch Back Result in module Web Interface as tabular, graphs and charts form.

Additional Accomplishments

  • ETL Module Scheduler: For scheduling ETL tasks.
  • Auto Login with current OpenMRS MySQL Instance.
  • Downloading DW Queries Results as Excel Sheet, CSV and TSV.
  • Raw SQL Join Queries can include for transformation.
  • Saving/Loading Queries from the local Storage.

Next Step and The Future

  • Support for Other database as DW.
  • Real-time Manipulation with the Analytics Data in the module.
  • Support for Email and Sharing.

Learned So Far

  • Web Technologies : Java Servlets, Spring MVC, DWR, Ajax etc.
  • Web Designing : HTML, CSS, JavaScript and jQuery.
  • Big Data Technologies : Hadoop, Hive, Sqoop etc.
  • Coding for Open Source : Developers Communication, Patience and Friendship.

Resources

4 Likes