Module for ETL & Predictive Modeling across Multiple Platform (DW) Endterm Presentation

Greetings Everyone,

This GSoC session with OpenMRS is like a wonderful journey for me and for all of us. We learned a lot from the platform which is provided for us. I am very thankful to OpenMRS, Google Summer of Code and God for providing me such a wonderful opportunity to learn something new and build something big.

Here is a short presentation video about my work that I have done in last three months. Please feel free to add your comments and review about my work. Enjoy watching this video.

Problem Definition Healthcare provider can check upon the predictive modeling result based on historical data they are having/loading.

Problem Solution

1. Extract, Transform and Loading

  • Login to the database.
  • Select and Choose your data.
  • Login to Datawarehouse.
  • Load data in the DW.

2. Predictive Analysis

  • Login to DW
  • Run the analytics query on the DW
  • Fetch Back the Results.
  • Preview Results in Tabular, Graphs and Charts.

Timeline Accomplishments

  • ETL Module Wizard : User Friendly wizard for gathering information from user.
  • Access to MySQL and Hive database in module Web Interface.
  • Drag-Drop support for selecting data columns.
  • Loading data into Hive-Hadoop.
  • Running Queries over Hive.
  • Fetch Back Result in module Web Interface as tabular, graphs and charts form.

Additional Accomplishments

  • ETL Module Scheduler: For scheduling ETL tasks.
  • Auto Login with current OpenMRS MySQL Instance.
  • Downloading DW Queries Results as Excel Sheet, CSV and TSV.
  • Raw SQL Join Queries can include for transformation.
  • Saving/Loading Queries from the local Storage.

Next Step and The Future

  • Support for Other database as DW.
  • Real-time Manipulation with the Analytics Data in the module.
  • Support for Email and Sharing.

Learned So Far

  • Web Technologies : Java Servlets, Spring MVC, DWR, Ajax etc.
  • Web Designing : HTML, CSS, JavaScript and jQuery.
  • Big Data Technologies : Hadoop, Hive, Sqoop etc.
  • Coding for Open Source : Developers Communication, Patience and Friendship.