GSOC 2023: o3-Draw-on-body-diagram app- Project Updates & Discussion

[thembo42 (thembo jonathan)]

Hello folks ,

I will be working on the project o3:Draw-on-body-diagram for the OpenMRS community with very high priority for orgs like @ICRC and many more.

The main objective of this project is to improve the diagramming feature in OpenMRS, a medical record system, by allowing the upload of any diagram as an image, annotating certain areas of the diagram with different shapes, saving and retrieving these diagrams, and downloading them as an image with edited annotations. The project has two parts, essential and desirable objectives(more to come in the further implementation), with the former providing basic functionalities for uploading and editing diagrams while the latter enhances the user experience by adding more features such as drawing free shapes and downloading annotated diagrams for easy sharing and printing.

This thread will be used for project updates and discussions.

Mentors

Primary: @heshan

Backup : @jayasanka

Student: @thembo42

RESOURCES

Project Details(still being worked on): Link

The ICRC war surgery manual : Link

Istanbul protocol to document torture/ill treatment: Link

All technical thoughts are welcome!

@grace @jesplana @pauladams @cduffy @mksd @ibacher @ball @johnblack @mogoodrich @burke @jayasanka @kdaud @heshan

[thembo42 (thembo jonathan)]

FIRST MENTOR MEETING

When: 2023-05-12T12:00:00Z

Where: Google meet

Link: Gsoc mentor meeting Friday, May 12 Time zone: Africa/Nairobi Google Meet joining info Video call link: https://meet.google.com/ean-udmv-nge

AGENDA

1. Pro Tips and Guidance

2. Project Timeline(realistic break down)

3. Technologies Required(update a few things)

4. UI/UX Designs(moving forward, I will love to hear from the @ICRC team)

@heshan @jayasanka @kdaud @jesplana and all that are interested

[burke (Burke Mamlin)]

Some random thoughts:

• Avoid building a body-specific solution when an image notation solution that can use a body could also be used for notation of other clinical images
• Provide methods for defining regions and then answering questions like “what notations are in this region” and “which regions include this notation”
• Design a portable solution that can be introduced where needed in the app rather than making assumptions about where the diagram tool will be used.
• Make it really easy to add a simple notation, since the most common use case for diagrams like this are one or two notations (e.g., show where a specific finding is located with detail provided elsewhere in a clinical note)

[thembo42 (thembo jonathan)]

Thanks so much @burke , that was insightful,

Just if I got you right(you have detailed; portability, flexibility and scalability)

1. Body-specific vs. general image notation: To avoid building a body-specific solution, the diagramming feature should be designed in a way that can be used to annotate any type of clinical image, not just body diagrams. This would require designing a flexible annotation tool that can be applied to different types of images.

Some general questions that can arise to make a more reliable solution and get a well rounded and grounded understanding of the whole project. Feel free to drop your thoughts.

1. How will the diagramming feature be integrated into the existing OpenMRS system? Will it be developed as a standalone micro-frontend or integrated into an existing module? How will it interact with the Java and Spring components of the core OpenMRS system?

2. What image formats will be supported for uploading and annotating images? Will the diagramming feature be able to handle large images, such as high-resolution medical images?(addressed some how in the proposal)

3. How will annotations be stored and retrieved from the system? Will they be stored as part of the patient record or in a separate database? How will annotations be associated with specific patients and encounters?

4. What user roles and permissions will be needed for the diagramming feature? Will different roles have different levels of access or permissions for creating, editing, or viewing annotations?

5. How will the diagramming feature be tested and validated(both ui/ux and the system functionality), both during development and after release? What metrics will be used to measure the success of the feature?

6. What overall tools and libraries will be used for developing the diagramming feature? Will they be compatible with the existing OpenMRS technology stack?

7. How will the project be scoped and prioritized, given the constraints of the GSoC timeline and the size of the project? What features or functionality should be considered essential versus desirable, and how will they be implemented and tested?

Some insights were shared in the proposal but now we can scope it down to the needs of openmrs and other orgs. I suppose answering these will get me up to speed for the next phase.

@heshan @jayasanka @ibacher @burke @grace @jesplana …

[thembo42 (thembo jonathan)]

Previously I had come up with an epic that mainly was tailored to refactoring the legacy drawing module, I know refactoring is quite expensive.

https://issues.openmrs.org/browse/O3-1817

GitHub - jona42-ui/openmrs-esm-drawing

I suppose i need guidance whether to continue with refactoring or get a fresh module(I know @heshan & @jayasanka will guide me here)

[thembo42 (thembo jonathan)]

@burke

Design a portable solution that can be introduced where needed in the app rather than making assumptions about where the diagram tool will be used.

This partly answers question one;

Portable solution: To ensure the diagramming feature can be introduced where needed in the app, it should be designed as a portable component that can be integrated into different parts of the OpenMRS system as needed. This might involve using standard APIs or designing a modular architecture that can be easily customized.

[thembo42 (thembo jonathan)]

FIRST MENTOR MEETING OUTCOMES

On May 12, 2023 3:00 PM i had my first mentor meeting and it was insightful with the agenda covered and mainly focusing on leveraging the workflow of uploading an annotated diagram with the best data structure and api implementation. With curiosity i first looked at the GitHub - openmrs/openmrs-module-attachments: UI components and backend web services to upload, view and manage attachments within OpenMRS. “Attachments” module that provides functionality for uploading and managing file attachments, including images.

Testing it out, it lacked some APIs(though it has most functionalty, will need guidance on how to still use it to avoid re-inventing the wheel @heshan @ibacher @dkayiwa @jayasanka ). I know I can Integrate the necessary APIs from the Attachments module into this project to enable image upload functionality but i think we can look at the new implementations then advise accordingly

===========================================================================

                     **BACK-END FOCUS**

I wanted to provide us with an update on the progress I’ve made so far on the scoped-down solution we discussed with @heshan .This week, the focus has been/is on allowing users to upload annotated images, storing them in the database, and implementing the necessary APIs for storage, retrieval, and display on the frontend.

Here are the details of the implementation:

1. Database Implementation:

• Created the required tables in the database to store image files and annotations.
• The Images table includes columns for filename, filesize, filetype, metadata, and data (LONGBLOB type).

• The DiagramAnnotations table includes columns for diagram ID, x and y coordinates, and description.

1. API Implementations:

• Implemented the necessary API endpoints for handling image upload, retrieval, storage, and download.
• The ImageController class includes methods for handling HTTP requests related to image operations.
• The endpoints accept the necessary parameters, such as file uploads, and interact with the corresponding service methods.

2. Service and Repository Implementations:

• Implemented the ImageService class responsible for handling the business logic of image operations.
• Implemented the ImageRepository class to interact with the database and perform CRUD operations on images.

Preparing a demo for this

Please let me know if there are any specific aspects you would like me to focus on or if there are any additional requirements or suggestions you have for the project. I’m looking forward to your feedback.

Thank you

[jesplana (John Mark Esplana)]

Hi @burke and @thembo42 , please see my comments below:

• Avoid building a body-specific solution when an image notation solution that can use a body could also be used for notation of other clinical images
  → Could we select from a template? For example, select the diagram name from a drop down, then the diagram loads. Once loaded, then we can start drawing or add notation. Additionally, the list from the drop down can also be based on the gender and other variables.

• Provide methods for defining regions and then answering questions like “what notations are in this region” and “which regions include this notation”
  → Could we then have the region question in the notation pop-up window? Also, could we pre-filter the diagram templates based on the region? For example, the user selects “Lower Limb (LL) - Right” in the region, then the diagram is filtered to show only diagrams? But i’m not if this can be saved in the table (i.e. obs_group to store region info, value_text to store the notation, etc.?)

[thembo42 (thembo jonathan)]

@jesplana

Could we select from a template?

sure this makes perfect sense. Some thing similar you shared some time back @jesplana

@burke does this make it possible to have varried clinical images and i suppose the data structure will handle the different diagrams . Did i get this right?

[thembo42 (thembo jonathan)]

@jesplana

Could we then have the region question in the notation pop-up window?

i suppose its possible one can display a prompt or a form field asking them to define the region. This way, when the notation is saved, it will be associated with the corresponding region information.

@jesplana

could we pre-filter the diagram templates based on the region?

i suppose this is too possible, just to refer to the table i defined above DiagramAnnotations table, I would suggest to add columns region_id or region_name to store the region information associated with each annotation.

Needs quite a bit of modification though. Am not sure if i got that well!

Quite confusing to get the right data structure sufficient here. the scope seems tricky @ibacher @heshan @jayasanka can we scope out this am in a valley of whether to setup the project with the Attachment module apis and make modifications(am skeptical of the cost) or i set up fresh structure and apis(afraid of re inventing the wheel)

[thembo42 (thembo jonathan)]

SECOND MENTOR MEETING

When: 2023-05-19T12:00:00Z

Where: Google meet

Link: 03-Daw-On-body meetings Friday, May 19 · 3:00 – 4:00pm Time zone: Africa/Nairobi Google Meet joining info Video call link: https://meet.google.com/qne-ewgm-vpv

AGENDA:

1. updates on weeks tasks
2. Tips and guidance moving foward
3. AOB

@heshan @jayasanka @kdaud @jesplana and all that are interested

[thembo42 (thembo jonathan)]

SECOND MEETING AND WRAPPING UP COMMUNITY BONDING

Hi

Quite abit of tasks and learning areas have gone through the last week. The main focus has been;

1. Organizing the software engineering thought process around building this o3 app. my mentor @heshan took me through a scoped down solution focusing on dividing the system into manageable pieces.

Firstly;

Looking at the data structure of my 03 application especially the diagramming and annotating structure, I was challenged to visualize what users will expect to see on the front end and then structure needed to incarnet this i.e User Centric Design. So there was need to update the previuosly designed data base structure to include more design anthetetics;

Diagram table;

DiagramAnnotations table;

UPNEXT

• Looking at the entire UI/UX and scoping down to ;

1. Visualizing the list of diagrams created

2. Create diagram with annotation visual workflow

3. Upload diagram visual workflow

• APIs for;

1. Pick diagrams from db to the front end
2. Array of diagrams
3. Create diagram
4. Delete diagram
5. Replication of the diagrams

Further Testing the APIs of the GitHub - openmrs/openmrs-module-attachments: UI components and backend web services to upload, view and manage attachments within OpenMRS.

@jayasanka you could guide me on leveraging this REST API

Getting a referesher in spring

Thanks @heshan and @jayasanka

cc/ @grace @jesplana @burke @ibacher @reagan @mozzy

[thembo42 (thembo jonathan)]

WRAPPING UP THE COMMUNITY BONDING PERIOD

A few questions I used to gather requirements that i hope would address this week’s tasks(Annotated Diagram upload);

• in what situation do they need to draw something on the body in an EMR? During the admission or assessment process, doctors will indicate which part of the body was wounded or need a specific operation.

  

• what number one thing do you want to do on the screen especially editing/anoatating screen

1. First, the user needs to be able to fill in a form related to the admission or the assessment being conducted,

   

2. then when necessary, choose a template to document the injury or body parts involved for the intervention.

   

3. draw on the image

   

4. image is saved as part of the form

5. Be able to view all of the images/attachments for the patient

   

6. Do a side-by-side comparison (but this can be done by viewing in a new window)

@thembo42

Upload diagram visual workflow

@thembo42

• APIs for;

1. Pick diagrams from db to the front end
2. Array of diagrams

Request: GET /diagrams

Response:

[
  {
    "id": 1,
    "filename": "diagram1.png",
    "filesize": 1024,
    "filetype": "image/png",
    "description": "Pain measurement",
    "created_at": "2023-05-17T10:30:00Z",
    "updated_at": "2023-05-17T10:30:00Z",
    "annotations": [
      {
        "id": 1,
        "diagram_id": 1,
        "x_coordinate": 100,
        "y_coordinate": 200,
        "description": "saures",
        "created_at": "2023-05-26:45:00Z",
        "updated_at": "2023-05-26:45:00Z"
      },
      {
        "id": 2,
        "diagram_id": 1,
        "x_coordinate": 300,
        "y_coordinate": 400,
        "description": "stars",
        "created_at": "2023-05-26:00:00Z",
        "updated_at": "2023-05-26:00:00Z"
      }
    ]
  },
  {
    "id": 2,
    "filename": "diagram2.png",
    "filesize": 2048,
    "filetype": "image/png",
    "description": "RTI scan",
    "created_at": "2023-05-26:15:00Z",
    "updated_at": "2023-05-26:15:00Z",
    "annotations": [
      {
        "id": 3,
        "diagram_id": 2,
        "x_coordinate": 150,
        "y_coordinate": 250,
        "description": "circles",
        "created_at": "2023-05-17T11:30:00Z",
        "updated_at": "2023-05-17T11:30:00Z"
      }
    ]
  },
  ...
]

More of the API implementation to be seen in week one that is next week. I want to welcome suggestions and adjustments to the designs as we prepare for week one coding period.

Thank you

@heshan @jayasanka @cduffy @pauladams @jesplana