Hi everyone,
I’m Tayo Imani, a PhD student at Towson University. My research asks whether AI tools can help with change impact analysis for healthcare software: when a change is made, predicting what else it affects and which qualities it puts at risk (security, privacy, safety, performance, auditability). That check is required for medical-device software under IEC 62304, and today it’s done entirely by hand.
To measure this fairly, I’m building an open benchmark from real OpenMRS and HAPI FHIR changes: JIRA tickets or GitHub issues plus their linked commits. I’m looking for two experienced engineer to act as a second and third, independent rater for the quality labels.
What the work involves (all in a browser page plus a spreadsheet, at your own pace):
- Calibration (~1 hour): read 90 short GitHub issues and mark whether each is about security, performance, or a feature.
- Pilot (~3–5 hours): for 40 real OpenMRS/HAPI FHIR changes, read the ticket and diff, and mark which of five qualities the change improves or threatens, with a short reason. A codebook explains every rule.
- Later, optional (~10–15 hours): the same for about 150 more changes.
Who I’m looking for:
- Several years of Java experience, ideally with OpenMRS core or HAPI FHIR
- Familiarity with health-IT concerns (patient privacy, audit trails, clinical data safety). Exposure to HIPAA or IEC 62304 is a plus, not required
- Able to work independently: no comparing notes with me during labeling, and no AI tools, because the labels have to be a human judgment
What you’d get: an acknowledgment in the paper and the benchmark will be released openly, so the community can use it to evaluate tools on OpenMRS and FHIR code.
All the data is public open-source code and tickets: no patient data, no confidential material.
I’d love to start the calibration and pilot this week. If you’re interested, or know someone who might be, please reply here or message me at [email protected]. Happy to answer questions or share the codebook first.
Thank you! Tayo