We discussed backend support for forms on yesterday’s Platform Team Meeting. I’m going to refer to this as “draft form data” (to distinguish from the ambiguity of “draft form”, which could refer to a definition of a form that hasn’t been completed).
We focus on support for draft form data
We need to agree on API endpoints
New API endpoints would be added via a Form Data module that would add REST endpoints by depending on & registering endpoints through the REST Web Services module (in a new FormDataService)
The basic properties of a draft form data would be
User
Patient
Which Form is associated with draft form data
Draft form data contents (serialized to JSON)
Timestamps for “created on” and “updated on”
We’re assuming draft form data will persist forever (i.e., until deleted) for now. Having timestamps will allow for TTLs to be applied later without having to be decided up front.
The next step is to agree on the endpoints needed. Our assumption would be:
CRUD (save, get, update, and delete draft form data)
POST /formdata
GET /formdata/:uuid
PUT /formdata/:uuid ← our REST module might use POST here
DELETE /formdata/:uuid
Search draft form data (filtered by patient, user, and/or form), for example:
GET /formdata?patient=:uuid&user=:uuid
GET /formdata?patient=:uuid&form=:uuid
Are there any other endpoints you would need?
@ibacher & @dkayiwa, please let me know if I misrepresented any of our conversation.
FormDataService
Initially, I thought these endpoints would belong in our FormService; however the OpenMRS FormService is focused on form definition management – i.e., metadata, not clinical data. After a little more thought, form data (like FHIR’s QuestionaireResponse ) represents data that are (or are going to be) part of an encounter, similar to observations, orders, encounter diagnoses, notes, etc. So, it might make more sense to introduce this as FormData a better fit for managing form data (even in draft) would be within the EncounterService or, better yet, within a new FormDataService to parallel ObsService, OrderService, etc.
Thanks for the recap @burke . It sounds like there is a solid plan in place. @dkayiwa , @ibacher , is there any capacity in the platform team to start work on this?
This is on my list of things to do either this week or next week, so yes…
One short-coming of our current forms model is that we don’t really persist the “form as answered” (which is what a QuestionaireResponse really is). This means that all our form engines end up doing work to go from data as persisted (Obs, Orders, etc.) back to data as entered. (Of course, the upside is that this means there’s no possibility of data errors where what was captured on the form and what became an Obs differ).
From our discussions, I think, at least for the immediate use, we’re going to end up storing something that looks more like the full encounter payload in the formdata service, since the AMPATH engine already knows how to translate an encounter payload into pre-entered data in a form.
I’m digressing a little so feel free to ignore or branch out a new topic, but, would there be any value in at least recording just the actual question that was asked? As a middle ground between 1) what we’ve got now and 2) keeping a record of the entire questionnaire as it was used (which is what you alluded to).
Of course this would require to expand the Obs model to hold a member like “actual question asked” or “exact question asked”, whose value would always be in the actual language used on the form.
This not even a hard requirement from us. We haven’t go that far in the data science that someone asked what the actual question asked was for each recorded observation. But I remember @jdick once mentioning this as well over a call a while back.
The original design for forms in OpenMRS was to record form & form_field definitions that would contain the actual questions asked, linking to the dictionary for concepts used and linking to obs for data generated.
We also envisioned adding “form data” as another type of data generated within an encounter (alongside obs, orders, diagnoses, and notes) to store arbitrary form data collected during an encounter. The idea was form data could (but would not have to) generate obs – you could generate obs for form data that you needed in flowsheets, decision support, or reports, and leave other form data (without generated obs) for auditing purposes (e.g., questions used for billing, to set attributes on the encounter/visit, or be used for workflows that didn’t need obs).
Our form service is focused on form design/definitions (like FHIR’s Questionnaire). A form data service could focus on storing/retrieving instances of data collected using a form (FHIR’s QuestionnaireResponse). We could link form data to obs generated from them (like we have currently done loosely with form_namespace_and_path so edits to form data could update associated obs when they exist.