HFE: Conditions should have a pointer for the form_field it was created through

Development
1

Hi all,

HFE has an inherent limitation explained as follows below:-

After submitting data with a form, when opening that form again in “view or edit mode”, there is no existing smart mechanism for matching a question on the form to it’s corresponding answer. Let’s use the <obs/> tag as an example it being the most popular HFE tag.

I will mention out that an attempt was made to resolve this same issue with the <obs/>. The Obs data model had to be adjusted by adding a pointer for the form_field to the Obs class however, refactoring the tag implementation to reply on the updated Obs model was never completed(see HTML-163 and HTML-10).

Expand for details on how to reproduce this bug with the <obs/> tag (GIF inclusive)
  1. Create a form with many questions(using the <obs/>) but having the same concept question(conceptId) ie:-
                            <section>
                                                  Field 1:
                                                  <obs conceptId="CIEL:123" style="textarea" col="6" />
                                                  Field 2:
                                                  <obs conceptId="CIEL:123" style="textarea" col="6" />
                                                  Field 3:
                                                  <obs conceptId="CIEL:123" style="textarea" col="6" />
                                                 Field 4:
                                                  <obs conceptId="CIEL:123" style="textarea" col="6" />
                             </section>
  1. See the bug unmasked below:

2020-06-22 17-10-57.2020-06-22 17_12_13

Did you see how answers were poorly mapped to their questions in edit mode?

The Condition class also needs to have a pointer for the form-field for cases where the condition is being recorded by HFE specifically the <condition /> tag as a solution to the problem described above.

Actually, since HFE framework is Encounter centric, @mksd suggests we could add this support at all qualifying members linked to an Encounter all in one ticket?

Thoughts, ideas, early findings are all welcome.

@mseaton, @mogoodrich, @ssmusoke

2

Thanks @samuel34!

My initial thought is that we should (finally!) go ahead and use the form_field property you mentioned above (which was released in 1.11) to link obs to specific form fields. Then, I think we could expand other domain objects that need this functionality to include this same property (perhaps via an interface?).

That’s after about 5 minutes of thought, however, so open to other suggestions!

Take care, Mark

2 Likes
3

@samuel34 that is the purpose of the obsgroup, you would wrap each field with a different obs group concept to provide the linking

4

Yeah sure,

@ssmusoke What is actually happening is that, the <obs/> tag uses the associated concept identified by conceptId(or possibly the obsGroup like you mentioned) param to map an answer(recorded Obs) to the underlying question on the form. Which I guess was just a workaround.

A fully fledged designed to solve this was proposed, this is why TRUNK-2069 exists.

5

@samuel34 if you use an obsgroup (different conceptId) around each field, then the issue is resolved

Why is that solution (however tedious) not viable?

6

So if the form_field is being added to the obs group, does this mean that HFE will support the ability to add data to an encounter from multiple forms?

Which can lead to ability to have a wizard like feature with sub-forms?

7

It’s interesting that other insightful ideas are propagating.

I will mention that it’s true that many implementations out there have not faced this issue with the <obs/> tag, probably by providing some smart configuration; however this issue can still happen in complex forms like @mseaton reported here.

Like I mentioned earlier, work was done in core(adding a pointer for the form_field to the Obs class ) to achieve a proper mechanism for obs but this is not the case for other parts in the data model.

We would like to sway in the direction of having similar work done for the Condition domain to have the <condition/> tag working in a proper way

/cc: @dkayiwa, @ibacher

8

@mogoodrich that was also my idea. @angshuonline, @dkayiwa, @ibacher, @mseaton, thoughts?

9

I completely agree with the proposal of expanding other domain objects that need this functionality to include this same property.

2 Likes
10

+1 from me too.

1 Like
11

I think we can go forward with doing actual development on this - will create a ticket soon.

I suggest the interface to be named HtmlFormEntryRecordable (better naming can be proposed) and below is the list of qualifying members, correct me if I added a wrong one or remind me if I missed some.

  • Obs
  • Condition
  • Diagnosis
  • Order

I would love your approval on the above suggestion :slightly_smiling_face:

/cc @mogoodrich, @mksd

12

I’d suggest using something a little bit less tied to HFE. Presumably (since we’re talking about a field added to everyone’s database), this same structure could be leveraged by other modules besides HFE. Something like FormRecordable or the like might be a better name.

3 Likes
13

Thansk @samuel34 for taking this on!

I think this is a big enough that it might be worth starting with a design page on the Wiki before breaking up into actual tickets. Not sure what others think… @mksd?

I’ll start trying to think through some thoughts myself… I’ll hope to weigh back in later today. FYI I’m off a lot over the next week and a half, so I might not respond quickly after today, but will try to stay up on things and at least weigh in when I’m back in the office.

Take care, Mark

14

Yes, I agree with @ibacher , we shouldn’t include “Html” in the name.

I also would skip Orders for now, as they are generally a special case sine they are immutable in a lot of ways.

My thoughts for a first pass:

  • Add formNamespaceAndPath property to Condition and Diagnosis classes and DB tables in Core (Question: backport to 2.3?) (Question: define an interface “FormRecordable” for this?)

  • Then, in HtmlFormEntry support a new “formPath” attribute on all the tags that we want to support this. For instance, for Obs I think we’d want to support something like this:

<!-- (made up mapping, probably don't actually exist) -->
<obs conceptId="CIEL:Weight" formPath="weight"/>

<obsgroup conceptId="CIEL:Blood Pressure" formPath="blood_pressure">
    <obs conceptId="CIEL:Systolic" formPath="systolic"/>
    <obs conceptId="CIEL:Diastolic" formPath="diastolic"/>
</obsgroup>

Then, when this is saved, it would populate the form namespace and path by concatenating the form name and paths. For instance, in the DB you’d have the following obs:

obs concept_id="(primary key of CIEL:Weight concept)"
form_namespace_and_path="vitals^weight"

obs concept_id="(primary key of CIEL:Blood Pressure concept)"
form_namespace_and_path="vitals^blood_pressure"

obs concept_id="(primary key of CIEL:Systolic concept)"
form_namespace_and_path="vitals^blood_pressure^systolic"

obs concept_id="(primary key of CIEL:Diastolic concept)"
form_namespace_and_path="vitals^blood_pressure^diastolic"

Then, the FormEntryContext.removeExistingObs methods could be updated to take this path variable, if present, into account when matching an obs to an obs tag.

Then hopefully follow a similar pattern for the Condition and Diagnosis tags.

I just chose the caret (^) as a delimiter, but there could be a better choice.

Also, not sure if we’d want to use the existing “formName” property on the “htmlform” tag or use “formPath” there as well.

Take care, Mark

1 Like
15

Thanks all for your input! Especially to @mogoodrich for all this insight about HFE, that’s great.

@samuel34 could you research how form_namespace_and_path ends up being filled in Bahmni, as well as how the form controls are configured? Surely @zouchine can help you get to this real quick. My hope would be that we can maybe adopt the same pattern.

1 Like
16

Ah thanks @mksd! If form_namespace_and_path is used by Bahmni, we’d definitely want to consider using the same/similar pattern for consistency.

17

Bahmni forms 2 internally depends on form-controls(For those of you not familiar with Bahmni’s form controls see documentation). Every form-control has an id. This is what is used to map an existing obs, to the form field through which it was created.

A typical value for form_namespace_and_path looks like '{bahmniNamespace}^{formName}.{formVersion}/{formControl.id}' eg. 'Bahmni^A New Form.1/1-0' and here is the function.

An observation is persisted with it’s form_namespace_and_path, and the json of the form scheme is also persisted in a json file ie:-

{
  "name": "A New Form",
  "id": 5,
  "uuid": "da036a23-0cf9-48a7-a95a-c4216b93b46a",
  "defaultLocale": "en",
  "controls": [
    {
      "type": "obsControl",
      "label": {
        "translationKey": "WEIGHT_1",
        "id": "1",
        "units": "(kg)",
        "type": "label",
        "value": "WEIGHT"
      },
      "properties": {
        "mandatory": false,
        "notes": false,
        "addMore": false,
        "hideLabel": false,
        "controlEvent": false,
        "location": {
          "column": 0,
          "row": 0
        },
        "abnormal": false
      },
      "id": "1",
      "concept": {
        "name": "WEIGHT",
        "uuid": "5089AAAAAAAAAAAAAAAAAAAAAAAAAAAA",
        "datatype": "Numeric",
        "conceptClass": "Misc",
        "conceptHandler": null,
        "answers": [],
        "properties": {
          "allowDecimal": true
        }
      },
      "units": "kg",
      "hiNormal": null,
      "lowNormal": null,
      "hiAbsolute": 700,
      "lowAbsolute": 0
    }
  ],
  "events": {},
  "translationsUrl": "/openmrs/ws/rest/v1/bahmniie/form/translations"
}

So later in view/edit mode, Bahmni can get the form control’s id from the obs’s form_namespace_and_path and map the obs to it’s correct form control by obsControl.id.

18

@samuel34 you mean

"{form namespace}^{form name}.{form version}/{(obs) control id}-0"

?

Where in the case of an obs control the id is set like this:

{
  "type": "obsControl",
  "id": "1",
  ...
}

@angshuonline two questions:

  1. What was the intent of the form namespace?
  2. What was the intent of the "-0" suffix (see here)?
19

Correct @mksd

20

What was the intent of the form namespace?

So that we can edit a saved obs using the same form, or can render the obs using the same form same version. Imagine a form that is modified and has a different version, and possibly the data was captured using an older version - form namespace allow us to get the observations and use the right form version to render the data in a form (read only in Bahmni if the “encounter-session” expires)

What was the intent of the "-0" suffix

In Bahmni we have a concept of allowing “Add more”. Imagine readings for some observations (E.g. family member history) or complaints - 0 is index of control in such cases. If there is more than 1 value bound to same control, then the index will be 1 or above