Existing numeric obs with no obs_reference_range

Development
1

In OpenMRS 2.7.0 (see TRUNK-6245), concept_reference_range and obs_reference_range were introduced.

Prior to running this version, reference ranges for numeric observations were only associated with the concept_numeric itself, in absolute, critical, and normal range values that only had a single possible value. Displaying or reporting on reference ranges associated with an Obs involved looking up this associated metadata on the Concept Numeric.

With the advent of the new feature in 2.7, a change was made to directly associate reference ranges with observations at the time those obs were created . From the point in which an existing implementation upgraded to 2.7+, any new Obs associated with a concept numeric would also have an entry in obs_reference_range. And due to a quirk in the way this was implemented, there are also situations in which these obs_reference_ranges would not have been created even after a 2.7 upgrade (eg. if the implementation had validation.disable = true set in their global properties), but that’s a discussion for another thread and ticket.

As far as I can tell, no migration was ever created to retroactively create reference ranges for existing obs. Is this true and if so, what was the thinking at the time?

The upshot is that implementations find themselves in a situation where newly created obs have obs reference ranges, and previously existing obs created before this feature do not.

Have others run into this and how have they handled it? Does a migration script exist in the community (eg. maybe in the 2.7 release notes as an optional step to execute due to the size of the obs table and how long it would take to run)? Some thoughts:

  • My guess is that 99% of implementations have no concept_reference_ranges defined at all, and a gated liquibase changeset could retroactively create obs_reference_ranges for these under those conditions (though this might be prohibitively slow)
  • We could also have a liquibase changeset that migrates any obs whose concept does not have a row in concept_reference_range, since that would always be assumed to use the values defined in concept_numeric up to that point (same slowness concern)
  • Or we could not migrate the data, but fall back to the values defined in concept_numeric when retrieving this data from the API, if no specific obs_reference_range exists (easiest to implement, but does this introduce possibilities of showing incorrect ranges)

Looking for input from those who built these features in, or who have expertise and/or opinions about what would be the best approach with data quality, data completeness, and patient safety in mind. I’m sure there are other options than the one’s I have initially laid out.

@ibacher / @burke / @akanter / @dkayiwa

Thanks,

Mike

Should reference ranges be updated when obs are edited?
2

Yes. For the most part, though, we’ve tried to make the APIs responsible for returning anything to do with reference ranges fallback to the concept reference range.

There are a few considerations here:

  • Obs tables are large and a migration to create a reference range for every numeric obs would be massive task; we generally don’t want to create changesets that cause upgrades to take hours by default.
  • We didn’t change any existing semantics; reference ranges prior to 2.7.x were provided by the concept_reference_range and for obs generated before the obs_reference_range existed, that’s presumably the controlling reference range.
  • Running the obs reference ranges, especially for years-old obs, might actually attach the wrong reference range to an obs where guidelines and thresholds have changed (rare, but it does happen).
  • Sometimes reference ranges vary a little bit depending on the equipment used to test them. I don’t think we have enough data to determine if that’s the case or not retroactively.
  • We didn’t (at the time) have a good community database with a large variety of observations, so writing such a migration with the change would’ve meant shipping something that may or may not be fit-for-purpose.
3

Thanks @ian . In our app, we are just retrieving Obs and then displaying their associated reference range, which is only populated for newly created obs (and only then if validation is not disabled), afaict.

Are you saying that you intended obs.getReferenceRange() to fallback to a reference range created on the fly from the underlying obs.getConcept()'s associated reference range? Or is this the responsibility of the all independent apps to handle and figure out on their own? How do we ensure this is done consistently across the app and module ecosystem?