Cohorts and Cohort Searches

[mseaton (Mike Seaton)]

Hi @femi,

There are more options available for these needs:

@femi

Concept & Observation a. We can’t see a cohort definition that supports searching patients who have (or does not have) a particular observation

These certainly exist. You should be able to use NumericObsCohortDefinition, CodedObsCohortDefinition, DateObsCohortDefinition, and TextObsCohortDefinition

@femi

Encounter a. There is no parameter that supports fields like (within the last months and days, since and until). We had a workaround as regards this by using date but we don’t know if that will suffice.

This is possible by use of expressions when mapping parameters. There is some explanation of what is available to do this here. It is also possible by creating an appropriate definition library that takes in the number of months/days and units, and converts this to a date.

@femi

c. Search by form also generates an error, the link to the error is here encounter form search - Pastebin.com. The encounterAdvancedSearch definition library key was added here REPORT-801: Add an anyEncounterDuringPeriodWithOccurrence to the BuiltInCohortDefinitionLibrary by fortunee · Pull Request #140 · openmrs/openmrs-module-reporting · GitHub

Based on the error message I see here:

java.lang.NullPointerException\n\tat org.openmrs.module.reporting.report.util.SqlUtils.openmrsObjectIdListHelper(SqlUtils.java:213)\n\tat org.openmrs.module.reporting.cohort.query.db.hibernate.HibernateCohortQueryDAO.getPatientsHavingEncounters(HibernateCohortQueryDAO.java:567)\n\tat org.openmrs.module.reporting.cohort.query.service.CohortQueryServiceImpl.getPatientsHavingEncounters(CohortQueryServiceImpl.java:156)\n\tat

My best guess is that this is caused more by bad test data than by any problem in the reporting framework. Were you testing this on patients and encounters that were previously saved, or just created in memory and not yet persisted?

@femi

Programme Enrollment a. No definition library key

I’m not sure what this means, but we should be able to just add in more library methods to meet our needs. There are several program-related cohort definitions available: ProgramEnrollmentCohortDefinition, PatientStateCohortDefinition, InProgramCohortDefinition, InStateCohortDefinition.

@femi

Drug Order a. No cohort definition

This does appear to be missing. We’d need to ticket this and get it added to the reporting framework.

Hope this helps a little. I’m on vacation this week, so I may be slower following up than normal. @darius FYI.

Mike

[darius (Darius Jazayeri)]

@mseaton

Concept & Observation a. We can’t see a cohort definition that supports searching patients who have (or does not have) a particular observation

These certainly exist. You should be able to use NumericObsCohortDefinition, CodedObsCohortDefinition, DateObsCohortDefinition, and TextObsCohortDefinition

By the way, the old cohort builder has a feature to search for something like “patients with any observations for which this concept is the answer”. I think we should get rid of this function. (@ssmusoke please confirm this). Getting rid of it should mean that the existing definitions that Mike lists here are sufficient.

@mseaton

This does appear to be missing. We’d need to ticket this and get it added to the reporting framework.

@jeffneiman can we schedule to talk on the next design forum about “what drug order search functionality to support in the Reporting module”

[jeffneiman]

Will do! Shall we say for this Monday?

[ethan (Chukwuemeka Nwankwo)]

Hi @mseaton

Thanks for pointing out these observation cohort definitions. We have tried looking at them, however, we are experiencing challenges making patient observation searches with them. When we worked on encounter search, we added a definition library to the built-in cohort definition file in this PR REPORT-801: Add an anyEncounterDuringPeriodWithOccurrence to the BuiltInCohortDefinitionLibrary by fortunee · Pull Request #140 · openmrs/openmrs-module-reporting · GitHub, using that, we were able to compose a JSON of the format shown below for the reportingrest/adhocquery resource:

{
  "type": "org.openmrs.module.reporting.dataset.definition.PatientDataSetDefinition",
  "columns": [
    {
      "type": "org.openmrs.module.reporting.data.patient.definition.PatientDataDefinition",
      "name": "Given Name",
      "key": "reporting.library.patientDataDefinition.builtIn.preferredName.givenName"
    },
    {
      "type": "org.openmrs.module.reporting.data.patient.definition.PatientDataDefinition",
      "name": "Family Name",
      "key": "reporting.library.patientDataDefinition.builtIn.preferredName.familyName"
    }
  ],
  "customRowFilterCombination": "1",
  "rowFilters": [
    {
    	"type": "org.openmrs.module.reporting.cohort.definition.PatientDataDefinition",
    	"key": "reporting.library.cohortDefinition.builtIn.encounterSearchAdvanced",
    	"parameterValues": {
    		"encounterTypes": ["ADULTINITIAL"]
    	}
    }
  ]
}

We would like to clarify how we can specify parameterValues for the query as shown above for the observation search form shown below for Hepatotoxicity:

Thanks.

[mseaton (Mike Seaton)]

Hi @ethan,

So in this case, just like you did with the encounter search, you would take a similar approach and create a “codedObsSearchAdvanced” definition (or some better name). This would instantiate a new CodedObsCohortDefinition, and expose each of it’s configuration properties. Among these are:

• “question” - reference to the Hepatotoxicity concept in your example
• “onOrAfter” - the “since” parameter in your example
• “onOrBefore” - the “until” parameter in your example

To support the last X months or Y days, it is a bit more complicated. Now that I look at it, probably the easiest thing to do would be to convert these into dates in UI layer, in JS, and then pass these dates through to the “onOrAfter” and “onOrBefore” as if the user entered these as dates directly.

Does this make sense? @darius, any other comments?

Mike

[ethan (Chukwuemeka Nwankwo)]

Hi @mseaton,

Thanks again for the pointers and clarification, we’ll work in this line and reach out again if we encounter further challenges.

Regards, Ethan

[darius (Darius Jazayeri)]

@mseaton

To support the last X months or Y days, it is a bit more complicated.

I would just skip these in the first version, and put a ticket in the backlog to add them later.

(@ssmusoke do you know if in practice the “obs in last X months / Y days” is used much in the cohort builder? (E.g. is just having the date range a sufficient MVP?)

[ssmusoke (Stephen Senkomago Musoke)]

@darius I would keep a date range for now, since the in the last X months or Y days still amounts to a date range which is more important the time from today, since the the typical questions are to do what happened in a specific month or quarter.

[ethan (Chukwuemeka Nwankwo)]

Hi @mseaton,

Hope you’re doing great.

We’ve taken extensive look at the CodedObsCohortDefinition cohort definition and tried adding a codedObsSearchAdvanced definition library like we discussed but we discovered that CodedObsCohortDefinition does not have question, onOrAfter or onOrBefore configuration properties like we have in the other cohort definitions, instead it has just operator and valueList configuration properties.

We have explored the option of adding these configuration properties (i.e. question, onOrAfter or onOrBefore) to the CodedObsCohortDefinition and specify getters and setters for them and expose them to the new codedObsSearchAdvanced but that too does not solve the problem.

We’re wondering how we can use the already specified configuration properties (i.e. operator and valueList) to make searches based on observations?

We also tried using the DateObsCohortDefinition cohort definition to achieve this to no avail.

Thanks.

[mseaton (Mike Seaton)]

@ethan, sorry for the confusion. The CodedObsCohortDefinition does have these configuration properties. They are defined in a superclass - BaseObsCohortDefinition - that the more specific ObsCohortDefinition implementations extend.

Mike

[ethan (Chukwuemeka Nwankwo)]

Yes @mseaton, I thought about that too, but then I discovered BaseObsCohortDefinition cannot be instantiated given that it is an abstract class.

Is there a way we can make the public sub-classes inherit these configuration properties from BaseObsCohortDefinition?

Also, I came across ObsInEncounterCohortDefinition which I extended to create the definition library and it returned search results, and we were wondering if this is sufficient for performing the observation searches?

Thanks, Ethan

[mseaton (Mike Seaton)]

@ethan, the public sub-classes do inherit these configuration properties.

[darius (Darius Jazayeri)]

@ethan, to be more verbose, in Java a subclass inherits the methods of its superclass.

So for example you can do:

CodedObsCohortDefinition cd = new CodedObsCohortDefinition();
cd.setQuestion(questionConcept); // this method is inherited from BaseObsCohortDefinition
cd.addValue(answerConcept);

Or, in the case of definition libraries you would addParameter instead of calling the setter.

[ethan (Chukwuemeka Nwankwo)]

Thanks @darius and @mseaton, I will take these into account and restructure the new definition library.

[ethan (Chukwuemeka Nwankwo)]

Hi @darius and @mseaton,

Like you pointed out, we’ve tried creating the cohort definition using the CodedObsCohortDefinition and specifying the question, onOrBefore and onOrAfter parameters since it extends the BaseObsCohortDefinition as shown below but it still does not perform the searches:

    @DocumentedDefinition("codedObsSearchAdvanced")
    public CohortDefinition getPatientsHavingObs() {
        CodedObsCohortDefinition cd = new CodedObsCohortDefinition();
        cd.addParameter(new Parameter("question", "reporting.parameter.question", Concept.class));
        cd.addParameter(new Parameter("onOrAfter", "reporting.parameter.startDate", Date.class));
        cd.addParameter(new Parameter("onOrBefore", "reporting.parameter.endDate", Date.class));
        return new MappedParametersCohortDefinition(cd, "onOrAfter", "startDate", "onOrBefore", "endDate", "question", "question");
    }

However, when the ObsInEncounterCohortDefinition is used in creating the definition library as shown below, It returns search results:

    @DocumentedDefinition("obsSearchAdvanced")
    public CohortDefinition getPatientsHavingObs() {
        ObsInEncounterCohortDefinition cd = new ObsInEncounterCohortDefinition();
        cd.addParameter(new Parameter("question", "reporting.parameter.question", Concept.class));
        cd.addParameter(new Parameter("encounterOnOrAfter", "reporting.parameter.startDate", Date.class));
        cd.addParameter(new Parameter("encounterOnOrBefore", "reporting.parameter.endDate", Date.class));
        return new MappedParametersCohortDefinition(cd, "encounterOnOrAfter", "startDate", "encounterOnOrBefore", "endDate", "question", "question");
    }

So we were wondering how ObsInEncounterCohortDefinition differs from CodedObsCohortDefinition as it concerns making observation searches?

Thanks again for putting up with my many requests

[burke (Burke Mamlin)]

@darius

Someone had turned the wiki off in the project settings. I don’t know whoor why.

I have turned it back on, and the documentation is available again.

As mentioned here, we have discouraged GitHub wikis to avoid splitting wiki content between our wiki and GitHub, thereby making it harder for people to find content. Based on that, I routinely run a script to clean up the OpenMRS GitHub organization’s repos which ensures Dev Stage teams are assigned to new repos and wiki & issues are disabled on all repos.

I’ve added openmrs-module-reportingrest as an exception, but will continue to auto-disabled wiki & issues for repos, encouraging people to use the wiki for documentation.

[mseaton (Mike Seaton)]

@ethan, I don’t know why your query would not work off-hand. I was going to point out that it was probably failing since you had set the onOrAfter parameter twice, but I see you have gone back and edited that in your original post. I assume now that this did not fix your problem?

The difference between the two definitions is that the ObsInEncounterCohortDefinition first finds the appropriate encounter for a patient, and then queries for obs within that encounter. So it allows you to do things like check for a particular Obs in a particular encounter.

From what I can see, the ObsInEncounterCohortDefinition only has support for Date values, so you would not be able to use it as is to query for CodedObs, aside from what you are doing above, which is just querying for the presence of those Obs, rather than any particular values (which is generally what you’d want).

Can you point me to some code somewhere that demonstrates the definition library using the coded obs definition not working?

Mike

[darius (Darius Jazayeri)]

@ethan, one trick you can try, if you are running the server, is to set this in your openmrs-runtime.properties file:

hibernate.show_sql=true

This will print a lot of info into your log file (so much that it slows down the server), but specifically this will print out the SQL command that is executed when you run that query. You could then look at this query, and the database, and see why it’s not finding results.

(If you do enable this setting, you’ll want to disable it after you’re done.)

[ethan (Chukwuemeka Nwankwo)]

Thanks @mseaton and @darius, the observation search returns results now.

After setting

hibernate.show_sql=true

in the openmrs-runtime.properties like @darius pointed out, there was a more detailed error on the server log showing

TimeModifier 'null' not recognized

which led me to take another look at the BaseObsCohortDefinition cohort definition where I discovered there was a TimeModifier enum which is a required parameter, I had to restructure the new definition library to add the parameter shown below:

cd.addParameter(new Parameter("timeModifier", "reporting.parameter.timeModifier", CodedObsCohortDefinition.TimeModifier.class));

Now passing the timeModifier parameter in the search query which could be either of these ANY, NO; returns results as expected.

Thanks for the insights.

[mseaton (Mike Seaton)]

Thanks @ethan, good catch. Other definitions in the reporting module default this to “ANY” if it is not otherwise specified, but it looks like the BaseObsCohortDefinition requires this to be explicitly done. I created this ticket, and issued a fix here. If you are able to test this out with your previous code, let me know how this works for you.

Thanks, Mike