Pros and cons of assessing distributions
The Friday presentation and the discussion at the end of the Bahmni session Friday late afternoon reminded me of some additional points on the pros and cons of evaluation distributions, whether for “certification” or "endorsement. These principles apply to endorsement of services providers as well, though I was not at that session, and am not sure exactly how that was discussed.
These issues are very relevant to the reference application and service provider disussions, so I’ll cross post a link to this talk topic there:
Ways in which distributions can be assessed:
Distributions can be assessed individually, taking into account their goals, unique features, trajectories and stated intents, etc. This may appear to be the most generous, inclusive type of assessment, but inherent in this is a huge amount of subjectivity. The results of such an assessment are easily subject to challenge. I believe this was the basis for the early CDC and WHO EMR assessment in Kenya.
Alternatively, distribution against a set of common criteria. This is “unfair” as the criteria cannot take into account individual strengths, but it does provide an “above the line” Measure of how well a distribution fit a common vision. The success of thisapproach lies in selecting criteria on which theere truly is consensus, such as security, deployability, active clinical use, and and on keeping those criteria fairly minimal and as objective as possible. This is the approach taken by the assessment which followed development of the Kenya Standards and Guidelines document, and the adoption of that document by the ministry.
Assessing distributions, through either method, can provide “guideposts” to help implementers and usually an understanding some Basic attributes of different distribution. Obviously, the goal of this is to improve trust in distributions that assess well and support implementers in choosing the distribution that best fits their needs.
However, it’s important to note that both types of assessments weaken the community of developers. Distribution switch feel unfairly judged by the more tailored assessment methods, or distributions that are “below the line” in a more objective system, have a tremendous incentive to fork in order to preserve their self assessed value. This is what played out in Kenya – the EMRs that felt unfairly judged by the first to assessments manage to block any use of those assessments and move the country forward. The standards and guidelines approach in Kenya managed to identify for EMR’s which could go forward on a national level, but the other already implemented EMR’s in Kenya essentially disappeared, and those developers and organizations no longer contributed to improving public sector health information management.
The value of weakening the developer community (better software, more focus, etc.), may be worth the cost (fewer participants, more political resistance), or it may not be. Tough call.