Our wonderful member & 2022 GSOC contributor @anjisvj is blocked right now from incorporating his work into dev3 (it’s all stuck on his local). He is refactoring the Cohort Module for O3. He can’t verify that everything is working as expected because the dev3 server won’t start both the Reporting and Reporting REST modules. @ibacher has been meaning to look into this but doesn’t have time.
@raff might you be able to help? Can we get dev3 server running w/ reporting + reporting rest module? Otherwise the dev3 server works fine and both modules are theoretically there, but they won’t start.
@slubwama shared that this they are working in UgEMR (which is on RefApp 2.x modules) so I assume this is something specific to O3/dev3.
@alaboso & @amugume I’m curious, did you run into any issues like this in your OHRI work?
There seems to be a problem with DB on dev3.openmrs.org. Reporting DB migrations cannot complete. Is there data that needs to be preserved on dev3 or can we wipe it and start fresh?
I’m taking a backup just in case and wiping out the DB to start fresh and see if it helps.
Wiping out data did not help. There’s some bad combination of modules installed that cannot be deployed. Can dev3.openmrs.org be updated to the latest O3 or does it need to run any specific version?
@grace this is exactly one of the reasons why i personally want to have a local backend server for 03. That way, you do not get blocked whenever the online server has issues. But i have struggled, in vain, to have a combination of esm versions that work together when trying to set up my local 03 backend server. Is there any where that i should look? @slubwama seems to be under the same struggle for UgandaEMR.
Found it… the issue was something @alaboso actually fixed here but it hasn’t been released yet, so the latest released version of the reporting module didn’t have the necessary changes.
It can always be updated to the latest of everything.
The intention is the dev3 is completely wipeable, though I don’t think the deployment stuff currently wipes it by default.
Well, you’ll still get blocked if the issue is “how do I get this thing that works locally to run on the integrated environment” (@anjisvj has been working locally).
@anjisvj The modules are at least running on dev3 now.
There’s still some import errors when starting up for the first time, which are related to missing concepts, but things seem to work regardless. I’ll try to address the errors.
So the idea of dev3 is that it’s an integrated development environment that serves the latest content committed to any of the repos master or main branch. The CI scripts for all of our ESM repos are setup to publish those changes directly to a S3 bucket on Digital Ocean kindly provided for the community by AMPATH, so in the dev3 environment, we’re serving the files from the Digital Ocean CDN directly. Basically, the Digital Ocean CDN provides all the components that would be in the frontend version.
The process for publishing stuff to DO pre-existed the dev3 environment so it seemed easiest just to take advantage of the existing build process rather than figure out how to automatically apply all updates to the 3.x branch of the distro.
Got you. We need a build process that can be replicated locally and on CI. How one would go about deploying locally what is on dev3? Can we have spa-build-config that is following up on updates and not locked to exact versions?
In theory (and what we used to do previously) it was tied to the next tag from npmjs. However, this meant that builds weren’t repeatable (since the next tag will simply point to whatever the latest version published on npmjs is, making the build dependent on the state of npmjs.com at the time the build was run). It also made it a pain to figure out which versions a given Docker image contained.
We’d also need to set things up so that each ESM repo triggered a Bamboo build on pre-release. Honestly, my strategy here was going to be to have spa-build-config.json tied to the latest released version and build the rest of the infrastructure on top of that (releases are less frequent than commits).
I actually don’t know that exactly replicated dev3 locally is a desirable feature… (dev3 can be unstable).
There’s also a timeliness issue: building the dev3 stack and releasing it takes ~20 minutes on top of the ~20 minutes it takes for the repo to run through the full release process meaning changes wouldn’t be available until ~40 minutes after commit. With the current setup, we have this down to around ~20 minutes, which isn’t great, but is closer to the desirable state.
@ibacher, we need to be able to replicate any environment locally (especially the latest dev version). The problem is that it is sometimes hard to debug a remote environment or it becomes unavailable for any reason, which blocks development.
The solution for npm is to use pre-release tags e.g. 1.2.2-snapshot.1, 1.2.2-snapshot.2, etc. as nicely laid out at Workaround for snapshot versions in Npm - Stack Overflow. It’s quite simple and works with npm update. If we have that in place dev3 could be deployed full-stack in the time it takes to complete the distro build i.e. ~10 minutes. It can be further sped up by enabling caching on CI with docker buildkit, which I plan to do next.
Following up on pre-release tags. I see that we actually publish pre-release tags on every commit already. It’s with the -pre.#’ suffix. Is that right? If that’s the case then we just need to have an explicit step in distro build to call npm update.
There are a few limitations to using npm update, but the most important is that npm update works by updating the package-lock.json file and the node_modules, but neither or those entities are involved in our current frontend build process.
I talked with @zacbutko and I think we came up with a decent strategy where we’ll modify the openmrs assemble command to emit a sort of manifest file with all the resolved versions. This will allow us to switch things to just using the next tag (in spa-build-config.json) but still resolve things to unambiguous versions (so that it will be clear what’s actually running in the environment).
I was assuming our openmrs assemble is a thin wrapper and calls npm to fetch and manage versions. I need to look more into that tool to have better understanding. Is it using npm local cache or fetching items each time from npmjs.com?
FWIW, i have just run the docker compose build and docker compose up commands for this branch: GitHub - openmrs/openmrs-distro-referenceapplication at 3.x and set up a local version of 03. But it still does not behave like dev3 in the sense that searching for a patient results into a Loading.. message that stays on forever, and the Dev Tools popup window is a black screen which does not load any esms. So, what is so special with Dev3?
