OpenMRS SDK 3.1.1. released!

gutkowski · August 5, 2016, 1:20pm

@ssmusoke we took a bit different approach - openmrs-distro.properties file is full description of initial server setup. As @raff said, when you change anything, you should change name/version as well, because you are creating new independent configuration.

Of course, something like inheritance/update system would be more efficient and comfortable, but at the moment it was out of scope of this release, I think first we need to see how people use distro properties feature, so all feedback is very valuable for us. All suggestions will certainly be considered to make upcoming versions of SDK better.

mseaton · August 5, 2016, 1:46pm

@raff, I am planning to test this out soon and I’ll certainly report back any missing features that I’d love to see. @mogoodrich has been my guinea pig and he seems thrilled with it so far, so I’ll likely make the switch regardless and work to optimize the SDK however needed.

Thanks to you and all others involved with this.

Mike

darius · August 6, 2016, 6:54am

I just created a new server, and I really like the wizard! Unfortunately it didn’t actually work for me. (More details later.)

First, a couple random comments:

There seem to be a lot of heavy dependencies (downloading over our TW office network in India), especially the Atlassian ones seem to be big.

I disagree with the “platform” options that are offered as shortcuts:

You can deploy the following versions of a platform:

2.1.0-SNAPSHOT

2.0.1-SNAPSHOT

2.0.0

1.12.1-SNAPSHOT

1.12.0-beta

1.11.7-SNAPSHOT

Other…

(1) I prefer that we only have shortcuts for released versions, not any snapshots (or at most we can show the very latest snapshot) (2) We should show the latest maintenance version of each release line (3) 1.12.0 should be shown, not 1.12.0-beta

So I suggest we show (assuming we stick with 6, plus Other):

2.1.0-SNAPSHOT
2.0.0
1.12.0
1.11.6
1.10.4
1.9.11

The workflow seems to be what I intended. Small suggestions is that in this workflow the dbUri default value could default to jdbc:mysql://{docker machine IP}:3306/@DBNAME@

Also, it failed at the end. Here’s the flow:

NAdjazayer:~ djazayer$ mvn openmrs-sdk:setup Java HotSpot™ 64-Bit Server VM warning: ignoring option MaxPermSize=1024m; support was removed in 8.0 [INFO] Scanning for projects… [INFO]
[INFO] ------------------------------------------------------------------------ [INFO] Building Maven Stub Project (No POM) 1 [INFO] ------------------------------------------------------------------------ [INFO] [INFO] — openmrs-sdk-maven-plugin:3.1.2:setup (default-cli) @ standalone-pom —

Would you be willing to help us improve SDK by sending us once in a while anonymous usage statistics (you can always change your mind by going to sdk-stats.properties and setting statsEnabled to false) [Y/n]:

Setting up a new server…

Specify server id (-DserverId) (default: ‘server’): cohortbuilder

You can setup the following servers:

Distribution

Platform

Which one do you choose? [1/2]: 2

You can deploy the following versions of a platform:

2.1.0-SNAPSHOT

2.0.1-SNAPSHOT

2.0.0

1.12.1-SNAPSHOT

1.12.0-beta

1.11.7-SNAPSHOT

Other…

Which one do you choose? [1/2/3/4/5/6/7]: 7

Please specify platform version: 1.12.0 [INFO] Configured Artifact: org.openmrs.web:openmrs-webapp:1.12.0:war [INFO] Copying openmrs-webapp-1.12.0.war to /Users/djazayer/openmrs/cohortbuilder/openmrs-1.12.0.war

Which database would you like to use?:

H2

MySQL 5.6 (requires pre-installed MySQL 5.6)

MySQL 5.6 in SDK docker container (requires pre-installed Docker)

Existing docker container (requires pre-installed Docker)

Which one do you choose? [1/2/3/4]: 4

Please specify Docker Machine URL (find out by running docker-machine url): tcp://192.168.99.100:2376

Please specify your container id/name/label (you can get it using command docker ps -a): openmrs-sdk-mysql

Please specify DB username: root

Please specify DB password: openmrs

Please specify database uri (-DdbUri) (default: ‘jdbc:mysql://localhost:3306/@DBNAME@’): jdbc:mysql://192.168.99.100:3306/openmrs

Starting ‘openmrs-sdk-mysql’ DB docker container… [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 03:23 min [INFO] Finished at: 2016-08-06T11:41:19+05:30 [INFO] Final Memory: 35M/280M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.openmrs.maven.plugins:openmrs-sdk-maven-plugin:3.1.2:setup (default-cli) on project standalone-pom: Unable to execute mojo: Failed to find the ‘openmrs-sdk-mysql’ container. Run docker ps to see available containers. → [Help 1]

Doing docker start openmrs-sdk-mysql at the command line works, so I know that container exists. (This is the container’s name. I haven’t tried specifying by container ID.)

Also, another bug related to this failure mode: at this point, the SDK has created a folder for my SDK and downloaded the OpenMRS WAR file into it, but nothing else, so the SDK doesn’t recognize it as a server, and I can’t delete it, or view its info, or anything:

NAdjazayer:~ djazayer$ mvn openmrs-sdk:delete -DserverId=cohortbuilder … [ERROR] Failed to execute goal org.openmrs.maven.plugins:openmrs-sdk-maven-plugin:3.1.2:delete (default-cli) on project standalone-pom: There is no server properties file in this directory → [Help 1]

I would expect that the properties file should have been written, since I provided all the info. Then I could try changing one property and running it again, in case I had just gotten the docker container name wrong or something.

darius · August 6, 2016, 7:50am

I tried the “MySQL 5.6 in SDK docker container (requires pre-installed Docker)” option and found another bug: it tries to use what I guess is a hardcoded docker container name, and it fails if there’s already a container with this name.

[ERROR] Failed to execute goal org.openmrs.maven.plugins:openmrs-sdk-maven-plugin:3.1.2:setup (default-cli) on project standalone-pom: Unable to execute mojo: Execution null of goal org.openmrs.maven.plugins:openmrs-sdk-docker-maven-plugin:3.1.2:create-mysql failed: Conflict. The name “/openmrs-sdk-mysql” is already in use by container 11dad082e3dddb23aaa301261506e02bab0754a507bc031083fdfd3d5b31c851. You have to remove (or rename) that container to be able to reuse that name. → [Help 1]

If this is only a problem that I happen to have a container with that name, then I can remove that, and it’s no big deal. But I’d like to make sure I can use this same option for two different SDK servers.

So, after I remove my preexisting docker container and try this again, it still doesn’t work:

… Creating ‘openmrs-sdk-mysql’ DB docker container…

Starting ‘openmrs-sdk-mysql’ DB docker container…

Trying to connect to the DB… [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 46.898 s [INFO] Finished at: 2016-08-06T13:24:43+05:30 [INFO] Final Memory: 36M/680M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.openmrs.maven.plugins:openmrs-sdk-maven-plugin:3.1.2:setup (default-cli) on project standalone-pom: Unable to execute mojo: Failed to connect to the DB in the ‘openmrs-sdk-mysql’ container at ‘jdbc:mysql://192.168.99.100:3307’ → [Help 1]

At this point nothing is running (according to docker ps) but I see it was created and failed. It’s not clear to me why this would fail. Did this work for you all during development? (I’m on a Mac with Docker version 1.11.1, build 5604cbe, if that matters.)

NAdjazayer:~ djazayer$ docker logs openmrs-sdk-mysql Initializing database 2016-08-06 07:47:45 0 [Note] /usr/sbin/mysqld (mysqld 5.6.28) starting as process 14 … 2016-08-06 07:47:45 14 [Note] InnoDB: Using atomics to ref count buffer pool pages 2016-08-06 07:47:45 14 [Note] InnoDB: The InnoDB memory heap is disabled 2016-08-06 07:47:45 14 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2016-08-06 07:47:45 14 [Note] InnoDB: Memory barrier is not used 2016-08-06 07:47:45 14 [Note] InnoDB: Compressed tables use zlib 1.2.8 2016-08-06 07:47:45 14 [Note] InnoDB: Using Linux native AIO 2016-08-06 07:47:45 14 [Note] InnoDB: Using CPU crc32 instructions 2016-08-06 07:47:45 14 [Note] InnoDB: Initializing buffer pool, size = 128.0M 2016-08-06 07:47:45 14 [Note] InnoDB: Completed initialization of buffer pool 2016-08-06 07:47:45 7f710fe45740 InnoDB: Operating system error number 13 in a file operation. InnoDB: The error means mysqld does not have the access rights to InnoDB: the directory. 2016-08-06 07:47:45 7f710fe45740 InnoDB: Operating system error number 13 in a file operation. InnoDB: The error means mysqld does not have the access rights to InnoDB: the directory. 2016-08-06 07:47:45 14 [ERROR] InnoDB: Creating or opening ./ibdata1 failed! 2016-08-06 07:47:45 14 [ERROR] InnoDB: Could not open or create the system tablespace. If you tried to add new data files to the system tablespace, and it failed here, you should now edit innodb_data_file_path in my.cnf back to what it was, and remove the new ibdata files InnoDB created in this failed attempt. InnoDB only wrote those files full of zeros, but did not yet use them in any way. But be careful: do not remove old data files which contain your precious data! 2016-08-06 07:47:45 14 [ERROR] Plugin ‘InnoDB’ init function returned error. 2016-08-06 07:47:45 14 [ERROR] Plugin ‘InnoDB’ registration as a STORAGE ENGINE failed. 2016-08-06 07:47:45 14 [ERROR] Unknown/unsupported storage engine: InnoDB 2016-08-06 07:47:45 14 [ERROR] Aborting

2016-08-06 07:47:45 14 [Note] Binlog end 2016-08-06 07:47:45 14 [Note] /usr/sbin/mysqld: Shutdown complete

darius · August 6, 2016, 8:32am

Note: if you’re only reading email notifications, I edited my previous comment to describe another bug, so you should look on Talk.

Another small issue is that it’s not possible to install the serialization.xstream module via openmrs-sdk:deploy, because it has an omod extension rather than a jar one. I’m not sure if it’s right to put a special case in the SDK code to deal with this non-standard module artifact or not, but this does prevent me from using the SDK to completely set up a platform server with the reporting module.

raff · August 8, 2016, 8:53am

Thanks Darius! We tested the feature on Linux and Windows (Docker Toolbox), but skipped Mac not having access to any. As expected there are differences between Docker implementation on those 3 platforms. As soon as we get our hands on a Mac will do debugging and try to address the issues.

Here are the issues for reference:

mogoodrich · August 10, 2016, 7:05pm

Unfortunately, I’m having some problems today with the SDK, and I can’t give the best error report because it’s not 100% clear to me what is going on. There have been a lot of module updates over the past few days, and the SDK isn’t pulling the right values.

I figure there should be a command that just assures that the “/modules” folder for a deployment contains all the modules listed in the distro, with the correct versions of each as listed in the config. Is there a way to do this? The SDK seems to be pulling in a lot of mistaken versions, and I think my only option right now is to kill this server and start from scratch.

Mark

mogoodrich · August 10, 2016, 7:32pm

Now I’m completely hosed… NPE when I try to set up a new environment from scratch. Might just be a typo or something else stupid on my part, but the error messages aren’t very helpful:

Mark

ssmusoke · August 10, 2016, 7:47pm

@mogoodrich Not an expert on SDK but I reproduced your error, how come your distro is -Ddistro=org.openmrs.module:mirebalais-api:1.2-SNAPSHOT? Thought Distros would be like Ref App etc

gutkowski · August 10, 2016, 8:04pm

@mogoodrich I have found the cause of error you encountered and fix prepared, I believe we will repair it ASAP. About wrong versions, without any further details I can only recommend you to use SDK with ‘-U’ flag to force maven to check all dependencies in remote repsitories.

@ssmusoke this is be perfectly valid distro, as this jar contains openmrs-distro.properties file.

ssmusoke · August 10, 2016, 8:06pm

@gutkowski Thanks for the clarification on what a valid distro can be

mogoodrich · August 10, 2016, 8:20pm

Thanks @gutkowski. I tried to isolate things a bit more. Here’s an example of something that I assumed would work, but doesn’t appear to be.

Today we updated the version of the appointment scheduling module used from version 1.5 to 1.6 (this actually happened automatically when a new version of appointment scheduling was released).

You can see that the pom in the Mirebalais module has been updated:

github.com

PIH/openmrs-module-mirebalais/blob/master/pom.xml#L95


 <appuiVersion>1.8.0</appuiVersion>
 <dispensingVersion>1.0</dispensingVersion>
 <uicommonsVersion>2.7.0-SNAPSHOT</uicommonsVersion>
 <registrationcoreVersion>1.8.0</registrationcoreVersion>
 <registrationappVersion>1.12.0</registrationappVersion>
 <radiologyappVersion>1.4.0</radiologyappVersion>
 <coreappsVersion>1.20.0-SNAPSHOT</coreappsVersion>
 <webservicesRestVersion>2.23.0-SNAPSHOT</webservicesRestVersion>
 <appointmentschedulingVersion>1.9.0</appointmentschedulingVersion>
 <appointmentschedulinguiVersion>1.7.0</appointmentschedulinguiVersion>
 <testutilsVersion>1.6.0</testutilsVersion>
 <printerVersion>1.4.0</printerVersion>
 <allergyuiVersion>1.8.1</allergyuiVersion>
 <labtrackingappVersion>1.1.0-SNAPSHOT</labtrackingappVersion>
 <haiticoreVersion>1.0.0</haiticoreVersion>
 <attachmentsVersion>2.0.0</attachmentsVersion>
 <legacyuiVersion>1.3.3</legacyuiVersion>
 <owaVersion>1.9.0</owaVersion>
 <orderentryuiVersion>1.0</orderentryuiVersion>
 <cohortBuilderOwaVersion>1.0.0</cohortBuilderOwaVersion>
<!-- <chartsearchVersion>2.0</chartsearchVersion>-->

And the distro builds from that pom:

github.com

PIH/openmrs-module-mirebalais/blob/master/api/src/main/resources/openmrs-distro.properties#L33


omod.htmlformentryui=${htmlformentryuiVersion}
omod.appui=${appuiVersion}
omod.event=${eventVersion}
omod.event.groupId=org.openmrs
omod.coreapps=${coreappsVersion}
omod.webservices.rest=${webservicesRestVersion}
omod.reportingrest=${reportingrestVersion}
omod.allergyui=${allergyuiVersion}
omod.appointmentscheduling=${appointmentschedulingVersion}
omod.appointmentschedulingui=${appointmentschedulinguiVersion}
omod.mirebalaismetadata=${mirebalaismetadataVersion}
omod.mirebalaisreports=${mirebalaisreportsVersion}
omod.edtriageapp=${edtriageappVersion}
omod.labtrackingapp=${labtrackingappVersion}
omod.paperrecord=${paperrecordVersion}
omod.emr=${emrVersion}
omod.namephonetics=${namephoneticsVersion}
omod.pacsintegration=${pacsintegrationVersion}
omod.addresshierarchy=${addresshierarchyVersion}
omod.importpatientfromws=${importpatientfromwsVersion}
omod.dispensing=${dispensingVersion}

I’ve deployed the Mirebalais module to the OpenMRS maven snapshot repo and the latest version appears to be there with a recent last updated date:

http://mavenrepo.openmrs.org/nexus/service/local/repositories/snapshots/content/org/openmrs/module/mirebalais/1.2-SNAPSHOT/maven-metadata.xml

Note that I do not currently have appointment scheduling checked out and therefore am not watching it.

Still, I would have assumed that doing a “mvn openmrs-sdk:deploy -DserverId=mirebalais” would result in the version of the appointment scheduling module in the mirebalais/modules folder being updated from 1.5 to 1.6. However, this didn’t happen. The -U seems liked a good guess, but I tried that and still no luck.

Am I doing something wrong, or is this a bug?

Mark

mogoodrich · August 10, 2016, 8:51pm

Hmm… I might have found some helpful debug messages for the above error. I’ve created a ticket and posted the error message there:

Thanks!

Mark

gutkowski · August 11, 2016, 5:59am

I see, in this case problem is that property placeholders are resolved during packaging by maven, so if you used distro file as it is, with module={moduleVersion}, SDK looks for artifact with version {moduleVersion}, and of course is not able to find it. That’s a shame we didn’t foresee this usecase, resolving placeholders would be really useful and easy to implement.

I’ve created ticket https://issues.openmrs.org/browse/SDK-146, and edited 144 a bit https://issues.openmrs.org/browse/SDK-144

raff · August 11, 2016, 10:37am

Thanks @gutkowski and @adamg for addressing Mark’s issues so quickly.

A new release 3.1.3 with fixes should go public within a few hours.

@mogoodrich, thanks for helping us improve the SDK!

BTW, we have a bug fixing sprint / swim lane for addressing issues in SDK as fast as possible. The board is here https://issues.openmrs.org/secure/RapidBoard.jspa?rapidView=95&view=detail. If you see your issue there it means it’s on our priority list, if not please remind us as we may have lost track of it.

mogoodrich · August 11, 2016, 1:27pm

Cool, thanks!

Not resolving property placeholders would have been one of my first guesses… but, strangely, I was able to use openmrs-sdk:setup to setup a new server using this distro file and it worked fine… so maybe in some cases it already is resolving placeholders?

Thanks for being so responsive on this! I want to keep with the SDK, because it will be a valuable tool, but these would be blockers…

Thanks, Mark

mogoodrich · August 12, 2016, 7:51pm

Been having problems today with watching modules. It’ s a bit of a blocker, not sure exactly what is going on, but this is what I’m seeing:

Mark

ssmusoke · August 13, 2016, 7:47am

What do I do in case I want to remove a module from openmrs-distro.properties, in this case referenceappdemodata?

What I have found is the following:

When I move the module from my own distro.properties file then run mvn openmrs-sdk:deploy -DserverId=serverId -Ddistro=/path/to/file then the oprnmrs-distro.properties file is updated with the new modules
Then everything now works with the new module versions

Is this the expected behavior?

raff · August 16, 2016, 9:50am

@ssmusoke, to remove a module just call mvn openmrs-sdk:undeploy or create a distro file without that module and deploy it like you did.

@mogoodrich, I see you cancelled the issue, thanks.

mogoodrich · August 16, 2016, 2:25pm

@raff Do you think it would make sense to have -Ddebug parameter default to be “true” when doing a openmrs-sdk:run. I keep forgetting it (was partially what led me to log the non-bug SDK-150) and I figure since this is an SDK, in most cases the user would want debug functionality.

Thanks! Mark