Why the SDK can not find this plugin

No, that’s fine! I’m glad it was sorted by a reinstall though. Saves me having to dig through web requests to be sure we’re doing the right thing.

Double oops: @ball is now getting this error, also with SDK 4.0.1, Docker version 20.10.2, API version 1.41.

How can we facilitate the investigation of this?

[ERROR] Failed to execute goal org.openmrs.maven.plugins:openmrs-sdk-maven-plugin:4.0.1:run-tomcat (default-cli) on project standalone-pom: Unable to execute mojo: Execution null of goal org.openmrs.maven.plugins:openmrs-sdk-docker-maven-plugin:4.0.1:run-db failed: Status 500: OCI runtime create failed: invalid mount {Destination:openmrs-sdk-mysql-v3-2-data:/var/lib/mysql Type:bind Source:/var/lib/docker/volumes/2a49060c829510be289e527317fb3c0ae59f6636e927c967429478561111f1e2/_data Options:[rbind]}: mount destination openmrs-sdk-mysql-v3-2-data:/var/lib/mysql not absolute: unknown → [Help 1]

@ball can you confirm that you didn’t experience e.g. a big data corruption event with Ubuntu entering emergency mode over the weekend? Any other relevant events?

I rebooted over the weekend (which hasn’t happened in awhile :wink: Running Ubuntu 20.4.2

Let’s first try to work out if this issue is caused by 4.0.1 or by something else. @ball does mvn org.openmrs.maven.plugins:openmrs-sdk-maven-plugin:3.13.9:setup work? (This should force using 3.13.9 version).

My current hypothesis is that deleting this line of code wasn’t the right idea. It wasn’t necessary on either my Mac or my Linux VM, but perhaps Ubuntu / Debian’s version of Docker doesn’t automatically create a volume specified in a create container operation?

Nope. I tried to setup with my existing container with 3.13.9 and 4.0.1 but both fail with the existing sql container.

Meet me on slack if you have immediate ideas or I will delete the container and start again. It’s rough because I have a few dev SDKs for various implementations :frowning:

Just to document the information Ellen was kind enough to provide to me, it looks like (somehow) the Docker image for SDK MySQL server got messed up. When she ran docker inspect openmrs-sdk-mysql-v3-2, it reported two mounts:

"Mounts": [
    {
        "Type": "volume",
        "Name": "7dab10d28a0b6b84a934b550f3565b82987b4d0b6a28e132a181f8409be4c771",
        "Source": "/var/lib/docker/volumes/7dab10d28a0b6b84a934b550f3565b82987b4d0b6a28e132a181f8409be4c771/_data",
        "Destination": "/var/lib/mysql",
        "Driver": "local",
        "Mode": "",
        "RW": true,
        "Propagation": ""
    },
    {
        "Type": "volume",
        "Name": "2a49060c829510be289e527317fb3c0ae59f6636e927c967429478561111f1e2",
        "Source": "/var/lib/docker/volumes/2a49060c829510be289e527317fb3c0ae59f6636e927c967429478561111f1e2/_data",
        "Destination": "openmrs-sdk-mysql-v3-2-data:/var/lib/mysql",
        "Driver": "local",
        "Mode": "",
        "RW": true,
        "Propagation": ""
    }
],

One of those (the first) is a correctly created anonymous mount for /var/lib/mysql (created because the image lists that directory as a volume). The other of which is a somehow messed-up version of the named volume the SDK tries to create (openmrs-sdk-mysql-v3-2-data) instead mounted as yet another anonymous volume. Since openmrs-sdk-mysql-v3-2-data:/var/lib/mysql is clearly not a valid file path, we get the error Ellen saw.

We were able to get Ellen running again by removing the existing SDK container (but not the volumes) using docker container rm openmrs-sdk-mysql-v3-2 and then creating a new container that used the “good” volume using this command:

docker run --name openmrs-sdk-mysql-v3-2 -e MYSQL_ROOT_PASSWORD=Admin123 -e MYSQL_USER=openmrs -e MYSQL_PASSWORD=Admin123 -p 3308:3306 --mount type=volume,source=7dab10d28a0b6b84a934b550f3565b82987b4d0b6a28e132a181f8409be4c771,destination=/var/lib/mysql --restart unless-stopped -d mysql:5.6 --character-set-server=utf8 --collation-server=utf8_unicode_ci

The important point of which is the --mount type=volume,source=<identifier for volume we want to keep>,destination=/var/lib/mysql which mounts the volume from the previously working container to the /var/lib/mysql directory of the new MySQL container.

1 Like

Given @bistenes’s two reports of issues, my best guess is that there’s some flaw in upgrading to Docker 20.10 with existing containers that introduces this issue (this is possibly also why Miguel was having so many other Docker-related issues). Obviously, the easiest fix when this issue comes up is to blow away the existing Docker container and let the SDK create a fresh one, but if, like Ellen, you can’t do that or don’t want to due to data loss issues, hopefully my above post can help you work out how to recover from this issue.

Huge thanks to @ibacher for this and everything else he has done for the past few years. Remarkable contributions.

Ian is worthy of a lifetime OpenMRS Ninja award. He can scale the walls with the greats. (I’m thinking of you, @dkayiwa )

@ball, he climbed to /dev/5 in the shortest time ever recorded in the community! :grinning_face_with_smiling_eyes:

2 Likes

IIRC, I was thinking about writing a post to ask to be a /dev/2 when you bumped me up to /dev/4 :grinning:.

1 Like