Why the SDK can not find this plugin

When I’m not using the root user to run the openmrs-sdk or mysql to connect or login the MySQL server, I failed, and report this ERROR 1698. So, the folowing words is my solution:

When I using MySQL 5.7, the root user’s plugin is not mysql_native_password, so we need to change the plugin.

Run these commands:

mysql> USE mysql;
mysql> UPDATE user SET plugin='mysql_native_password' WHERE User='root';
mysql> FLUSH PRIVILEGES;
mysql> exit;

The original solution link is here

1 Like

Ir might be better to do an INSERT rather than UPDATE so you can continue using the socket as root (socket connections are MySQL’s preferred mode of connection, it’s just hard to do in a platform-agnostic way from Java).

2 Likes

What version of Docker are y’all running? I have a colleague in Mexico, Jose Ramon (I don’t think he has a Talk account yet) who is running into this issue. He’s running Docker Engine 20.10.9. I’m running Docker Engine 19.03.3 and have no problems. This seems to me like it might be something that Docker broke for us in the 19-20 upgrade. @ibacher , @mseaton , @d404d , @navareth ?

I’m running Docker Engine 20.10.6.

Yeah, I can reproduce this with Docker Engine 20.10.6 on Linux.

1 Like

This issue should be fixed in SDK v4.0.1, which I just released to the OpenMRS Maven Repo. It should be on Central shortly. Please let me know if this doesn’t work for you.

3 Likes

Things are working for Jose Ramon. Thank you so much @ibacher !

1 Like

Hey @ibacher , bad news… someone’s getting that error again with SDK 4.0.1 :frowning:

I got this from my colleague Miguel (no Talk account yet) in Peru:

Failed to execute goal org.openmrs.maven.plugins:openmrs-sdk-maven-plugin:4.0.1:setup (default-cli) on project standalone-pom: Failed to setup server: Unable to execute mojo: Execution null of goal org.openmrs.maven.plugins:openmrs-sdk-docker-maven-plugin:4.0.1:run-db failed: Status 500: OCI runtime create failed: invalid mount {Destination:openmrs-sdk-mysql-v3-2-data:/var/lib/mysql Type:bind Source:/var/lib/docker/volumes/58507bd05553afb8b464a36575921e6c49c59d45f47d57ad25222795ec36feb6/_data Options:[rbind]}: mount destination openmrs-sdk-mysql-v3-2-data:/var/lib/mysql not absolute: unknown

However, for him this is taking place right after an incident where his Ubuntu went into emergency mode, and when he had recovered from it all his docker containers were gone. So there could be some other weirdness going on (something wrong with his docker installation, most likely).

Hmmmm… that’s troubling. Do you know what version of the Docker Engine Miguel is running?

Oop, he reinstalled Docker and everything is fine. Sorry to trouble you.

1 Like

No, that’s fine! I’m glad it was sorted by a reinstall though. Saves me having to dig through web requests to be sure we’re doing the right thing.

Double oops: @ball is now getting this error, also with SDK 4.0.1, Docker version 20.10.2, API version 1.41.

How can we facilitate the investigation of this?

[ERROR] Failed to execute goal org.openmrs.maven.plugins:openmrs-sdk-maven-plugin:4.0.1:run-tomcat (default-cli) on project standalone-pom: Unable to execute mojo: Execution null of goal org.openmrs.maven.plugins:openmrs-sdk-docker-maven-plugin:4.0.1:run-db failed: Status 500: OCI runtime create failed: invalid mount {Destination:openmrs-sdk-mysql-v3-2-data:/var/lib/mysql Type:bind Source:/var/lib/docker/volumes/2a49060c829510be289e527317fb3c0ae59f6636e927c967429478561111f1e2/_data Options:[rbind]}: mount destination openmrs-sdk-mysql-v3-2-data:/var/lib/mysql not absolute: unknown → [Help 1]

@ball can you confirm that you didn’t experience e.g. a big data corruption event with Ubuntu entering emergency mode over the weekend? Any other relevant events?

I rebooted over the weekend (which hasn’t happened in awhile :wink: Running Ubuntu 20.4.2

Let’s first try to work out if this issue is caused by 4.0.1 or by something else. @ball does mvn org.openmrs.maven.plugins:openmrs-sdk-maven-plugin:3.13.9:setup work? (This should force using 3.13.9 version).

My current hypothesis is that deleting this line of code wasn’t the right idea. It wasn’t necessary on either my Mac or my Linux VM, but perhaps Ubuntu / Debian’s version of Docker doesn’t automatically create a volume specified in a create container operation?

Nope. I tried to setup with my existing container with 3.13.9 and 4.0.1 but both fail with the existing sql container.

Meet me on slack if you have immediate ideas or I will delete the container and start again. It’s rough because I have a few dev SDKs for various implementations :frowning:

Just to document the information Ellen was kind enough to provide to me, it looks like (somehow) the Docker image for SDK MySQL server got messed up. When she ran docker inspect openmrs-sdk-mysql-v3-2, it reported two mounts:

"Mounts": [
    {
        "Type": "volume",
        "Name": "7dab10d28a0b6b84a934b550f3565b82987b4d0b6a28e132a181f8409be4c771",
        "Source": "/var/lib/docker/volumes/7dab10d28a0b6b84a934b550f3565b82987b4d0b6a28e132a181f8409be4c771/_data",
        "Destination": "/var/lib/mysql",
        "Driver": "local",
        "Mode": "",
        "RW": true,
        "Propagation": ""
    },
    {
        "Type": "volume",
        "Name": "2a49060c829510be289e527317fb3c0ae59f6636e927c967429478561111f1e2",
        "Source": "/var/lib/docker/volumes/2a49060c829510be289e527317fb3c0ae59f6636e927c967429478561111f1e2/_data",
        "Destination": "openmrs-sdk-mysql-v3-2-data:/var/lib/mysql",
        "Driver": "local",
        "Mode": "",
        "RW": true,
        "Propagation": ""
    }
],

One of those (the first) is a correctly created anonymous mount for /var/lib/mysql (created because the image lists that directory as a volume). The other of which is a somehow messed-up version of the named volume the SDK tries to create (openmrs-sdk-mysql-v3-2-data) instead mounted as yet another anonymous volume. Since openmrs-sdk-mysql-v3-2-data:/var/lib/mysql is clearly not a valid file path, we get the error Ellen saw.

We were able to get Ellen running again by removing the existing SDK container (but not the volumes) using docker container rm openmrs-sdk-mysql-v3-2 and then creating a new container that used the “good” volume using this command:

docker run --name openmrs-sdk-mysql-v3-2 -e MYSQL_ROOT_PASSWORD=Admin123 -e MYSQL_USER=openmrs -e MYSQL_PASSWORD=Admin123 -p 3308:3306 --mount type=volume,source=7dab10d28a0b6b84a934b550f3565b82987b4d0b6a28e132a181f8409be4c771,destination=/var/lib/mysql --restart unless-stopped -d mysql:5.6 --character-set-server=utf8 --collation-server=utf8_unicode_ci

The important point of which is the --mount type=volume,source=<identifier for volume we want to keep>,destination=/var/lib/mysql which mounts the volume from the previously working container to the /var/lib/mysql directory of the new MySQL container.

1 Like

Given @bistenes’s two reports of issues, my best guess is that there’s some flaw in upgrading to Docker 20.10 with existing containers that introduces this issue (this is possibly also why Miguel was having so many other Docker-related issues). Obviously, the easiest fix when this issue comes up is to blow away the existing Docker container and let the SDK create a fresh one, but if, like Ellen, you can’t do that or don’t want to due to data loss issues, hopefully my above post can help you work out how to recover from this issue.

Huge thanks to @ibacher for this and everything else he has done for the past few years. Remarkable contributions.

Ian is worthy of a lifetime OpenMRS Ninja award. He can scale the walls with the greats. (I’m thinking of you, @dkayiwa )

@ball, he climbed to /dev/5 in the shortest time ever recorded in the community! :grinning_face_with_smiling_eyes:

2 Likes

IIRC, I was thinking about writing a post to ask to be a /dev/2 when you bumped me up to /dev/4 :grinning:.

1 Like