Environment variable to Docker container not works - docker

in /app folder, I have a Google Cloud credential file ocviam-dev-201b2db30d36.json
> docker exec -ti 9ac6de5f24586320ac41c6e2de9895f03ca87874745257642214852e755a4a99 bash
root#9ac6de5f2458:/# cd /app/
root#9ac6de5f2458:/app# ll
total 252
drwxr-xr-x 2 root root 4096 Apr 20 13:42 ./
drwxr-xr-x 1 root root 4096 Apr 20 13:44 ../
-rwxr-xr-x 1 root root 10875 Apr 20 12:29 function.R*
-rwxr-xr-x 1 root root 284 Apr 20 09:40 gfs_data_temp12042022.csv*
-rwxr-xr-x 1 root root 183311 Apr 12 07:33 gfs_data_temp_FULL.csv*
-rwxr-xr-x 1 root root 2331 Mar 22 15:49 ocviam-dev-201b2db30d36.json*
-rwxr-xr-x 1 root root 3355 Apr 20 12:34 plumber.R*
-rwxr-xr-x 1 root root 33467 Apr 13 09:39 .RData*
-rwxr-xr-x 1 root root 405 Apr 13 09:39 .Rhistory*
But when I run docker run -e option with GOOGLE_APPLICATION_CREDENTIALS variable and /app/ocviam-dev-201b2db30d36.json value, I get an error:
docker run --rm -it -p 80:8000 app1 -e GOOGLE_APPLICATION_CREDENTIALS=ocviam-dev-201b2db30d36.json
>>
R version 4.1.3 (2022-03-10) -- "One Push-Up"
Copyright (C) 2022 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> pr <- plumber::plumb(rev(commandArgs())[1]); args <- list(host = '0.0.0.0', port = 8000); if (packageVersion('plumber') >= '1.0.0') { pr$setDocs(TRUE) } else { args$swagger <- TRUE }; do.call(pr$run, args)
Error in plumber::plumb(rev(commandArgs())[1]) :
File does not exist: GOOGLE_APPLICATION_CREDENTIALS=ocviam-dev-201b2db30d36.json
Execution halted
(base) PS C:\Chantiers\repos\gcp-gfs-weather> docker run --rm -it -p 80:8000 app1 -e GOOGLE_APPLICATION_CREDENTIALS=/app/ocviam-dev-201b2db30d36.json
>>
R version 4.1.3 (2022-03-10) -- "One Push-Up"
Copyright (C) 2022 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
> pr <- plumber::plumb(rev(commandArgs())[1]); args <- list(host = '0.0.0.0', port = 8000); if (packageVersion('plumber') >= '1.0.0') { pr$setDocs(TRUE) } else { args$swagger <- TRUE }; do.call(pr$run, args)
Error in plumber::plumb(rev(commandArgs())[1]) :
File does not exist: GOOGLE_APPLICATION_CREDENTIALS=/app/ocviam-dev-201b2db30d36.json
Execution halted
Is it a problem of syntax?
Thanks.

The -e option in docker run commands is for passing environment variables as -e MYVAR1 or --env MYVAR2=foo. If you need to pass a file which has the environment variables stored in them, create a file (say "env.list") with contents like this:
Variable1=Value1
Variable2=Value2
Post which you can run docker run --env-file env.list which will set all the environment variables mentioned in the file.
If your original intention was to authenticate to your Google Cloud Project using the JSON, you should do so using gcloud auth:
gcloud auth activate-service-account [ACCOUNT] --key-file=KEY_FILE --project=PROJECT_NAME

Related

Tomcat 9 running on docker - Cannot find /usr/local/tomcat/bin/setclasspath.sh

I'm running a tomcat (tomcat:9-jre11) on docker, when launching it, it logs the following, then crashes :
Cannot find /usr/local/tomcat/bin/setclasspath.sh
This file is needed to run this program
My first issue was actually getting inside the container because I can't use docker exec on a crashed container, but I managed it by setting an entry point as /bin/bash in Rancher.
Now setclasspath.sh is very much in the /usr/local/tomcat/bin/ inside the container. It previously had all read and execution rights, I've set it to 777 just to be sure, still have the same issue. Same goes with changing the owner (tomcat seems to be using root, even if I launch the catalina.sh manually on another user, having changed the file owner). I used the heavy handed approch and set the whole damn folder as 777, and still the same :
drwxrwxrwx 1 root root 4096 Jun 29 14:53 .
drwxr-xr-x 1 root root 4096 Jun 29 14:31 ..
-rwxrwxrwx 1 root root 34699 Jun 2 21:08 bootstrap.jar
-rwxrwxrwx 1 root root 25523 Jun 29 14:00 catalina.sh
-rwxrwxrwx 1 root root 1664 Jun 2 21:08 catalina-tasks.xml
-rwxrwxrwx 1 root root 2007 Jun 28 03:01 ciphers.sh
-rwxrwxrwx 1 root root 25410 Jun 2 21:08 commons-daemon.jar
-rwxrwxrwx 1 root root 211777 Jun 2 21:08 commons-daemon-native.tar.gz
-rwxrwxrwx 1 root root 1932 Jun 28 03:01 configtest.sh
-rwxrwxrwx 1 root root 9110 Jun 28 03:01 daemon.sh
-rwxrwxrwx 1 root root 1975 Jun 28 03:01 digest.sh
-rwxrwxrwx 1 root root 3392 Jun 28 03:01 makebase.sh
-rwxrwxrwx 1 root root 3718 Jun 28 03:01 setclasspath.sh
-rwxrwxrwx 1 root root 1912 Jun 28 03:01 shutdown.sh
-rwxrwxrwx 1 root root 1914 Jun 28 03:01 startup.sh
-rwxrwxrwx 1 root root 46898 Jun 2 21:08 tomcat-juli.jar
-rwxrwxrwx 1 root root 5550 Jun 28 03:01 tool-wrapper.sh
-rwxrwxrwx 1 root root 1918 Jun 28 03:01 version.sh
I've looked at the catalina.sh script, the part which cause the issue is the following :
if [ -r "$CATALINA_HOME"/bin/setclasspath.sh ]; then
. "$CATALINA_HOME"/bin/setclasspath.sh
else
echo "Cannot find $CATALINA_HOME/bin/setclasspath.sh"
echo "This file is needed to run this program"
fi
The -r inside the condition is borked. I've read it looked if the file exists and is readable, it fill all conditions. I've added elif with -a and -f condition and the do return true, but the rights seems to be the issue despite them being set to 777 or not. I've add a whoami inside the script as well, and it's the root user, so not an issue of ownership.
The startup.sh script has a similar issue, with a -x condition, where it cannot find the catalina.sh ...
We just stumbled over this very problem today.
We have an Ubuntu 18.04 server that was upgraded from 16.04. The versions of the docker packages read:
docker-ce/now 5:19.03.1~3-0~ubuntu-xenial amd64
docker-ce-cli/now 5:19.03.1~3-0~ubuntu-xenial amd64
docker-compose/bionic,bionic,now 1.17.1-2 all
Kernel is: 4.15.0-154-generic x86_64
On this machine, running a current version of tomcat:9-jre11 [0] results in the same problem as depicted in your question.
To narrow it down, we just started a bash like this:
docker run -it --rm --entrypoint=/bin/bash tomcat:9-jre11
Now here comes the strange behavior you observed, which is completely unrelated to tomcat:
root#f338debf92f6:/usr/local/tomcat# [[ -r /bin/bash ]]
root#f338debf92f6:/usr/local/tomcat# echo $?
1
On any other machine we tested, the result is as expected, e.g.:
root#0083a80a9ec2:/usr/local/tomcat# [[ -r /bin/bash ]]
root#0083a80a9ec2:/usr/local/tomcat# echo $?
0
Unfortunately I was not able to reproduce the behavior using a freshly installed Ubuntu 18.04. I even downgraded the kernel version and installed docker from the xenial repo.
Trying to google a solution I found:
https://github.com/alpinelinux/docker-alpine/issues/156#issuecomment-912645029
So I tried strace, and here the problem is visible:
On our Ubuntu 18.04:
...
read(255, "#!/bin/bash\n[[ -r /bin/bash ]]\n", 31) = 31
faccessat2(AT_FDCWD, "/bin/bash", R_OK, AT_EACCESS) = -1 EPERM (Operation not permitted)
read(255, "", 31) = 0
...
And on any other machine I tested:
...
read(255, "#!/bin/bash\n[[ -r /bin/bash ]]\n", 31) = 31
faccessat2(AT_FDCWD, "/bin/bash", R_OK, AT_EACCESS) = -1 ENOSYS (Function not implemented)
faccessat(AT_FDCWD, "/bin/bash", R_OK) = 0
read(255, "", 31)
...
Researching the faccessat2 system call shows that it should not return EPERM [1]. I could not quite pinpoint where this behavior is introduced - somewhere between glibc and seccomp, but it all boils down to the runtime being too old for this new syscall.
Here are the solutions we came up with:
Upgrade your machine - this might not be feasible, though :)
Use a tomcat image based on an older version of Debian/Ubuntu. For us
tomcat:9.0.64-jre11-openjdk-slim-bullseye worked fine.
Run the container using the --privileged switch. This circumvents the syscall privilege problem, but would be generally a bad idea
References
digest sha256:f0c2eb420166a7d609c0031699e0778e11256f280cc2bfb5bfd61cde7ae45c61
https://man7.org/linux/man-pages/man2/faccessat.2.html
The Problem is descriped here:
https://github.com/docker-library/tomcat/issues/269
The Base Image (Eclipse Temurin) from the Tomcat Container was updated to
Ubuntu LTS 22.04 Jammy based Temurin image.
If you use an old Docker Version and libseccomp on your Host you will run into the Problem with the "-r" Flag in bash.
Our Solution was to use the Tomcat tomcat:9-jdk11-temurin-focal
Updating Docker to latest version helped me to launch tomcat
I had the same problem running a tomcat:9-jdk8 image, running on a debian 10.3 VM that was no more up-to-date.
Upgrading the whole system by
sudo apt-get update
sudo apt upgrade
-> reboot VM
solved the problem. Actual versions now: docker-client: 20.10.17, docker engine: 19.03.9, kernel: 4.19.0-21-amd64
Interestigly: The problem only occured when running the image that was built on this outdated system. The 'same' tomcat image built on our jenkins server started without problems on my local outdated VM.

Error Running TensorFlow Serving from Dockerfile

I am working on a container to run TensorFlow Serving.
Here is my Dockerfile:
FROM tensorflow/serving:latest
WORKDIR /
COPY models.config /models/models.config
COPY models/mnist /models/mnist
Here is my models.config for a simple mnist model:
model_config_list {
config: {
name: "mnist",
base_path: "/models/mnist"
model_platform: "tensorflow"
model_version_policy {
specific {
versions: 1646266834
}
}
version_labels {
key: 'stable'
value: 1646266834
}
}
}
The models directory is setup as follows:
$ ls -Rls models
total 0
0 drwxr-xr-x 3 david staff 96 Mar 2 16:21 mnist
models/mnist:
total 0
0 drwxr-xr-x 6 david staff 192 Mar 2 16:21 1646266834
models/mnist/1646266834:
total 304
0 drwxr-xr-x 2 david staff 64 Mar 2 16:21 assets
32 -rw-r--r-- 1 david staff 15873 Mar 2 16:20 keras_metadata.pb
272 -rw-r--r-- 1 david staff 138167 Mar 2 16:20 saved_model.pb
0 drwxr-xr-x 4 david staff 128 Mar 2 16:21 variables
models/mnist/1646266834/assets:
total 0
models/mnist/1646266834/variables:
total 1424
1416 -rw-r--r-- 1 david staff 722959 Mar 2 16:20 variables.data-00000-of-00001
8 -rw-r--r-- 1 david staff 2262 Mar 2 16:20 variables.index
The problem is that when I build and run my container, I receive an error.
$ docker build -t example.com/example-tf-serving:1.0 .
$ docker run -it -p 8500:8500 -p 8501:8501 --name example-tf-serving --rm example.com/example-tf-serving:1.0
The error is as follows Not found: /models/model:
2022-03-03 00:48:06.242923: I tensorflow_serving/model_servers/server.cc:89] Building single TensorFlow model file config: model_name: model model_base_path: /models/model
2022-03-03 00:48:06.243215: I tensorflow_serving/model_servers/server_core.cc:465] Adding/updating models.
2022-03-03 00:48:06.243254: I tensorflow_serving/model_servers/server_core.cc:591] (Re-)adding model: model
2022-03-03 00:48:06.243899: E tensorflow_serving/sources/storage_path/file_system_storage_path_source.cc:365] FileSystemStoragePathSource encountered a filesystem access error: Could not find base path /models/model for servable model with error Not found: /models/model not found
How do I fix my Dockerfile so that the above command will work?
For this solution, the quick and easy way will not work for me, so I cannot accept it as a solution:
docker run --name=the_name -p 9000:9000 -it -v "/path_to_the_model_in_computer:/path_to_model_in_docker" tensorflow/serving:1.15.0 --model_name=MODEL_NAME --port=9000
https://www.tensorflow.org/tfx/serving/docker
Optional environment variable MODEL_NAME (defaults to model)
Optional environment variable MODEL_BASE_PATH (defaults to /models)
You are using default values of these env variables, so Tensorflow is trying to find model in /models/model. You have different model path in the container, so /models/model not found is correct.
I would say simple configuration of MODEL_NAME env variable should solve the problem:
$ docker run -it -p 8500:8500 -p 8501:8501 \
--name example-tf-serving \
-e MODEL_NAME=mnist \
--rm example.com/example-tf-serving:1.0
For multiple models https://www.tensorflow.org/tfx/serving/serving_config#model_server_configuration
The easiest way to serve a model is to provide the --model_name and --model_base_path flags (or setting the MODEL_NAME environment variable if using Docker). However, if you would like to serve multiple models, or configure options like polling frequency for new versions, you may do so by writing a Model Server config file.
You may provide this configuration file using the --model_config_file flag and instruct Tensorflow Serving to periodically poll for updated versions of this configuration file at the specifed path by setting the --model_config_file_poll_wait_seconds flag.
See docker doc: https://www.tensorflow.org/tfx/serving/docker#passing_additional_arguments
You need to set CMD in the Dockerfile (so you don't need to specify it in run time, because requirement is to use only Dockerfile), e.g.:
FROM tensorflow/serving:latest
WORKDIR /
COPY models.config /models/models.config
COPY models/mnist /models/mnist
CMD ["--model_config_file=/models/models.config"]

jmxterm: "Unable to create a system terminal" inside Docker container

I have a Docker image which contains JRE, some Java web application and jmxterm. The latter is used for running some ad-hoc administrative tasks. The image is used on the CentOS 7 server with Docker 1.13 (which is pretty old but is the latest version which is supplied via the distro's repository) to run the web application itself.
All works well, but after updating jmxterm from 1.0.0 to the latest version (1.0.2), I get the following warning when entering the running container and starting jmxterm:
WARNING: Unable to create a system terminal, creating a dumb terminal (enable debug logging for more information)
After this, jmxterm does not react to arrow keys (when trying to navigate through the command history), nor does it provide autocompletion.
Some quick investigation shows that the problem may be reproduced in the clean environment with CentOS 7. Say, this is how I could bootstrap the system and the container with all stuff I need:
$ vagrant init centos/7
$ vagrant up
$ vagrant ssh
[vagrant#localhost ~]$ sudo yum install docker
[vagrant#localhost ~]$ sudo systemctl start docker
[vagrant#localhost ~]$ sudo docker run -it --entrypoint bash openjdk:11
root#0c4c614de0ee:/# wget https://github.com/jiaqi/jmxterm/releases/download/v1.0.2/jmxterm-1.0.2-uber.jar
And this is how I enter the container and run jmxterm:
[vagrant#localhost ~]$ sudo docker exec -it 0c4c614de0ee sh
root#0c4c614de0ee:/# java -jar jmxterm-1.0.2-uber.jar
WARNING: Unable to create a system terminal, creating a dumb terminal (enable debug logging for more information)
root#0c4c614de0ee:/# bea<TAB>
<Nothing happens, but autocompletion had to appear>
Few observations:
the problem does not appear with older jmxterm no matter which image do I use;
the problem arises with new jmxterm no matter which image do I use;
the problem is not reproducible on my laptop (which has newer kernel and Docker);
the problem is not reproducible if I use latest Docker (from the external repo) on the CentOS 7 server instead of CentOS 7's native version 1.13.
What happens, and why the error is reproducible only in specific environments? Is there any workaround for this?
TLDR: running new jmxterm versions as java -jar jmxterm-1.0.2-uber.jar < /dev/tty is a quick, dirty and hacky workaround for having the autocompletion and other stuff work inside the interactive container session.
A quick check shows that jmxterm tries to determine the terminal device used by the process — probably to obtain the terminal capabilities later — by running the tty utility:
root#0c4c614de0ee:/# strace -f -e 'trace=execve,wait4' java -jar jmxterm-1.0.2-uber.jar
execve("/opt/java/openjdk/bin/java", ["java", "-jar", "jmxterm-1.0.2-uber.jar"], 0x7ffed3a53210 /* 36 vars */) = 0
...
[pid 432] execve("/usr/bin/tty", ["tty"], 0x7fff8ea39608 /* 36 vars */) = 0
[pid 433] wait4(432, [{WIFEXITED(s) && WEXITSTATUS(s) == 1}], 0, NULL) = 432
WARNING: Unable to create a system terminal, creating a dumb terminal (enable debug logging for more information)
The utility fails with the status of 1, which is likely the reason for the error message. Why?
root#0c4c614de0ee:/# strace -y tty
...
readlink("/proc/self/fd/0", "/dev/pts/3", 4095) = 10
stat("/dev/pts/3", 0x7ffe966f2160) = -1 ENOENT (No such file or directory)
...
write(1</dev/pts/3>, "not a tty\n", 10not a tty
) = 10
The utility says "not a tty" while we definitely have one. A quick check shows that... There is really no PTY device in the container though the standard streams of the shell are connected to one!
root#0c4c614de0ee:/# ls -l /proc/self/fd
total 0
lrwx------. 1 root root 64 Jun 3 21:26 0 -> /dev/pts/3
lrwx------. 1 root root 64 Jun 3 21:26 1 -> /dev/pts/3
lrwx------. 1 root root 64 Jun 3 21:26 2 -> /dev/pts/3
lr-x------. 1 root root 64 Jun 3 21:26 3 -> /proc/61/fd
root#0c4c614de0ee:/# ls -l /dev/pts
total 0
crw-rw-rw-. 1 root root 5, 2 Jun 3 21:26 ptmx
What if we check the same with latest Docker?
root#c0ebd608f79a:/# ls -l /proc/self/fd
total 0
lrwx------ 1 root root 64 Jun 3 21:45 0 -> /dev/pts/1
lrwx------ 1 root root 64 Jun 3 21:45 1 -> /dev/pts/1
lrwx------ 1 root root 64 Jun 3 21:45 2 -> /dev/pts/1
lr-x------ 1 root root 64 Jun 3 21:45 3 -> /proc/16/fd
root#c0ebd608f79a:/# ls -l /dev/pts
total 0
crw--w---- 1 root tty 136, 0 Jun 3 21:44 0
crw--w---- 1 root tty 136, 1 Jun 3 21:45 1
crw-rw-rw- 1 root root 5, 2 Jun 3 21:45 ptmx
Bingo! Now we have our PTYs where they should be, so jmxterm works well with latest Docker.
It seems pretty weird that with older Docker the processes are connected to some PTYs while there are no devices for them in /dev/pts, but tracing the Docker process explains why this happens. Older Docker allocates the PTY for the container before setting other things up (including entering the new mount namespace and mounting devpts into it or just entering the mount namespace in case of docker exec -it):
[vagrant#localhost ~]$ sudo strace -p $(pidof docker-containerd-current) -f -e trace='execve,mount,unshare,openat,ioctl'
...
[pid 3885] openat(AT_FDCWD, "/dev/ptmx", O_RDWR|O_NOCTTY|O_CLOEXEC) = 9
[pid 3885] ioctl(9, TIOCGPTN, [1]) = 0
[pid 3885] ioctl(9, TIOCSPTLCK, [0]) = 0
...
[pid 3898] unshare(CLONE_NEWNS|CLONE_NEWUTS|CLONE_NEWIPC|CLONE_NEWNET|CLONE_NEWPID) = 0
...
[pid 3899] mount("devpts", "/var/lib/docker/overlay2/3af250a9f118d637bfba5701f5b0dfc09ed154c6f9d0240ae12523bf252e350c/merged/dev/pts", "devpts", MS_NOSUID|MS_NOEXEC, "newinstance,ptmxmode=0666,mode=0"...) = 0
...
[pid 3899] execve("/bin/bash", ["bash"], 0xc4201626c0 /* 7 vars */ <unfinished ...>
Note the newinstance mount option which ensures that the devpts mount owns its PTYs exclusively and does not share them with other mounts. This leads to the interesting effect: the PTY device for the container stays on the host and belongs to the host's devpts mount, while the containerized process still has access to it, as it obtained the already-open file descriptors just in the beginning of its life!
The latest Docker first mounts devpts for the container and then allocates the PTY, so the PTY belongs to container's devpts mount and is visible inside the container's filesystem:
$ sudo strace -p $(pidof containerd) -f -e trace='execve,mount,unshare,openat,ioctl'
...
[pid 14043] unshare(CLONE_NEWNS|CLONE_NEWUTS|CLONE_NEWIPC|CLONE_NEWPID|CLONE_NEWNET) = 0
...
[pid 14044] mount("devpts", "/var/lib/docker/overlay2/b743cf16ab954b9a4b4005bca0aeaa019c4836c7d58d6073044e5b48446c3d62/merged/dev/pts", "devpts",
MS_NOSUID|MS_NOEXEC, "newinstance,ptmxmode=0666,mode=0"...) = 0
...
[pid 14044] openat(AT_FDCWD, "/dev/ptmx", O_RDWR|O_NOCTTY|O_CLOEXEC) = 7
[pid 14044] ioctl(7, TIOCGPTN, [0]) = 0
[pid 14044] ioctl(7, TIOCSPTLCK, [0]) = 0
...
[pid 14044] execve("/bin/bash", ["/bin/bash"], 0xc000203530 /* 4 vars */ <unfinished ...>
Well, the problem is caused by inappropriate Docker behavior, but how comes that older jmxterm worked well in the same environment? Let's check (note, that Java 8 image is used here, as older jmxterm does not play well with Java 11):
root#504a7757e310:/# wget https://github.com/jiaqi/jmxterm/releases/download/v1.0.0/jmxterm-1.0.0-uber.jar
root#504a7757e310:/# strace -f -e 'trace=execve,wait4' java -jar jmxterm-1.0.0-uber.jar
execve("/usr/local/openjdk-8/bin/java", ["java", "-jar", "jmxterm-1.0.0-uber.jar"], 0x7fffdcaebdd0 /* 10 vars */) = 0
...
[pid 310] execve("/bin/sh", ["sh", "-c", "stty -a < /dev/tty"], 0x7fff1f2a1cc8 /* 10 vars */) = 0
So, older jmxterm just uses /dev/tty instead of asking tty for the device name, and this works, as this device is present inside the container:
root#504a7757e310:/# ls -l /dev/tty
crw-rw-rw-. 1 root root 5, 0 Jun 3 21:36 /dev/tty
The huge difference between these versions of jmxterm is that newer tool version uses higher major version of jline, which is the library responsible for interaction with the terminal (akin to the readline in the C world). The difference between major jline versions leads to the difference in jmxterm's behavior, and current versions just rely on tty.
This observation leads us to the quick and dirty workaround which does not require neither updating Docker nor patching the jline/jmxterm tandem: we may just attach jmxterm's stdin to /dev/tty forcibly and thus make jline use this device (which is now referenced by /proc/self/fd/0) instead of the /dev/pts entry (which, formally, is not always correct, but is still enough for ad-hoc use):
root#0c4c614de0ee:/# java -jar jmxterm-1.0.2-uber.jar < /dev/tty
Welcome to JMX terminal. Type "help" for available commands.
$>bea<TAB>
bean beans
Now we have the autocompletion, history and other cool things we need to have.
If you are trying to run an interactive application (that needs tty) inside a docker container or a pod in kubernetes, then the following should work.
For docker-compose use:
image: image-name:2.0
container_name: container-name
restart: always
stdin_open: true
tty: true
For kubernetes use:
spec:
containers:
- name: web
image: web:latest
tty: true
stdin: true

Mounted Docker volume has different ownership when using Travis

This question relates to this repository with the most relevant Travis job here.
The repository is for static site built from Jupyter notebooks. The notebooks are converted using build/build.py which, for each post, builds a Docker image, starts a corresponding container with the post notebook directory mounted, and uses nbconvert to convert the notebook to Markdown. One step of nbconvert's conversion involves creating a supporting file directory. This fails on Travis due to a permission issue.
In attempting to debug this problem, I found that the ownership and permissions of the repo are the same on my local machine and Travis (with my username switched for travis) before running Docker. Despite this, inside the mounted volume of the Docker container, the ownerships are different:
Local:
drwxrwxr-x 3 jovyan 1000 4096 Dec 10 19:56 .
drwsrwsr-x 1 jovyan users 4096 Dec 3 21:51 ..
-rw-rw-r-- 1 jovyan 1000 105 Dec 7 09:57 Dockerfile
drwxr-xr-x 2 jovyan 1000 4096 Dec 10 12:09 .ipynb_checkpoints
-rw-r--r-- 1 jovyan 1000 154229 Dec 10 12:28 post.ipynb
Travis:
drwxrwxr-x 2 2000 2000 4096 Dec 10 19:58 .
drwsrwsr-x 1 jovyan users 4096 Nov 8 16:37 ..
-rw-rw-r-- 1 2000 2000 101 Dec 10 19:58 Dockerfile
-rw-rw-r-- 1 2000 2000 35271 Dec 10 19:58 post.ipynb
Both my local machine and Travis are running Ubuntu 20.04, have the same version of Docker, and all other tools come from Conda so should behave the same. I am struggling to understand where this difference in ownership is coming from.
Try running the docker again with this command, so the uid outside the container is propagated inside:
docker run -u `id -u`
alternative, as pointed by #anemyte:
docker run -u $(id -u)
This should involve the creation of the new files inside the docker to be owned by "jovyan".
If you are able to guess that mounting points will exist, you could also pre-create them so the ownership of the files inside is also correct:
docker run -v /path/on/host:/path/in/container ...
If you set the permissions of your local path (/path/on/host) as 777, that will also be propagated to the mounting point: no permission error will be thrown regardless of the user that docker uses to create those files.
After that, you'll be free to restore permissions, if needed.

How to run Tomcat 8 and MySQL in single Docker container

I have a Tomcat 8 / MySQL application I want to run in a docker container. I run Ubuntu 16.04 today in test and production and wanted use the Ubuntu 16.04 "latest" as the base FROM to my docker file and add Tomcat 8 and MySQL from there.
I know I can get a Tomcat 8 docker file as my base from https://hub.docker.com/_/tomcat/ but I did not see an Ubuntu base OS for those and I wanted to stay consistent with Ubuntu. Also, it seemed odd to add MySQL to a Tomcat container.
I worked through this issue and am posting my findings in case it helps others with similar issues.
Short answer: Running multiple services (tomcat / mysql) in a single container is not recommended. Yes, there is supervisor.d, etc. But this is discouraged. There is also baseimage-docker if you are committed to multiple services in one container.
The remainder of this answer shows how I got it working it if you really are determined...
The Tomcat 8 distro version on Ubuntu 16.04 is unfortunately only configured to run as a service (described in detail below). Issues with running a service in a docker container are documented well in many posts across stack exchange (it is discouraged). I was able to get tomcat 8 working as a service by adding a "tail -f /var/log/tomcat8/catalina.out" to the end of the "service tomcat8 start" command and starting the container with the "--cap-add SYS_PTRACE" option.
CMD service tomcat8 start && tail -f /var/log/tomcat8/catalina.out
The recommended way to start tomcat8 is to use the commands in /usr/share/tomcat8/bin. However, the distro version's soft links are incorrect and the server fails to start.
Using the commands ./catalina.sh run or ./startup.sh both produce an error such as this:
SEVERE: Cannot find specified temporary folder at /usr/share/tomcat8/temp
WARNING: Unable to load server configuration from [/usr/share/tomcat8/conf/server.xml]
SEVERE: Cannot start server. Server instance is not configured.
The distro splits tomcat8 across /usr/share/tomcat8 and /var/lib/tomcat8 which separates the bin files (catalina.sh and startup.sh) from the config and logs soft links in /var/lib/tomcat8. This makes these commands fail.
Files in /usr/share/tomcat8:
root#85d5fe47b66a:/usr/share/tomcat8# ls -la
total 32
drwxr-xr-x 4 root root 4096 Mar 9 22:18 .
drwxr-xr-x 117 root root 4096 Mar 9 23:29 ..
drwxr-xr-x 2 root root 4096 Mar 9 22:18 bin
-rw-r--r-- 1 root root 39 Mar 31 2017 defaults.md5sum
-rw-r--r-- 1 root root 1929 Apr 10 2017 defaults.template
drwxr-xr-x 2 root root 4096 Mar 9 22:18 lib
-rw-r--r-- 1 root root 53 Mar 31 2017 logrotate.md5sum
-rw-r--r-- 1 root root 118 Apr 10 2017 logrotate.template
Files in /var/lib/tomcat8:
root#85d5fe47b66a:/var/lib/tomcat8# ls -la
total 16
drwxr-xr-x 4 root root 4096 Mar 9 22:18 .
drwxr-xr-x 41 root root 4096 Mar 9 23:29 ..
lrwxrwxrwx 1 root root 12 Sep 28 14:43 conf -> /etc/tomcat8
drwxr-xr-x 2 tomcat8 tomcat8 4096 Sep 28 14:42 lib
lrwxrwxrwx 1 root root 17 Sep 28 14:43 logs -> ../../log/tomcat8
drwxrwxr-x 3 tomcat8 tomcat8 4096 Mar 9 22:18 webapps
lrwxrwxrwx 1 root root 19 Sep 28 14:43 work -> ../../cache/tomcat8
Running ./version.sh reveals that both CATALINA_BASE and CATALINA_HOME are set to /usr/share/tomcat8
Using CATALINA_BASE: /usr/share/tomcat8
Using CATALINA_HOME: /usr/share/tomcat8
Using CATALINA_TMPDIR: /usr/share/tomcat8/temp
Using JRE_HOME: /usr
Using CLASSPATH: /usr/share/tomcat8/bin/bootstrap.jar:/usr/share/tomcat8/bin/tomcat-juli.jar
Server version: Apache Tomcat/8.0.32 (Ubuntu)
Server built: Sep 27 2017 21:23:18 UTC
Server number: 8.0.32.0
OS Name: Linux
OS Version: 4.4.0-116-generic
Architecture: amd64
JVM Version: 1.8.0_161-b12
JVM Vendor: Oracle Corporation
Setting CATALINA_BASE explicitly to /var/lib/tomcat8 inside catalina.sh solved the problem in using ./catalina.sh run to start tomcat. In the past, I have alternatively added the soft links to conf, logs and work under the /usr/share/tomcat8 directory so it could find those files and start up properly with the catalina.sh run command.
BTW, even thought the JRE_HOME is clearly wrong in the version.sh dump above, the service does start correctly (when I append the tail -f command as described earlier). It also starts using catalina.sh run when I manually add the correct CATALINA_BASE variable to catalina.sh. So I spent no time looking into why that listed out incorrectly.
In the end, I realized three things:
Running multiple services (tomcat / mysql) in a single container is not recommended. Yes, there is supervisor.d, etc. But this is discouraged. There is also baseimage-docker if you are committed to multiple services in one container.
Even running a single service in a container is not recommended but there are documented ways to make it work (which I did for tomcat8 by adding the && tail -f ... to the end of the CMD).
In Ubuntu 16.04 (did not test other distros), to make tomcat8 run as a command (not a service) you need to either:
a) grab the tar file for Tomcat 8 and install that, since it puts all of the files under one directory and therefore there is no soft link issue. Or, b) if you insist on using the distro tomcat8 from apt-get, b.1) you need to modify a version of catalina.sh by adding the CATALINA_BASE and copy it to the proper installation directory or b.2) add the soft links.

Resources