File copied by Docker seen as a directory - docker

I'm trying to dockerize a Stardog 3.1.3 community edition server. The container fails to start because it sees a directory instead of a license file. For the record, I'm on Windows. These are the steps I'm following:
Create a data container
docker create -v /data/stardog:/data/stardog --name stardog_data busybox /bin/true
Copy the local license key to the data container (not done in the Dockerfile that is mentioned below as the license is environment specific)
docker cp .\stardog\stardog-license-key.bin stardog_data:/stardog-license-key.bin
Create an image based on the following Dockerfile
docker build -t me/stardog .
FROM java:openjdk-8-jdk
ENV STARDOG_VER stardog-3.1.3
ENV STARDOG_HOME /data/stardog
COPY ${LOCAL_PATH}/${STARDOG_VER}.zip /
RUN unzip ${STARDOG_VER}.zip
WORKDIR /${STARDOG_VER}
CMD rm $STARDOG_HOME/system.lock || true && bin/stardog-admin server start && (tail -f $STARDOG_HOME/stardog.log &) && while (pidof java > /dev/null); do sleep 1; done
Run a Stardog container
docker run -d -p 5820:5820 --volumes-from stardog_data --name stardog me/stardog
When I execute docker ps -a, I see that the container is stopped after a couple of seconds:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9559b22473e1 me/stardog "/bin/sh -c 'rm $STAR" 26 minutes ago Exited (1) 26 minutes ago stardog
2b929329e35e busybox "/bin/true" 32 minutes ago Created stardog_data
When I check the logs with docker logs stardog, I'm getting this:
com.clarkparsia.license.InvalidLicenseException: java.io.FileNotFoundException: /data/stardog/stardog-license-key.bin (Is a directory)
at com.clarkparsia.license.LicenseValidator.validate(LicenseValidator.java:157)
at com.complexible.stardog.StardogLicense.findLicense(StardogLicense.java:127)
at com.complexible.stardog.StardogLicense.<init>(StardogLicense.java:70)
at com.complexible.stardog.Stardog.<init>(Stardog.java:158)
at com.complexible.stardog.Stardog.initialize(Stardog.java:263)
at com.complexible.stardog.Stardog.initialize(Stardog.java:254)
at com.complexible.stardog.Stardog.buildServer(Stardog.java:247)
at com.complexible.stardog.cli.impl.ServerStart.call(ServerStart.java:144)
at com.complexible.stardog.cli.impl.ServerStart.call(ServerStart.java:47)
at com.complexible.stardog.cli.CLIBase.execute(CLIBase.java:54)
at com.complexible.stardog.cli.admin.CLI.main(CLI.java:194)
Caused by: java.io.FileNotFoundException: /data/stardog/stardog-license-key.bin (Is a directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at com.clarkparsia.license.LicenseValidator.validate(LicenseValidator.java:113)
... 10 more
Your Stardog license is invalid. Please contact support#clarkparsia.com for information on obtaining a new license.
It seems that the license file is considered to be a directory. What am I doing wrong?
I'm using the following Docker version:
Client:
Version: 1.10.3
API version: 1.22
Go version: go1.5.3
Git commit: 20f81dd
Built: Thu Mar 10 21:49:11 2016
OS/Arch: windows/amd64
Server:
Version: 1.10.3
API version: 1.22
Go version: go1.5.3
Git commit: 20f81dd
Built: Thu Mar 10 21:49:11 2016
OS/Arch: linux/amd64

This seems to be a Windows related problem. I've tried these exact same steps on a native Ubuntu (14.04) machine and it works as expected. I've filed this as a bug and hopefully this gets fixed soon.

Related

Unable to start Flink JobManager container in Docker

I am unable to start a Flink JobManager Docker container on M1 MacBook running Monterey. Below is the docker command pulled from the Flink Docs and the resulting java.io.IOException
docker run \
--rm \
--name=jobmanager \
--network flink-network \
--publish 8081:8081 \
--env FLINK_PROPERTIES="${FLINK_PROPERTIES}" \
flink:1.16.0-scala_2.12 jobmanager
INFO org.apache.flink.runtime.entrypoint.ClusterEntrypoint [] - Shutting StandaloneSessionClusterEntrypoint down with application status FAILED. Diagnostics java.io.IOException: Could not create the working directory /tmp/jm_ba47b82cf8d85068faa1c41d30126b5d.
at org.apache.flink.runtime.entrypoint.WorkingDirectory.createDirectory(WorkingDirectory.java:58)
at org.apache.flink.runtime.entrypoint.WorkingDirectory.<init>(WorkingDirectory.java:39)
at org.apache.flink.runtime.entrypoint.WorkingDirectory.create(WorkingDirectory.java:88)
at org.apache.flink.runtime.entrypoint.ClusterEntrypointUtils.lambda$createJobManagerWorkingDirectory$2(ClusterEntrypointUtils.java:241)
at org.apache.flink.runtime.entrypoint.DeterminismEnvelope.map(DeterminismEnvelope.java:49)
at org.apache.flink.runtime.entrypoint.ClusterEntrypointUtils.createJobManagerWorkingDirectory(ClusterEntrypointUtils.java:239)
at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.initializeServices(ClusterEntrypoint.java:356)
at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runCluster(ClusterEntrypoint.java:282)
at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.lambda$startCluster$1(ClusterEntrypoint.java:232)
at org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28)
at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:229)
at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:729)
at org.apache.flink.runtime.entrypoint.StandaloneSessionClusterEntrypoint.main(StandaloneSessionClusterEntrypoint.java:59)
My docker version is:
Client:
Cloud integration: v1.0.29
Version: 20.10.22
API version: 1.41
Go version: go1.18.9
Git commit: 3a2c30b
Built: Thu Dec 15 22:28:41 2022
OS/Arch: darwin/arm64
Context: default
Experimental: true
Why does the container not allow creating the required directory?
Looks like this was related to disk space issue. Running docker system prune cleared some space and the container is running nicely.

Docker secrets not working when password has special characters

I am trying to run jenkins swarm agent using the docker-compose provided here.
Issue is I am using a service account provided by my admin team to authenticate against jenkins master and the password for the service account contains special characters \ (backslash) and / (forward slash). Example:
abcdefghifjd12\ab/
The docker secret I created using this password doesn't work. I tried with my personal account which doesn't have any special characters and this works as expected. So I believe issue is with how docker is interpreting these special characters when creating secrets. I tried escaping the back slash and using single quotes for the password but doesn't work. I tried below
#use password as it is in double quotes
echo "abcdefghifjd12\ab/" | docker secret create jenkins-user -
#use backslash to escape backslash
echo "abcdefghifjd12\\ab/" | docker secret create jenkins-user -
#use single quotes
echo 'abcdefghifjd12\ab/' | docker secret create jenkins-user -
None worked. How do I resolve this issue?
Server:
Version: 17.06.2-ee-6
API version: 1.30 (minimum version 1.12)
Go version: go1.8.3
Git commit: e75fdb8
Built: Mon Nov 27 22:44:25 2017
OS/Arch: linux/amd64
Experimental: false
With docker upgraded version I was able to do the same thing.
On SWARM I create a secret on manager-1:
[manager1] (local) root#192.168.0.41 ~
$ echo "abcdefghifjd12\ab/" | docker secret create jenkins-user -
jxykdlqklpo5ml81c4bfa9a4o
[manager1] (local) root#192.168.0.41 ~
$ docker service create --secret jenkins-user alpine sleep 1d
sgmrof1cwwubmhz1qqibu4aof
overall progress: 1 out of 1 tasks
1/1: running
verify: Service converged
[manager1] (local) root#192.168.0.41 ~
$ docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
sgmrof1cwwub wizardly_boyd replicated 1/1 alpine:latest
[manager1] (local) root#192.168.0.41 ~
[manager2] (local) root#192.168.0.43 ~
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f4552c4718c5 alpine:latest "sleep 1d" 2 minutes ago Up 2 minutes wizardly_boyd.1.4gb4nwgiqagfyn10vuvt9pb4v
[manager2] (local) root#192.168.0.43 ~
$ docker exec -it f45 sh
/ # cd /run
/run # cd secrets/
/run/secrets # ls
jenkins-user
/run/secrets # cat jenkins-user
abcdefghifjd12\ab/
Docker Version information:
$ docker version
Client:
Version: 18.03.1-ce
API version: 1.37
Go version: go1.9.2
Git commit: 9ee9f40
Built: Thu Apr 26 07:12:25 2018
OS/Arch: linux/amd64
Experimental: false
Orchestrator: swarm
Server:
Engine:
Version: 18.03.1-ce
API version: 1.37 (minimum version 1.12)
Go version: go1.9.5
Git commit: 9ee9f40
Built: Thu Apr 26 07:23:03 2018
OS/Arch: linux/amd64
Experimental: true
[manager1] (local) root#192.168.0.41 ~

GitLab (via Docker) on a QNAP NAS with ARM CPU ("exec format error")

I just bought a QNAP TS-832X NAS (Firmware: 4.3.4.0695 Build 20180830).
This machine comes with an ARM CPU (Annapurna Labs Alpine AL324 Quad-core ARM Cortex-A57 CPU # 1.70GHz).
I bought it only to install GitLab on it, but the official image doesn't seem to work.
When I try to run the image it fails.
[~] # docker run -d --name gitlab-server --hostname build1 -p 10080:10080 -p 10022:22 -p 10443:443 -v /share/GitLab/config:/etc/gitlab -v /share/GitLab/logs:/var/log/gitlab -v /share/GitLab/data:/var/opt/gitlab --restart always gitlab/gitlab-ce:latest
[~] # docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a176158729ad gitlab/gitlab-ce:latest "/assets/wrapper" 5 seconds ago Restarting (1) 1 second ago gitlab-server
[~] # docker logs a1
standard_init_linux.go:185: exec user process caused "exec format error"
standard_init_linux.go:185: exec user process caused "exec format error"
standard_init_linux.go:185: exec user process caused "exec format error"
standard_init_linux.go:185: exec user process caused "exec format error"
standard_init_linux.go:185: exec user process caused "exec format error"
standard_init_linux.go:185: exec user process caused "exec format error"
standard_init_linux.go:185: exec user process caused "exec format error"
After googling I figured it might be caused by the host architecture, so I tried running ulm0/gitlab, but with the same result.
I also tried other images with "ARM" in their tags like arm64v8/ubuntu. This one didn't even give any logs.
[~] # docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2b2b68bc912c arm64v8/ubuntu:latest "/bin/bash" 7 seconds ago Restarting (0) 1 second ago ubuntu-arm
a176158729ad gitlab/gitlab-ce:latest "/assets/wrapper" 2 hours ago Restarting (1) 51 seconds ago gitlab-server
[~] # docker logs 2b
[~] #
uname -a
Linux build1 4.2.8 #2 SMP Thu Aug 30 07:33:01 CST 2018 aarch64 GNU/Linux
docker version
Client:
Version: 17.09.1-ce
API version: 1.32
Go version: go1.8.3
Git commit: a9fd393
Built: Fri Aug 3 04:31:20 2018
OS/Arch: linux/arm64
Server:
Version: 17.09.1-ce
API version: 1.32 (minimum version 1.12)
Go version: go1.8.3
Git commit: a9fd393
Built: Fri Aug 3 04:31:20 2018
OS/Arch: linux/arm64
Experimental: false
Sorry to hear about your problem, unfortunately I don't believe there is any official GitLab Docker image for ARM devices.
From personal experience I've found that most developers will make a Docker image for Intel devices but not work on ARM Devices.
This topic has been discussed on the QNAP Forums already:
My QNAP is Intel based, so I can't corroborate your results, but quoting a few sentences from a page about docker on Raspberry Pi:
"Docker-based apps you use have to be packaged specifically for ARM architecture! Docker-based apps packaged for x86/x64 will not work and will result in an error such as:
FATA[0003] Error response from daemon: Cannot start container 0f0fa3f8e510e53908e6a459e817d600b9649e621e7dede974d6a65761ad39e5: exec format error
Keep this in mind when searching for apps on the Docker Hub - the source for Docker apps/images. If you see the keyword RPI or ARM in the heading or description, this app can usually be used for the Raspberry Pi."
The TS-831X has a "AnnapurnaLabs, an Amazon company Alpine AL-314 Quad-core 1.7 GHz Cortex-A15 processor" CPU, which is an ARM architecture much like the Raspberry Pi..
So, I suspect you may be limited in what Docker images you have access to and unless an official/canonical maintainer of an app also makes an ARM build, you may be stuck with either rolling your own or trusting a 3rd party hobbyist to do so...
I hate to say this but I'd say you should have picked up an Intel one instead.
I have a QNAP TS-251+ (Intel based) with 8GB RAM and 2x8TB in a RAID Configuration and this works perfectly for my Gitlab instance, in addition to running PLEX and using it as a Webserver as well.
I would also suggest when you do finally get it up and running to map the volumes to directories that are easy to access so you can make configuration changes easily.

Docker image created by Bazel does not have repository/tag, files are packed with strange names and do not run

I'm trying to pack an application compiled with Bazel into a Docker image, using Bazel.
I have an empty WORKSPACE file, the following main.cc file:
#include <stdio.h>
int main() { printf("Hello World\n"); }
and the following BUILD file (all in the same folder):
load("#bazel_tools//tools/build_defs/docker:docker.bzl", "docker_build")
cc_binary(
name = "hello",
srcs = ["main.cc"],
)
docker_build(
name = "hello_docker",
files = [":hello"],
cmd = ["./hello"],
)
Gist is available here.
When I run bazel run :hello, I can see "Hello World" successfully printed. Then I run bazel build :hello_docker.tar which is supposed to generate a full image for docker in bazel-bin/hello_docker.tar. Looks like it does.
Then I run:
$ docker load -i bazel-bin/hello_docker.tar
51525b594688: Loading layer [==================================================>] 10.24 kB/10.24 kB
Loaded image ID: sha256:252367f43640968343ecd78b2a3533c8ed99c4be66d73fc3ec6ccb137df19625
Then I list all images:
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
ubuntu 14.04 b969ab9f929b 3 weeks ago 188 MB
docker/whalesay latest 6b362a9f73eb 21 months ago 247 MB
<none> <none> 252367f43640 292 years ago 8.048 kB
Looks like the image which was just loaded does not have any repository or tag associated with it. Why can that be and how do I fix that? The "created" value is huge as expected (Bazel clears timestamps for reproductibility).
When I look inside hello_docker.tar, I see something related to the expected repository/tag inside the repository file:
{
"bazel/": {
"hello_docker": "3d529b3ceeb0d9ef33f6225e655f3f5a0e061888f25cd6a3fac11604778ef6a8"
}
}
Another problem: when I try to run the image in a new container, it fails:
$ docker run -t -i 25236
docker: Error response from daemon: oci runtime error: exec: "./hello": stat ./hello: no such file or directory.
I looked inside the generated tar file one more time and found out that there is an executable here, but it's named o for some reason. Another attempt and a different error:
$ docker run -t -i 25236 ./o
write pipe: bad file descriptor
Strangely enough, if I pack that hello executable with my own Dockerfile like this, "Hello World" is printed as expected:
FROM ubuntu:14.04
ADD hello .
CMD ./hello
Can anyone suggest how to fix any of these three issues? I think that they're interconnect and I have wrong BUILD file (maybe I should somehow specify which Linux image to use as a base?). Recap:
Repository/tag name is not displayed after image generated by Bazel is imported into Docker.
hello file is added to the image under the name o.
Even when run as o, Dockered executable yields "bad file descriptor" message.
My software versions:
$ bazel version
Build label: 0.4.4
Build target: bazel-out/local-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Wed Feb 1 18:54:21 2017 (1485975261)
Build timestamp: 1485975261
Build timestamp as int: 1485975261
$ docker version
Client:
Version: 1.12.3
API version: 1.24
Go version: go1.6.2
Git commit: 6b644ec
Built: Mon, 19 Dec 2016 09:20:48 +1300
OS/Arch: linux/amd64
Server:
Version: 1.12.3
API version: 1.24
Go version: go1.6.2
Git commit: 6b644ec
Built: Mon, 19 Dec 2016 09:20:48 +1300
OS/Arch: linux/amd64

Docker: Container command '/bin/sh' not found

When i try to build docker images using the command docker build -t opencv:2.4.11 . I get following error.
Sending build context to Docker daemon 189.8 MB
Step 1 : FROM ubuntu:trusty
---> b72889fa879c
Step 2 : RUN apt-get update
---> Running in 689f34e138c2
Container command '/bin/sh' could not be invoked.
My docker client and server version are:
Client:
Version: 1.11.0
API version: 1.23
Go version: go1.5.4
Git commit: 4dc5990
Built: Wed Apr 13 18:34:23 2016
OS/Arch: linux/amd64
Server:
Version: 1.11.0
API version: 1.23
Go version: go1.5.4
Git commit: 4dc5990
Built: Wed Apr 13 18:34:23 2016
OS/Arch: linux/amd64
Any idea?
I have softlinked my /var/lib/docker directory to one of directories present in my home. And that directory is owned by root. Is this the source of problem?
Before moving and softlinking it was fine. I ran out of space in my root partition and so I had to change it to my home partition.
And my home partition is ext4 file system.
Regards
It looks badly hosed.
I have softlinked my /var/lib/docker directory to one of directories present in my home.
I would recommend amending the daemon parameters to include -g your_other_directory instead. Then stop and restart Docker.
If it still doesn't work, delete both Docker directories, re-install, and start at the -g step again.

Resources