Why is the following failing on a Docker image for Debian Jessie?
# curl --silent --show-error --location https://download.owncloud.org/community/owncloud-8.0.3.tar.bz2 | tar xjv
tar (grandchild): bzip2: Cannot exec: No such file or directory
tar (grandchild): Error is not recoverable: exiting now
tar: Child died with signal 13
tar: Error is not recoverable: exiting now
curl: (23) Failed writing body (4096 != 16384)
I can't run that inside Dockerfile RUN statements, or in an interactive bash shell for the image "php:5.6-fpm" (based off "debian:jessie").
I can, however, run it on lots of other Debian and Ubuntu systems.
Is this a problem with Docker?
This error message seems pretty clear:
tar (grandchild): bzip2: Cannot exec: No such file or directory
bzip2 isn't present in the debian:jessie instance, so if you want to use bzip2 compression, you have to install bzip2 first with apt-get install bzip2.
Related
I need to extract the filesystem of a debian image onto the host, modify it, then repackage it back into a docker image. I'm using the following commands:
docker export container_name > archive.tar
tar -xf archive.tar -C debian/
modifying the file system here
tar -cpjf archive-modified.tar debian/
docker import archive-modified.tar debian-modified
docker run -it debian-modified /bin/bash
After I try to run the new docker image I get the following error:
docker: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "/bin/bash": stat /bin/bash: no such file or directory: unknown.
ERRO[0000] error waiting for container: context canceled
I've tried the above steps without modifying the file system at all and I get the same behavior. I've also tried importing the output of docker export directly, and this works fine. This probably means I'm creating the new tar archive incorrectly. Can anyone tell me what I'm doing wrong?
Take a look at the archive generated by docker export:
# tar tf archive.tar | sort | head
bin/
bin/bash
bin/cat
bin/chgrp
bin/chmod
bin/chown
bin/cp
bin/dash
bin/date
bin/dd
And then at the archive you generate with your tar -cpjf ... command:
# tar tf archive-modified.tar | sort | head
debian/
debian/bin/
debian/bin/bash
debian/bin/cat
debian/bin/chgrp
debian/bin/chmod
debian/bin/chown
debian/bin/cp
debian/bin/dash
debian/bin/date
You've moved everything into a debian/ top-level directory, so there is no /bin/bash in the image (it would be /debian/bin/bash, and probably wouldn't work anyway because your shared libraries aren't in the expected location, either.
You probably want to create the updated archive like this:
# tar -cpjf archive-modified.tar -C debian/ .
I am doing this tutorial to load batch data using Apache Hadoop and I try to run this command to build a Docker image named "druid-hadoop-demo" with version tag "2.8.5" :
docker build -t druid-hadoop-demo:2.8.5 .
but it gives me this error
Step 14/53 : RUN curl -s https://archive.apache.org/dist/hadoop/core/hadoop-2.8.3/hadoop-2.8.3.tar.gz | tar -xz -C /usr/local/
---> Running in 7baa699ccc29
gzip: stdin: unexpected end of file
tar: Child returned status 1
tar: Error is not recoverable: exiting now
The command '/bin/sh -c curl -s https://archive.apache.org/dist/hadoop/core/hadoop-2.8.3/hadoop-2.8.3.tar.gz | tar -xz -C /usr/local/' returned a non-zero code: 2
Any help is appreciated.
That's a network error, indicating the file you downloaded was either corrupted in transit (it downloaded correctly for me, so unlikely to be the source), or the connection was dropped before finishing the curl command. Check for network proxies, intermittent network failures, or anything else that could interrupt the connection. You can also try to manually run curl from within a container to see if it's successful, e.g.
curl -o hadoop.tgz https://archive.apache.org/dist/hadoop/core/hadoop-2.8.3/hadoop-2.8.3.tar.gz
tar -tvzf hadoop.tgz
I run these command:
docker run -ti --rm alpine
apk add --no-cache curl
curl https://registry.npmjs.org/monaco-editor/-/monaco-editor-0.15.6.tgz | tar xvz --strip 1 package/min
and got error tar: package/min: not found in archive.
I run the same command (curl | tar) in Mac terminal and docker ubuntu, all of them are success.
The tar included in Alpine isn't the usual (GNU) tar, but a component of BusyBox:
/scratch # tar --version
tar (busybox) 1.28.4
Apparently, this version of tar generates a (bogus) error message when run with
tar xvz --strip 1 package/min
(however, at first glance, it created the target directory just fine, so ignoring the error message might be ok).
To get rid of the annoying error, you should install GNU tar and use that:
/scratch # apk add --no-cache tar
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz
(1/1) Installing tar (1.31-r0)
Executing busybox-1.28.4-r2.trigger
OK: 7 MiB in 19 packages
/scratch # tar --version
tar (GNU tar) 1.31
<rest of message omitted>
Afterwards, running your original command works without any error messages.
In a build process I need to call zipalign which is on a certain path in the docker container that I'm using:
$ docker run nathansamson/flutter-builder-docker:v0.7.3 find . -iname zipalign
./opt/android-sdk-linux/build-tools/28.0.1/zipalign
This path can change, if the docker container is updated and there is a new android sdk. For example this could be the path in the future:
$ docker run nathansamson/flutter-builder-docker:v0.9.9 find . -iname zipalign
./opt/android-sdk-linux/build-tools/42.0.0/zipalign
So instead of hardcoding the call to
docker run nathansamson/flutter-builder-docker:v0.7.3 \
/opt/android-sdk-linux/build-tools/28.0.1/zipalign -h
I would like a generic solution that finds the path to zipalign automatically. I have tried it with a subshell
$ docker run nathansamson/flutter-builder-docker:v0.7.3 $(find . -iname zipalign) -h
docker: Error response from daemon: OCI runtime create failed:
container_linux.go:348: starting container process caused "exec: \"-h\":
executable file not found in $PATH": unknown.
ERRO[0001] error waiting for container: context canceled
and with a wildcard for the folder:
$ docker run nathansamson/flutter-builder-docker:v0.7.3 /opt/android-sdk-linux/build-tools/*/zipalign -h
docker: Error response from daemon: OCI runtime create failed:
container_linux.go:348: starting container process caused "exec:
\"/opt/android-sdk-linux/build-tools/*/zipalign\": stat /opt/android-
sdk-linux/build-tools/*/zipalign: no such file or directory": unknown.
ERRO[0001] error waiting for container: context canceled
So subshells and wildcard don't work in Docker. Any ideas how I can find the path to zipalign whenever I'm calling it?
In your Dockerfile you control the entire environment. It's often easiest to cause things to appear in their "natural" places, like /usr/bin. You also have the advantage that, within a single Docker image, there will only be one version of the tools installed.
I might do something like this:
RUN for f in $PWD/opt/android-sdk-linux/build-tools/*/*; do \
ln -s $f /usr/local/bin; \
done
CMD ["zipalign", "-h"]
Another approach that might work is to use a build argument or an environment variable to hold the version number. If you do that then you can set up a known path name.
ARG version
RUN curl -LO http://.../android-sdk-linux-${version}.tar.gz \
&& tar xzf android-sdk-linux-${version}.tar.gz \
&& rm -f android-sdk-linux-${version}.tar.gz \
&& cd opt/android-sdk-linux-build-tools \
&& ln -s ${version} current
CMD ["./opt/android-sdk-linux/build-tools/current/zipalign"]
find can execute something by using the -exec option.
find . -name zipalign -exec bash -c '"$0"' {} \;
In your example:
docker run nathansamson/flutter-builder-docker:v0.7.3 find . -name zipalign -exec bash -c '"$0"' {} \;
If there is nothing that can give you any hints, and you need to just find it, then find will probably do it. Something like:
find /opt -name zipalign -type f
If you can give it more specific starting point, instead of just /opt, then it will run faster.
If you want to execute it (and pass in "-h") in one line, you could do:
$(find /opt -name zipalign -type f) -h
I have a project running on Docker with docker-compose for dev environment.
I want to get it running on GitLabCI with a gitlab-ci-multi-runner "Docker mode" instance.
Here is my .gitlab-ci.yml file:
image: soullivaneuh/docker-bash
before_script:
- apk add --update bash curl
- curl --silent --location https://github.com/docker/compose/releases/download/1.5.2/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
- chmod +x /usr/local/bin/docker-compose
- ./configure
- docker-compose up -d
Note that soullivaneuh/docker-bash image is just a docker image with bash installed.
The script fails on docker-compose up -d command:
gitlab-ci-multi-runner 0.7.2 (998cf5d)
Using Docker executor with image soullivaneuh/docker-bash ...
Pulling docker image soullivaneuh/docker-bash:latest ...
Running on runner-1ee5079f-project-3-concurrent-1 via sd-59984...
Fetching changes...
Removing app/config/parameters.yml
Removing docker-compose.env
HEAD is now at 5c5e7ff remove docker service
From https://git.dummy.net/project/project
5c5e7ff..45e643d docker-ci -> origin/docker-ci
Checking out 45e643dd as docker-ci...
Previous HEAD position was 5c5e7ff... remove docker service
HEAD is now at 45e643d... Remove docker info commands
$ apk add --update bash curl
fetch http://dl-4.alpinelinux.org/alpine/v3.2/main/x86_64/APKINDEX.tar.gz
OK: 10 MiB in 28 packages
$ curl --silent --location https://github.com/docker/compose/releases/download/1.5.2/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
$ chmod +x /usr/local/bin/docker-compose
$ ./configure
$ docker-compose up -d
bash: line 30: /usr/local/bin/docker-compose: No such file or directory
ERROR: Build failed with: exit code 1
I have absolutly no idea why this is failing.
Thanks for help.
The No such file or directory is misleading. I've received that many times while trying to run dynamically linked binaries using alpine linux (which it appears you are using).
The problem (as I understand it) is that the binary was compiled and linked against glibc, but alpine uses musl, not glibc.
You could use ldd /usr/local/bin/docker-compose to tell you which libraries are missing (or run it with strace if all else fails).
To get it working, it might be easier to install from python source (https://docs.docker.com/compose/install/#install-using-pip), which is what the official compose image does (https://github.com/docker/compose/blob/master/Dockerfile.run).
Or you could use an image built on debian or some other distro that uses glibc.