All, i'm trying to persistently copy files from my host to an image so those files are available with every container launched based on that image. Running on debian wheezy 64bit as virtualbox guest.
the Dockerfile is fairly simple (installing octave image):
FROM debian:jessie
MAINTAINER GG_Python <[redacted]#gmail.com>
RUN apt-get update
RUN apt-get update
RUN apt-get install -y octave octave-image octave-missing-functions octave-nan octave-statistics
RUN mkdir /octave
RUN mkdir /octave/libs
RUN mkdir /octave/libs/jsonlab
COPY ~/octave/jsonlab/loadjson.m /octave/libs/jsonlab/.
I'm getting the following trace after issuing a build command: docker build -t octave .
Sending build context to Docker daemon 423.9 kB
Sending build context to Docker daemon
Step 0 : FROM debian:jessie
---> 58052b122b60
Step 1 : MAINTAINER GG_Python <[..]#gmail.com>
---> Using cache
---> 90d2dd2f7ee8
Step 2 : RUN apt-get update
---> Using cache
---> 4c72c25cd829
Step 3 : RUN apt-get update
---> Using cache
---> b52f0bcb9f86
Step 4 : RUN apt-get install -y octave octave-image octave-missing-functions octave-nan octave-statistics
---> Using cache
---> f0637ab96d5e
Step 5 : RUN mkdir /octave
---> Using cache
---> a2d278b2819b
Step 6 : RUN mkdir /octave/libs
---> Using cache
---> 65efbbe01c99
Step 7 : RUN mkdir /octave/libs/jsonlab
---> Using cache
---> e41b80901266
Step 8 : COPY ~/octave/jsonlab/loadjson.m /octave/libs/jsonlab/.
INFO[0000] ~/octave/jsonlab/loadjson.m: no such file or directory
Docker absolutely refuses to copy this file from the host into the image. Needless to say a the file loadjson.m is there (cat displays), all my attempts to change the path (relative, absolute, etc.) failed. Any advice why this simple task is problematic?
At the time I originally wrote this, Docker didn’t expand ~ or $HOME. Now it does some expansions inside the build context, but even so they are probably not what you want—they aren’t your home directory outside the context. You need to reference the file explicitly, or package it relative to the Dockerfile itself.
Docker can only copy files from the context, the folder you are minus any file listed in the dockerignore file.
When you run 'docker build' docker tars the context and it sends it to the docker daemon you are connected to. It only lets you copy files inside of the context because the daemon might be a remote machine.
I couldn't get COPY to work until I understood the context (I was trying to copy a file from outside of the context)
The docker build command builds an image from a Dockerfile and a context. The build’s context is the files at a specified location PATH. The PATH is a directory on your local filesystem.
A context is processed recursively. So, a PATH includes any subdirectories.
The build is run by the Docker daemon, not by the CLI. The first thing a build process does is send the entire context (recursively) to the daemon. In most cases, it’s best to start with an empty directory as context and keep your Dockerfile in that directory. Add only the files needed for building the Dockerfile.
Warning: Do not use your root directory, /, as the PATH as it causes the build to transfer the entire contents of your hard drive to the Docker daemon.
Reference:
https://docs.docker.com/engine/reference/builder/#usage
I had similar issue. I solved it by checking two things:
Inside your, docker-compose.yaml check context of the service, docker will not copy any file outside of this directory. For example if the context is app/ then you cannot copy anything from ../app
Check .dockerignore to be sure that you are not ignoring the file you want to copy.
I got it working by first checking what the context was,
setting an absolute path before the source file in
your Dockerfile to get that information:
# grep COPY Dockerfile
COPY /path/to/foo /whatever/path/in/destination/foo
Building with that:
docker build -t bar/foo .
you'll get an error, which states the context-path that Docker
is apparently looking into for its files, e.g.
it turns out to be:
/var/lib/docker/tmp # I don't remember
Copying(!) your set of build-files in that directory (here: /var/lib/docker/tmp),
cd into it, build from there.
See if that works, and don't forget to do some housekeeping cleaning up
the tmp, deleting your files before the next visit(or).
HTH
Michael
Got this error using a Dockerfile for a linux container on a Windows machine:
#24 1.160 Skipping project "/src/Common/MetaData/Metadata.csproj" because it was not found.
Restore worked perfectly on the host machine.
Turned out to be the error mentioned here:
https://stackoverflow.com/a/68592423/3850405
A .csproj file did not match casing in Visual Studio vs the file system.
Related
There is a directory inside the stain/jena-fuseki:4.0.0 image that cannot be copied while other directories can be. I have the following Dockerfile
FROM python:3.8.15-slim
COPY --from=stain/jena-fuseki:4.0.0 /fuseki /fuseki
If I run docker image build . I get the following response
Sending build context to Docker daemon 435.9MB
Step 1/2 : FROM python:3.8.15-slim
---> f0fe0cb74bac
Step 2/2 : COPY --from=stain/jena-fuseki:4.0.0 /fuseki /fuseki
COPY failed: stat fuseki: file does not exist
However, I looked into the image with docker run -it stain/jena-fuseki:4.0.0 and the directory does exist at the root level along with other directories which are copyable. E.g. following Dockerfile builds perfectly without any errors.
FROM python:3.8.15-slim
COPY --from=stain/jena-fuseki:4.0.0 /jena-fuseki /jena-fuseki
I have tried many things like changing the working directory with WORKDIR / and also things like COPY --from=stain/jena-fuseki:4.0.0 /fuseki/. /fuseki. However, none of them are working.
I have also not excluded anything with .dockerignore
fuseki is a run time directory. It is created when the container is instantiated. So, it is not present at the build time. Hence the error.
This is proved by the timestamp of the files in the screenshot below.
You need to use a multi-stage build, see here: https://docs.docker.com/build/building/multi-stage/#name-your-build-stages
In your case, that would like something like:
FROM stain/jena-fuseki:4.0.0 AS jena
FROM python:3.8.15-slim
COPY --from=jena /jena-fuseki /jena-fuseki
I have several projects linked to Github repositories that build on Docker Hub via a Dockerfile in each project. This arrangement has been working fine, but all my builds started failing 100% of the time a while back. I've been trying to trace it and realized I didn't really do anything and the issue it fails on is quite basic.
I'm using the archlinux base image and copying the source of my app to the container using something like:
COPY ./ /src
Next I move into that directory:
WORKDIR /src
Then I try to build the software. Normally this would involve running ./configure, but that dies with a weird error saying "Operation not permitted". It turn out this isn't just an issue for GNU Make, even ls can't read the directory! Specifically the directory, not the stuff inside it. I can run RUN ls -l and get a list of files in the source that got copied over, but I can't run RUN ls -ld to show the directory properties, it dies like this:
Step 17/28 : RUN ls -ald
---> Running in ec8f9f6c3604
ls: cannot access '.': Operation not permitted
Removing intermediate container ec8f9f6c3604
The command '/bin/sh -c ls -ald' returned a non-zero code: 2
I can run various other commands, but anything that tries to look at the directory itself dies like this. I can even create files in the directory.
Note these same Dockerfiles build just fine on my local system using docker build. What gives? What is different about building them on Docker Hub?
i would like to build my first Docker image, containing Apache Tomcat and a deployed web app. My Dockerfile is really small, based on Tomcat:8.0 image and is supposed to copy a WAR file into Tomcat's appBase.
Build of the image reports success, but the file is nowhere to be found in the container.
Copying from host to the container work w/o issues using "docker cp":
[root#edubox dock]# docker cp jdbcdemo_3.war 15dd44bbf992:/usr/local/tomcat/webapps/
My Dockerfile:
# we are extending everything from tomcat:8.0 image ...
FROM tomcat:8.0
MAINTAINER simo
# COPY path-to-your-application-war path-to-webapps-in-docker-tomcat
COPY ./jdbcdemo_3.war /usr/local/tomcat/webapps/
EXPOSE 8082
Image build:
root#edubox dock]# docker image build -t simo/jdbcdemo_3 --tag=recent ./
Sending build context to Docker daemon 10.24 kB
Step 1/4 : FROM tomcat:8.0
---> ef6a7c98d192
Step 2/4 : MAINTAINER simo
---> Using cache
---> 54d824d7258b
Step 3/4 : COPY ./jdbcdemo_3.war /usr/local/tomcat/webapps/
---> Using cache
---> f94330423a93
Step 4/4 : EXPOSE 8082
---> Running in 74b6dd0364b2
---> 9464f11ac18e
Removing intermediate container 74b6dd0364b2
Successfully built 9464f11ac18e
I would expect COPY to place the file where specified or an error message because of which this does not work.
Please try this way,
Keep the "jdbcdemo_3.war" where your Dockerfile exists. And make this change in the Dockerfile.
(remove ./ from your Dockerfile) like,
COPY jdbcdemo_3.war /usr/local/tomcat/webapps/
please check the permission side of the file.
you can give full permission and test once. (user:group)
try this: COPY jdbcdemo_3.war /tmp
In your Dockerfile. And build the image and check in the /tmp directory. If the file copied successfully, give the permission to /usr/local/tomcat/webapps/ Or copy to /tmp first and then copy from /tmp to /usr/local/tomcat/webapps/. Using COPY command in Dockerfile
Hi and many thanks for offering advice. The issue has been trivial in the end. I have not been examining the correct container.
I did not realize one needs to pick the freshly created image, run the container with this image and only afterwards peek for changes described in the Dockerfile in that container.
I have been looking into parent container which i now understand could not have worked.
Sry for wasting your time ;-)
I have a simple Java server app with a Gradle build. It works perfectly with gradle run on my host machine. However, I want to build this in a docker image and run as a docker container.
I'm using docker-machine (version 0.13.0):
docker-machine create --driver virtualbox --virtualbox-memory 6000 default
docker-machine start
eval $(docker-machine env default)
I have the following Dockerfile image build script in ./serverapp/Dockerfile:
FROM gradle:4.3-jdk-alpine
ADD . /code
WORKDIR /code
CMD ["gradle", "--stacktrace", "run"]
I can build perfectly:
➜ docker build -t my-server-app .
Sending build context to Docker daemon 310.3kB
Step 1/4 : FROM gradle:4.3-jdk-alpine
---> b803ec92baec
Step 2/4 : ADD . /code
---> Using cache
---> f458b0be79dc
Step 3/4 : WORKDIR /code
---> Using cache
---> d98d04eda627
Step 4/4 : CMD ["gradle", "--stacktrace", "run"]
---> Using cache
---> 869262257870
Successfully built 869262257870
Successfully tagged my-server-app:latest
When I try to run this image:
➜ docker run --rm my-server-app
FAILURE: Build failed with an exception.
* What went wrong:
Could not create service of type ScriptPluginFactory using BuildScopeServices.createScriptPluginFactory().
> Could not create service of type CrossBuildFileHashCache using BuildSessionScopeServices.createCrossBuildFileHashCache().
* Try:
Run with --info or --debug option to get more log output.
* Exception is:
org.gradle.internal.service.ServiceCreationException: Could not create service of type ScriptPluginFactory using BuildScopeServices.createScriptPluginFactory().
at org.gradle.internal.service.DefaultServiceRegistry$FactoryMethodService.invokeMethod(DefaultServiceRegistry.java:797)
<snip>
... 60 more
Caused by: org.gradle.api.UncheckedIOException: Failed to create parent directory '/code/.gradle/4.3' when creating directory '/code/.gradle/4.3/fileHashes'
at org.gradle.util.GFileUtils.mkdirs(GFileUtils.java:271)
at org.gradle.cache.internal.DefaultPersistentDirectoryStore.open(DefaultPersistentDirectoryStore.java:56)
Why would it have trouble creating that directory?
This should be a very easy task, can anyone tell me how they get this simple scenario working?
FYI, running current versions of everything. I'm using Gradle 4.3.1 on my host, and the official Gradle 4.3 base image from docker hub, I'm using the current version of JDK 8 on my host and the current version of docker, docker-machine, and docker-compose as well.
The fix was to specify --chown=gradle permissions on the /code directory in the Dockerfile. Many Docker images are designed to run as root, the base Gradle image runs as user gradle.
FROM gradle:4.3-jdk-alpine
ADD --chown=gradle . /code
WORKDIR /code
CMD ["gradle", "--stacktrace", "run"]
Ethan Davis suggested using /home/gradle rather than code. That would probably work as well, but I didn't think of that.
The docker image maintainer should have a simple getting started type reference example that shows the recommended way to get basic usage.
Based on the openjdk base image to the gradle image we can see that gradle projects are setup to run in /home/gradle. Check the code out here. gradle run is having trouble running in your new working directory, /code, because the .gradle folder is in the /home/gradle folder. If you copy/add your code into /home/gradle you should be able to run gradle run. This worked for me.
At windows 10, when I run:
$ docker build .
I get this error in console:
RUN /provision/provision.sh
---> Running in e66928c3c893
/bin/sh: 1: /provision/provision.sh: not found
My folder files:
/Dockerfile
/provision
/provision/provision.sh
So why I get /bin/sh: 1: /provision/provision.sh: not found error, although /provision/provision.sh exists ?
EDIT
I tried:
ADD provision /provision
And I tried:
COPY provision /provision
Didn't work..
Step 7 : COPY provision /provision
---> c5d07de6948d
Removing intermediate container 85768981a7f0
Step 8 : RUN /provision/provision.sh
---> Running in 8426a8526514
/bin/sh: 1: /provision/provision.sh: not found
SECURITY WARNING: You are building a Docker image from Windows against a non-Windows Docker host. All files and directories added to build context will have '-rwxr-xr-x' permissions. It is recommended to double check and reset permissions for sensitive files and directories.
The command '/bin/sh -c /provision/provision.sh' returned a non-zero code: 127
provision/provision.sh exists in your docker build context (on your host), but you need to copy (COPY) it first to the image:
COPY provision/provision.sh /provision/provision.sh
# or
COPY provision /provision/
RUN chmod 755 /provision/provision.sh && \
/provision/provision.sh
As mentioned in the doc:
If <src> is a directory, the entire contents of the directory are copied, including filesystem metadata.
Note: The directory itself is not copied, just its contents.
RUN will execute a command in an intermediate container launched with the Dockerfile content compiled up to that line.
It means it execute something in the context of the container, not in the context of your host.
In my case, the provision.sh file was a dos line ending file. I needed to re-save the file using Unix line endings.
I found those two answers very helpful to fix the problem:
RUN rm /bin/sh && ln -s /bin/bash /bin/sh
at Using the RUN instruction in a Dockerfile with 'source' does not work
And this:
./configure : /bin/sh^M : bad interpreter