I am trying to use a custom Dockerfile to build the LUIS container and copy the app file (app exported from the Luis portal) into the container. For this reason, I really don't need the mount points, since the .gz file will already live in the container. Is this possible? It seems that the mount points are always required...
I have to copy the files into the container and the move them to the input location at runtime (using an init.sh script). But, even then the container seemed to not load the app correctly. It behaves differently from that scenario compared to just putting the file in the host folder and mounting that to the container.
UPDATE: When I try to move the files around internally (at the start of the container), LUIS gives this output:
Using '/input' for reading models and other read-only data.
Using '/output/luis/fbfb798892fd' for writing logs and other output data.
Logging to console.
Submitting metering to 'https://southcentralus.api.cognitive.microsoft.com/'.
warn: Microsoft.AspNetCore.Server.Kestrel[0]
Overriding address(es) 'http://+:80'. Binding to endpoints defined in UseKestrel() instead.
Hosting environment: Production
Content root path: /app
Now listening on: http://0.0.0.0:5000
Application started. Press Ctrl+C to shut down.
fail: Luis[0]
Failed while prefetching App: AppId: d6fa5fd3-c32a-44d5-bb7f-d563775cf6ee - Slot: PRODUCTION Could not find file '/input/d6fa5fd3-c32a-44d5-bb7f-d563775cf6ee_PRODUCTION.gz'.
fail: Luis[0]
Failed while getting response for AppId: d6fa5fd3-c32a-44d5-bb7f-d563775cf6ee - Slot: PRODUCTION. Error: Could not find file '/input/d6fa5fd3-c32a-44d5-bb7f-d563775cf6ee_PRODUCTION.gz'.
warn: Microsoft.CloudAI.Containers.Controllers.LuisControllerV3[0]
Response status code: 404
Exception: Could not find file '/input/d6fa5fd3-c32a-44d5-bb7f-d563775cf6ee_PRODUCTION.gz'. SubscriptionId='' RequestId='d7dfee25-05d9-4af6-804d-58558f55465e' Timestamp=''
^C
nuc#nuc-NUC8i7BEK:/tmp/input$ sudo docker exec -it luis bash
root#fbfb798892fd:/app# cd /input
root#fbfb798892fd:/input# ls
d6fa5fd3-c32a-44d5-bb7f-d563775cf6ee_production.gz
root#fbfb798892fd:/input# ls -l
total 8
-rwxrwxrwx 1 root root 4960 Dec 2 17:35 d6fa5fd3-c32a-44d5-bb7f-d563775cf6ee_production.gz
root#fbfb798892fd:/input#
Notice that even though I can log into the container and browse the location of the model files and they are present, LUIS cannot load/find them.
UPDATE #2 - posting my Dockerfile:
FROM mcr.microsoft.com/azure-cognitive-services/luis:latest
ENV Eula=accept
ENV Billing=https://southcentralus.api.cognitive.microsoft.com/
ENV ApiKey=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
ENV Logging:Console:LogLevel:Default=Debug
RUN mkdir /app/inputfiles/
RUN chmod 777 /app/inputfiles/
COPY *.gz /app/inputfiles/
WORKDIR /app
COPY init.sh .
RUN chmod 777 /app/init.sh
ENTRYPOINT /app/init.sh && dotnet Microsoft.CloudAI.Containers.Luis.dll
Option 1
The models can be COPY'd directly into /input/.
e.g.
FROM mcr.microsoft.com/azure-cognitive-services/luis:latest
COPY *.gz /input/
This will work, but requires that you don't mount to /input at runtime as it will squash the COPY'd files. The message "A folder must be mounted" is only logged if the /input directory does not exist.
> docker build . -t luis --no-cache
Sending build context to Docker daemon 40.43MB
Step 1/2 : FROM aicpppe.azurecr.io/microsoft/cognitive-services-luis
---> df4e32e45b1e
Step 2/2 : COPY ./*.gz /input/
---> c5f41a9d8522
Successfully built c5f41a9d8522
Successfully tagged luis:latest
> docker run --rm -it -p 5000:5000 luis eula=accept billing=*** apikey=***
...
Using '/input' for reading models and other read-only data.
...
Application started. Press Ctrl+C to shut down.
Option 2
The configuration value Mounts:Input can be set to configure the input location.
This might be useful if you need your models to live in /app/inputfiles or if you need to mount to /input for another reason at runtime.
e.g.
FROM aicpppe.azurecr.io/microsoft/cognitive-services-luis
ENV Mounts:Input=/app/inputfiles
COPY ./*.gz /app/inputfiles/
This results in:
> docker build . -t luis --no-cache
Sending build context to Docker daemon 40.43MB
Step 1/3 : FROM aicpppe.azurecr.io/microsoft/cognitive-services-luis
---> df4e32e45b1e
Step 2/3 : ENV Mounts:Input=/app/inputfiles
---> Running in b6029a2b54d0
Removing intermediate container b6029a2b54d0
---> cb9a4e06463b
Step 3/3 : COPY ./*.gz /app/inputfiles/
---> 9ab1dfaa36e7
Successfully built 9ab1dfaa36e7
Successfully tagged luis:latest
> docker run --rm -it -p 5000:5000 luis eula=accept billing=*** apikey=***
...
Using '/app/inputfiles' for reading models and other read-only data.
...
Application started. Press Ctrl+C to shut down.
It's true that the input mount won't be necessary if your .gz file is already in the image, but the output mount is used for logging and you may still want that for active learning purposes.
To build your desired image, create a text file named Dockerfile (no extension) and populate it with the following lines:
FROM mcr.microsoft.com/azure-cognitive-services/luis:latest
ENV Eula=accept
ENV Billing={ENDPOINT_URI}
ENV ApiKey={API_KEY}
COPY ./{luisAppId}_PRODUCTION.gz /input/{luisAppId}_PRODUCTION.gz
You can find your {ENDPOINT_URI} and your {API_KEY} using the normal LUIS container instructions, and {luisAppId} will be found in the name of your .gz file of course. Once your Dockerfile is ready, run it from the same folder that contains your .gz file with this command:
docker build -t luis .
Your image will now be ready. All your teammate has to do is run this command:
docker run --rm -it -p 5000:5000
--memory 4g
--cpus 2
--mount type=bind,src={OUTPUT_FOLDER},target=/output luis
{OUTPUT_FOLDER} can be any local absolute path you want as long as it exists. You may also omit the output mount if you don't want any logging:
docker run --rm -it -p 5000:5000 --memory 4g --cpus 2 luis
Related
I am creating a docker container that will run a minecraft server. (Yes i know, these already exist). And of course i want the world to be saved when the container is turned off.
This is my dockerfile:
FROM anapsix/alpine-java
COPY ./ /home
CMD ["java","-jar","/home/main.jar"]
EXPOSE 25565
Then i build the container:
docker build -t minecraftdev .
Run the container:
docker run -dp 25565:25565 -v C:/Users/user/server:/home minecraftdev
And then the files in the image, server.properies, the server jar file and EULA.txt is wiped.
Is there another way i don't now of to get the container to store data? And this is without placing the files in the server folder.
Thank you for your answers, i was able to fix it by -v C:/Users/user/server/world:/home/world As the world files are stored in that folder, Instead of changing out all the files in the folder as i didn't know -v did.
Minecraft makes the server.jar file and i don't know how to change so it stores all the files in another place.
End of my Dockerfile:
ENTRYPOINT ["ls /etc"]
Terminal:
...Rest of the building above is fine
Step 8/8 : ENTRYPOINT ["ls /etc"]
---> Using cache
---> ea1f33b8ab22
Successfully built ea1f33b8ab22
Successfully tagged redis:latest
k#Karls ~/dev/docker_redis (master) $ docker run -d -p 6379:6379 --name red redis
71d75058b94f088ef872b08a115bc12cece288b53fe26d67960fe139953ed5c4
docker: Error response from daemon: OCI runtime create failed: container_linux.go:296: starting container process caused "exec: \"ls /etc\": stat ls /etc: no such file or directory": unknown.
For some reason, it won't find the directory /etc. I did a pwd and the current working directory is /. I also did a ls / on the entrypoint and that displayed the /etc directory fine.
OCI runtime create failed: container_linux.go:296
In my experience this is an error with the docker daemon itself, not the container you are trying to run. Try deleting all containers, restarting the daemon. I think we also had to clean up the docker networks.
I appear to be having the same issue. Here is what I am doing.
Dockerfile
FROM gcc:7.2.0
COPY src/ /usr/src/myapp
WORKDIR /usr/src/myapp
RUN set -x gcc -o myapp main.c
CMD ["./myapp"]
Build
$ docker build -t test .
Sending build context to Docker daemon 3.584kB
Step 1/6 : FROM gcc:7.2.0
...
---> 3ec35c7d2396
Successfully built 3ec35c7d2396
Successfully tagged test:latest
SECURITY WARNING: You are building a Docker image from Windows against a
non-Windows Docker host. All files and directories added to build context
will have '-rwxr-xr-x' permissions. It is recommended to double check and
reset permissions for sensitive files and directories.
Run
$ docker run -it test
D:\Docker Toolbox\docker.exe: Error response from daemon: OCI runtime create
failed: container_linux.go:296: starting container process caused "exec:
\"./myapp\": stat ./myapp: no such file or directory": unknown.
Changed CMD to ENTRYPOINT and removed the set -x seemed to resolve the problem. Though we are still unsure what the cause was or if this will also work for you.
Make sure that /etc exists or is created as the main.c wasn't compiling.
Dockerfile
FROM gcc:7.2.0
COPY src/ /usr/src/myapp
WORKDIR /usr/src/myapp
RUN gcc -o myapp main.c
ENTRYPOINT ["./myapp"]
On OSX, I fixed it by clearing the volume data manually. Close docker, and remove everything in ~/Library/Containers/com.docker.docker
I've expirienced the same issue after updating my Windows credentials, try following: Docker settings > Shared Drives > Reset credentials > Select drives again > Apply and re-enter your credentials. This solved the problem for me multiple times
The command you are trying to execute inside the container does not exist. In this case ls /etc does not exist in the image. There's a /bin/ls binary, but not a /bin/"ls /etc" binary, which itself would be invalid since the name of a file on the filesystem cannot include a /, though it can include a space.
Of course what you wanted to run was ls with the argument /etc, and for that, you need to separate each argument if you run with the exec syntax.
ENTRYPOINT ["ls", "/etc"]
Or if you wanted to allow a shell to parse the string, same way as if you were at a bash prompt inside the container running ls /etc on the command line, then switch to the string syntax that runs a shell:
ENTRYPOINT ls /etc
Requirement: when an new version of J2EE application go into production, because development environment and production is not the same, the DBA has to replace some configuration file in war package, those files which need to replace typically contains some sensitive data (like database account, password).
I think is a good idea to create a data volume container, which contain those configuration files that specific to production. In this case, configuration files can be share between containers (applications)'
Let say I have and J2EE application run with docker using tomcat 8, the dockerfile as follows:
FROM tomcat:8
WORKDIR $CATALINA_HOME
RUN midair -p /etc/foo
RUN touch /etc/foo/a
RUN touch /etc/foo/b
RUN touch /etc/foo/c
RUN touch /etc/foo/d
RUN echo "a" >> /etc/foo/a
RUN echo "b" >> /etc/foo/b
RUN echo "c" >> /etc/foo/c
RUN echo "d" >> /etc/foo/d
CMD ["catalina.sh", "run"]
The DBA has to replace the file b and c before the application go into production, as a result, we have a dockerfile as follow:
FROM centos:6.8
RUN mkdir -p /etc/foo
RUN touch /etc/foo/a
RUN touch /etc/foo/d
RUN echo "sub a" >> /etc/foo/a
RUN echo "sub d" >> /etc/foo/d
COPY ./run.sh /root
RUN chmod 755 /root/run.sh
CMD ["/root/run.sh"]
To test whether the data volume container would satisfy my requirement, the following command is run :
docker create -v /etc/foo --name configstore centos:6.8 /bin/bash
docker run -d --volumes-from configstore --name testsubcontainer tomcat:8
And finally I found that the “tomcat” container has “a” and “d” in the /etc/foo b and c has gone.
Q1: How to replace only the files not the same, instead of the whole directory when using data volume container as a share storage for other containers.
Q2: Is there any other better solution to satisfy my requirement
Q1: Using volumes as you are, this is not possible. When you mount a volume to a directory in a container, everything inside that directory is replaced by the data volume. Docker isn't mapping files, it's mounting the volume at a certain point in the container filesystem.
Q2: Have you explored ENV or ARG as a means to set values at runtime?
FROM ansible/ansible:ubuntu1604
MAINTAINER myname
RUN git clone http://github.com/ansible/ansible.git /tmp/ansible
RUN git clone http://github.com/othertest.git /tmp/othertest
WORKDIR /tmp/ansible
ENV PATH /tmp/ansible/bin:/sbin:/usr/sbin:/usr/bin:bin
ENV PYTHONPATH /tmp/ansible/lib:$PYTHON_PATH
ADD inventory /etc/ansible/hosts
WORKDIR /tmp/
EXPOSE 8888
When I build from this dockerfile, I get Cloning into /tmp/ansible and othertest in red text (I assume is an error). When I then run the container and peruse around, I see that all my steps from the dockerfile built correctly other than the git repositories which are missing.
I can't figure out what I'm doing wrong, I'm assuming its a simple mistake.
Building dockerfile:
sudo docker build --no-cache -f Dockerfile .
Running dockerfile:
sudo docker run -I -t de32490234 /bin/bash
The short answer:
Put your files anywhere other than in /tmp and things should work fine.
The longer answer:
You're basing your image on the ansible/ansible:ubuntu1604 image. If you inspect this image via docker inspect ansible/ansible:ubuntu1604 or look at the Dockerfile from which it was built, you will find that it contains a number of volume mounts. The relevant line from the Dockerfile is:
VOLUME /sys/fs/cgroup /run/lock /run /tmp
That means that all of those directories are volume mount points, which means any data placed into them will not be committed as part of the image build process.
Looking at your Dockerfile, I have two comments unrelated to the above:
You're explicitly setting the PATH environment variable, but you're neglecting to include /bin, which will cause all kinds of problems, such as:
$ docker run -it --rm bash
docker: Error response from daemon: oci runtime error: exec: "bash": executable file not found in $PATH.
You're using WORKDIR twice, but the first time (WORKDIR /tmp/ansible) you're not actually doing anything that cares what directory you're in (you're just setting some environment variables and copying a file into /etc/ansible).
I'm trying to write a docker image to run a simple webserver though netcat.
So I have in my docker build folder:
Dockerfile
index.html
run_netcat_webserver.sh
The run_netcat_webserver.sh is very simple and it works fine:
#!/bin/bash
while true ; do nc -l 8080 < index.html ; done
Here is my naive Dockerfile that of course is not working:
FROM ubuntu:14.04
CMD run_netcat_webserver.sh
How should I proceed to make this work inside a docker container?
You need to make the script part of the container. To do that, you need to copy the script inside using the COPY command in the Docker file, e.g. like this
FROM ubuntu:14.04
COPY run_netcat_webserver.sh /some/path/run_netcat_webserver.sh
CMD /some/path/run_netcat_webserver.sh
The /some/path is a path of your choice inside the container. Since you don't need to care much about users inside the container, it can be even just /.
Another option is to provide the script externally, via mounted volume. Example:
FROM ubuntu:14.04
VOLUME /scripts
CMD /scripts/run_netcat_webserver.sh
Then, when you run the container, you specify what directory will be mounted as /scripts. Let's suppose that your script is in /tmp, then you run the container as
docker run --volume=/tmp:/scripts (rest of the command options and arguments)
This would cause that your (host) directory /tmp would be mounted under the /scripts directory of the container.
Simple example
Write a "Dockerfile"
FROM ubuntu:14.04 - # image you are using for building the container
ENTRYPOINT ["/bin/echo"] - # command that will execute when container is booted
Verify that you are in the same directory as the "Dockerfile".
sudo docker build .
and you are done
sample output:
ahanjura#ubuntu:~$ vi Dockerfile
ahanjura#ubuntu:~$ sudo docker build .
Sending build context to Docker daemon 1.39 GB
Step 1 : FROM ubuntu:14.04
---> 67759a80360c
Step 2 : ENTRYPOINT /bin/echo
---> Running in c0e5f48f7d45
---> 7a96b36f528e
Removing intermediate container c0e5f48f7d45
Successfully built 7a96b36f528e