I have the following Dockerfile
############################################################
# Purpose : Dockerize Django App to be used in AWS EC2
# Django : 1.8.1
# OS : Ubuntu 14.04
# WebServer : nginx
# Database : Postgres inside RDS
# Python : 2.7
# VERSION : 0.1
############################################################
from ubuntu:14.04
maintainer Kim Stacks, kimcity#gmail.com
# make sure package repository is up to date
run echo "deb http://archive.ubuntu.com/ubuntu $(lsb_release -sc) main universe" > /etc/apt/sources.list
run apt-get update
# install python
# install nginx
Inside my VM, I did the following:
docker build -t ubuntu1404/djangoapp .
It is successful.
What do I do to run the docker image?
Where is the image or container?
I have already tried running
docker run ubuntu1404/djangoapp
Nothing happens.
What I see when I run docker images
root#vagrant-ubuntu-trusty-64:/var/virtual/Apps/DockerFiles/Django27InUbuntu# docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
ubuntu1404/djangoapp latest cfb161605c8e 10 minutes ago 198.3 MB
ubuntu 14.04 07f8e8c5e660 10 days ago 188.3 MB
hello-world latest 91c95931e552 3 weeks ago 910 B
When I run docker ps, nothing shows up
You have to give a command your container will have to process.
Example : sh
you could try :
docker run -ti yourimage sh
(-ti is used to keep a terminal open)
If you want to launch a daemon (like a server), you will have to enter something like :
docker run -d yourimage daemontolaunch
Use docker help run for more options.
You also can set a default behaviour with CMD instruction in your Dockerfile so you won't have to give this command to your container each time you want to run it.
EDIT - about container removing :
Containers and images are different.
A container is an instance of an image.
You can run several containers from the same image.
The container automatically stops when the process it runs terminates.
But the container isn't deleted (just stopped, so you can restart it).
But if you want to remove it (removing a container doesn't remove the image) you have two ways to do :
automatically removing it at the end of the process by adding --rm option to docker run.
Manually removing it by using the docker rm command and giving it the container ID or its name (a container has to be stopped before being removed, use docker stop for this).
A usefull command :
Use docker ps to list containers. -q to display only the container IDs, -a to display even stopped containers.
More here.
EDIT 2:
This could also help you to discover docker if you didn't try it.
How to test the container or image after docker build?
In order to test you can add write a bash script which will do the job https://blog.brazdeikis.io/posts/docker-image-tests
Btw, from the post, I see that it does not match the question from the title.
So, Added a link for the souls who arrived here based on the title...
Download the latest shaded dist from https://github.com/dgroup/docker-unittests/releases:
wget https://github.com/dgroup/docker-unittests/releases/download/s1.1.1/docker-unittests-app-1.1.1.jar
De fine an *.yml file with tests.
version: 1.1
setup:
- apt-get update
- apt-get install -y tree
tests:
- assume: java version is 1.9, Debian build
cmd: java -version
output:
contains:
- openjdk version "9.0.1"
- build 9.0.1+11-Debian
- assume: curl version is 7.xxx
cmd: curl --version
output:
startsWith: curl 7.
matches:
- "^curl\\s7.*\\n.*\\nProtocols.+ftps.+https.+telnet.*\\n.*\\n$"
contains:
- AsynchDNS IDN IPv6 Largefile GSS-API
- assume: Setup section installed `tree`
cmd: tree --version
output:
contains: ["Steve Baker", "Florian Sesser"]
Run tests for image
java -jar docker-unittests.jar -f image-tests.yml -i openjdk:9.0.1-11
https://i.stack.imgur.com/DSv72.png
Related
I have a dockerfile image based on ubuntu. Iam trying to make a bash script run each day but the cron never runs. When the container is running, i check if cron is running and it is. the bash script works perfectly and the crontab command is well copied inside the container. i can't seem to find where the problem is coming from.
Here is the Dockerfile:
FROM snipe/snipe-it:latest
ENV TZ=America/Toronto
RUN apt-get update \
&& apt-get install awscli -y \
&& apt-get clean \
&& apt-get install cron -y \
&& rm -rf /var/lib/apt/lists/*
RUN mkdir /var/www/html/backups_scripts /var/www/html/config/scripts
COPY config/crontab.txt /var/www/html/backups_scripts
RUN /usr/bin/crontab /var/www/html/backups_scripts/crontab.txt
COPY config/scripts/backups.sh /var/www/html/config/scripts
CMD ["cron","-f"]
The last command CMD doesn't work. And as soon as i remove the cmd command i get this message when i check the cron task inside the container:
root#fcfb6052274a:/var/www/html# /etc/init.d/cron status
* cron is not running
Even if i start the cron process before the crontab, the crontab is still not launched
This dockerfile is called by a docker swarm file (compose file type). Maybe the cron must be activated with the compose file.
How can i tackle this problem ??? Thank you
You need to approach this differently, as you have to remember that container images and containers are not virtual machines. They're a single process that starts and is maintained through its lifecycle. As such, background processes (like cron) don't exist in a container.
What I've seen most people do is have the container just execute whatever you're looking for it to do on a job like do_the_thing.sh and then using the docker run command on on the host machine to call it via cron.
So for sake of argument, let's say you had an image called myrepo/task with a default entrypoint of do_the_thing.sh
On the host, you could add an entry to crontab:
# m h dom mon dow user command
0 */2 * * * root docker run --rm myrepo/task
Then it's down to a question of design. If the task needs files, you can pass them down via volume. If it needs to put something somewhere when it's done, maybe look at blob storage.
I think this question is a duplicate, with a detailed response with lots of upvotes here. I followed the top-most dockerfile example without issues.
Your CMD running cron in the foreground isn't the problem. I ran a quick version of your docker file and exec'ing into the container I could confirm cron was running. Recommend checking how your cron entries in the crontab file are re-directing their output.
Expanding on one of the other answers here a container is actually a lot like a virtual machine, and often they do run many processes concurrently. If you happen to have any other containers running you might be able to see this most easily by running docker stats and looking at the PID column.
Also, easy to examine interactively yourself like this:
$ # Create a simple ubuntu running container named my-ubuntu
$ docker run -it -h my-ubuntu ubuntu
root#my-ubuntu$ ps aw # Shows bash and ps processes running.
root#my-ubuntu$ # Launch a ten minute sleep in the background.
root#my-ubuntu$ sleep 600 &
root#my-ubuntu$ ps aw # Now shows sleep also running.
I'm trying to use the tf-sentencepiece operation in my model found here https://github.com/google/sentencepiece/tree/master/tensorflow
There is no issue building the model and getting a saved_model.pb file with variables and assets. However, if I try to use the docker image for tensorflow/serving, it says
Loading servable: {name: model version: 1} failed:
Not found: Op type not registered 'SentencepieceEncodeSparse' in binary running on 0ccbcd3998d1.
Make sure the Op and Kernel are registered in the binary running in this process.
Note that if you are loading a saved graph which used ops from tf.contrib, accessing
(e.g.) `tf.contrib.resampler` should be done before importing the graph,
as contrib ops are lazily registered when the module is first accessed.
I am unfamiliar with how to build anything manually, and was hoping that I could do this without many changes.
One approach would be to:
Pull a docker development image
$ docker pull tensorflow/serving:latest-devel
In the container, make your code changes
$ docker run -it tensorflow/serving:latest-devel
Modify the code to add the op dependency here.
In the container, build TensorFlow Serving
container:$ tensorflow_serving/model_servers:tensorflow_model_server && cp bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server /usr/local/bin/
Use the exit command to exit the container
Look up the container ID:
$ docker ps
Use that container ID to commit the development image:
$ docker commit $USER/tf-serving-devel-custom-op
Now build a serving container using the development container as the source
$ mkdir /tmp/tfserving
$ cd /tmp/tfserving
$ git clone https://github.com/tensorflow/serving .
$ docker build -t $USER/tensorflow-serving --build-arg TF_SERVING_BUILD_IMAGE=$USER/tf-serving-devel-custom-op -f tensorflow_serving/tools/docker/Dockerfile .
You can now use $USER/tensorflow-serving to serve your image following the Docker instructions
I'm running this docker image to use the TICK Kapacitor locally.
The problem I face is that when I try to use User Defined Functions, e.g any of these examples I get the error message that /usr/bin/python2 does not exist.
I add the following to the kapacitor.conf:
[udf.functions]
[udf.functions.tTest]
prog = "/usr/bin/python2"
args = ["-u", "/tmp/kapacitor_udf/mirror.py"]
timeout = "10s"
[udf.functions.tTest.env]
PYTHONPATH = "/tmp/kapacitor_udf/kapacitor/udf/agent/py"
Further attempts from my side including altering the image used to build Kapacitor to install python works but the agent seems to fail to compile anyway.
Is there anyone who managed to get UDFs running using the Kapacitor Docker image?
Thanks
Docker image from the official repository: docker pull kapacitor does not have python installed inside. You can verify this by running shell in the container:
PS> docker exec -it kapacitor bash
and execute one of the command options:
$ python -VERSION
$ python: command not found
or
$ readlink -f $(which python) | xargs -I% sh -c 'echo -n "%:"; % -V'
$ readlink: missing operand
or
$ find / -type f -executable -iname 'python *'
void returns. And oppositely if python is available, commands return version and list of executable files
Note: Here and further all command snippets are given for Powershell on Windows. And all command snippets inside docker container are given for bash shell as Linux images are used.
Basicly, there is two options to get kapacitor image with python inside to execute UDFs:
Install the python in the kapacitor image, i.e. build new docker image from very kapacitor image.
Example could be found here:
Build a new verion of kapacitor image from one of the python official images
The second option is more natural as you get consistent python installation and keep efforts on doing work of installing python which already done by the docker community.
So following option 2 we'll perform:
Examine the Dockefile of the official kapacitor image
Choose an appropriate python image
Create new project and Dockerfile for kapacitor
Build and Check the kapacitor image
Examine Dockefile of official kapacitor image
General note:
For any image, the original Dockerfiles can be obtained in this way:
https://hub.docker.com/
-> Description Tab
-> Supported tags and respective Dockerfile links section
-> each of the tags is a link that leads to the Dockerfile
So for kapacitor everything is in the influxdata-docker git repository
Then in the Dockerfile we find that the image is created based on
FROM buildpack-deps: stretch-curl
here:
buildpack-deps
the image provided by the project of the same name https://hub.docker.com/_/buildpack-deps
curl
This variant includes just the curl, wget, and ca-certificates packages. This is perfect for cases like the Java JRE, where downloading JARs is very common and
necessary, but checking out code isn't.
stretch
short version name of the OS, in this case Debian 9 stretch https://www.debian.org/News/2017/20170617
Buildpack-deps images are in turn built based on
FROM debian: stretch
And Debian images from the minimum docker image
FROM: scratch
Choose appropriate python image
Among python images, for example 3.7, you can find similar versions inheriting from buildpack-deps
FROM buildpack-deps: stretch
Following the inheritance, we'll see:
FROM buildpack-deps: stretch
FROM buildpack-deps: stretch-smc
FROM buildpack-deps: stretch-curl
FROM debian: stretch
In other words, the python: 3.7-stretch image only adds functionality to the Debian compared to the kapacitor image.
This means that we can to rebuild kapacitor image on top of the python image: 3.7-stretch with no risk or gaining incompatibility.
Docker context folder preparation
Clone the repository
https://github.com/influxdata/influxdata-docker.git
Create the folder influxdata-docker/kapacitor/1.5/udf_python/python3.7
Copy the following three files into it from influxdata-docker/kapacitor/1.5/:
Dockerfile
entrypoint.sh
kapacitor.conf
In the copied Dockerfile FROM buildpack-deps: stretch-curl replace with FROM python: 3.7-stretch
Be carefuly! If we work on Windows and because of scientific curiosity open the entrypoint.sh file in the project folder, then be sure to check that it does not change the end-line character from Linux (LF) to Windows variant: (CR LF).
Otherwise, when you start the container later, you get an error:
or in the container log:
exec: bad interpreter: No such file or directory
or if you'll start debugging and, running the container with bash, will do:
$ root # d4022ac550d4: / # exec /entrypoint_.sh
$ bash: /entrypoint_.sh: / bin / bash ^ M: bad interpreter: No such file or directory
Building
Run PS> docker build -f. \ Dockerfile -t kapacitor_python_udf
Again, in case of Windows environment
If during the build execution an error occurs of the form:
E: Release file for http://security.ubuntu.com/ubuntu/dists/bionic-security/InRelease is not valid yet (invalid for another 9h 14min 10s). Updates for this repository will not be applied.
then your computer clock probably went out of sync and/or Docker Desktop incorrectly initialized the time after the system returned from sleep. See the issue)
To fix it, restart Docker Desktop and / or Windows settings -> Date and time settings -> Clock synchronization -> perform Sync
You can also read more here
Launch and check
Launching the container with the same actions as for the standard image. Example:
PS> docker run --name=kapacitor -d `
--net=influxdb-network `
-h kapacitor `
-p 9092:9092 `
-e KAPACITOR_INFLUXDB_0_URLS_0=http://influxdb:8086 `
-v ${PWD}:/var/lib/kapacitor `
-v ${PWD}/kapacitor.conf:/etc/kapacitor/kapacitor.conf:ro `
kapacitor
Check:
PS> docker exec -it kapacitor_2 bash
$ python -VERSION
$ Python 3.7.7
$ readlink -f $(which python) | xargs -I% sh -c 'echo -n "%:"; % -V'
$ /usr/local/bin/python3.7: Python 3.7.7
We are using Docker container and created a Dockerfile. Inside this container we deployed war file using tomcat image
and we can see tomcat logs at console but console logs is not updating
after sending a request to tomcat via URL.
Also we can not see any log file inside tomcat logs folder
Can anyone help me out that how we can see tomcat logs like localhost.logs/catalina.logs/manager.logs etc
MY Dockerfile is :-
FROM openjdk:6-jre
ENV CATALINA_HOME /usr/local/tomcat
ENV PATH $CATALINA_HOME/bin:$PATH
COPY tomcat $CATALINA_HOME
ADD newui.war $CATALINA_HOME/webapps
CMD $CATALINA_HOME/bin/startup.sh && tail -F $CATALINA_HOME/logs/catalina.out
EXPOSE 8080
Used below script to build
$ docker build -t tomcat .
and below used to run tomcat
$ docker run -p 8080:8080 tomcat
Here are a few things wrong with your dockerfile:
You mention that you need java 6, and yet the line FROM java as of this writing is set to use java:8.
You need to replace the FROM line with FROM java:6-jre or as suggested by the official page: FROM openjdk:6-jre if in 2018 you still need java 6, which is dangerous. I would also strongly suggest to use at least FROM tomcat:7 which should be able to run java 6 applets but will include some bug fixes including support for longer Diffie-Hellman primes for HTTPS (if you are serious about your app's security).
Copt tomcat $CATALINA_HOME you either miss-typed the line to SO, or your image should not build at all. It should be COPY tomcat $CATALINA_HOME
Given that you are using the COPY command there is no need to use RUN mkdir -p prior to this, since the COPY command will automatically create all the required folders.
CMD $CATALINA_HOME/bin/startup.sh && tail -f $CATALINA_HOME/logs/catalina.out
First the tail -f part: since you are looking to tail a log file which might be created and recreated during the server's operation instead of following the FD you should be following the path by doing tail -F (capital F)
startup.sh && tail - tail will never start until startup.sh exits. A better approach is to do tail -F $CATALINA_HOME/logs/catalina.out & inside your startup.sh right before you start your tomcat server. That way tail will be running in the background.
Regardless this is a somewhat dangerous approach and you risk zombie processes because bash does not manage its children processes and neither does docker. I would recommend to use supervisord or something similar.
(From https://docs.docker.com/engine/admin/multi-service_container/)
FROM ubuntu:latest
RUN apt-get update && apt-get install -y supervisor
RUN mkdir -p /var/log/supervisor
COPY supervisord.conf /etc/supervisor/conf.d/supervisord.conf
COPY my_first_process my_first_process
COPY my_second_process my_second_process
CMD ["/usr/bin/supervisord"]
Note: this dockerfile sample omits a few of the best practices, e.g. removing the apt cache in the same run command as doing the apt-get update.
Personal favorite is the phusion/baseimage, but it is harder to setup since you'll need to install everything including the java into the image.
If with all of these modifications you still have no luck in seeing the console update, then you'll need to also post the contents of your startup.sh file or other tomcat related configurations.
P.S.: it might be a good idea to do RUN mkdir -p $CATALINA_HOME/logs just to make sure that the logs folder exists for tomcat to write to.
P.P.S.: the java base image is actually using openjdk instead of the oracle one. Just thought I'd point it out
You should check tomcat logging settings. The default logging.properties in the JRE specifies a ConsoleHandler that routes logging to System.err. The default conf/logging.properties in Apache Tomcat also adds several FileHandlers that write to files.
Example logging.properties file to be placed in $CATALINA_BASE/conf:
handlers = 1catalina.org.apache.juli.FileHandler, \
2localhost.org.apache.juli.FileHandler, \
3manager.org.apache.juli.FileHandler, \
java.util.logging.ConsoleHandler
.handlers = 1catalina.org.apache.juli.FileHandler, java.util.logging.ConsoleHandler
############################################################
# Handler specific properties.
# Describes specific configuration info for Handlers.
############################################################
1catalina.org.apache.juli.FileHandler.level = FINE
1catalina.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
1catalina.org.apache.juli.FileHandler.prefix = catalina.
2localhost.org.apache.juli.FileHandler.level = FINE
2localhost.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
2localhost.org.apache.juli.FileHandler.prefix = localhost.
3manager.org.apache.juli.FileHandler.level = FINE
3manager.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
3manager.org.apache.juli.FileHandler.prefix = manager.
3manager.org.apache.juli.FileHandler.bufferSize = 16384
java.util.logging.ConsoleHandler.level = FINE
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
############################################################
# Facility specific properties.
# Provides extra control for each logger.
############################################################
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].level = INFO
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].handlers = \
2localhost.org.apache.juli.FileHandler
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/manager].level = INFO
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/manager].handlers = \
3manager.org.apache.juli.FileHandler
# For example, set the org.apache.catalina.util.LifecycleBase logger to log
# each component that extends LifecycleBase changing state:
#org.apache.catalina.util.LifecycleBase.level = FINE
Example logging.properties for the servlet-examples web application to be placed in WEB-INF/classes inside the web application:
handlers = org.apache.juli.FileHandler, java.util.logging.ConsoleHandler
############################################################
# Handler specific properties.
# Describes specific configuration info for Handlers.
############################################################
org.apache.juli.FileHandler.level = FINE
org.apache.juli.FileHandler.directory = ${catalina.base}/logs
org.apache.juli.FileHandler.prefix = servlet-examples.
java.util.logging.ConsoleHandler.level = FINE
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
More info at https://tomcat.apache.org/tomcat-6.0-doc/logging.html
we can not see the logs in Docker container until unless we mount it.
To build the Dockerfile:-
docker build -t tomcat
To run the Dockerfile Image:-
docker run -p 8080:8080 tomcat
To copy the logs of tomcat present in docker container to mounted container :-
Run this cmd to mount the container:
1stpath : 2ndpath
docker run \\-d \\-p 8085:8085 \\-v /usr/local/tomcat/logs:/usr/local/tomcat/logs \tomcat
or simply
docker run \\-d \\-v /usr/local/tomcat/logs:/usr/local/tomcat/logs \tomcat
1st:-/usr/local/tomcat/logs: path of root dir: where we want to copy
the logs or destination
2nd:- /usr/local/tomcat/logs: path of tomcat/logs folder present in
docker container
tomcat:-name of image
need to change the port if it is busy
now the container is get mount
to get the list of container run : docker ps -a
now get the container id of latest created container:
docker exec -it < mycontainer > bash
then we can see the logs by
cd /usr/local/tomcat/logs
usr/local/tomcat/logs# less Log Name Here
this to Copy any folder in docker container on root:-
docker cp <containerId>:/file/path/within/container /host/path/target
I have a Dockerfile that starts with installing the texlive-full package, which is huge and takes a long time. If I docker build it locally, the intermedate image created after installation is cached, and subsequent builds are fast.
However, if I push to my own GitLab install and the GitLab-CI build runner starts, this always seems to start from scratch, redownloading the FROM image, and doing the apt-get install again. This seems like a huge waste to me, so I'm trying to figure out how to get the GitLab DinD image to cache the intermediate images between builds, without luck so far.
I have tried using the --cache-dir and --docker-cache-dir for the gitlab-runner register command, to no avail.
Is this even something the gitlab-runner DinD image is supposed to be able to do?
My .gitlab-ci.yml:
build_job:
script:
- docker build --tag=example/foo .
My Dockerfile:
FROM php:5.6-fpm
MAINTAINER Roel Harbers <roel.harbers#example.com>
RUN apt-get update && apt-get install -qq -y --fix-missing --no-install-recommends texlive-full
RUN echo Do other stuff that has to be done every build.
I use GitLab CE 8.4.0 and gitlab/gitlab-runner:latest as runner, started as
docker run -d --name gitlab-runner --restart always \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /usr/local/gitlab-ci-runner/config:/etc/gitlab-runner \
gitlab/gitlab-runner:latest \
; \
The runner is registered using:
docker exec -it gitlab-runner gitlab-runner register \
--name foo.example.com \
--url https://gitlab.example.com/ci \
--cache-dir /cache/build/ \
--executor docker \
--docker-image gitlab/dind:latest \
--docker-privileged \
--docker-disable-cache false \
--docker-cache-dir /cache/docker/ \
; \
This creates the following config.toml:
concurrent = 1
[[runners]]
name = "foo.example.com"
url = "https://gitlab.example.com/ci"
token = "foobarsldkflkdsjfkldsj"
tls-ca-file = ""
executor = "docker"
cache_dir = "/cache/build/"
[runners.docker]
image = "gitlab/dind:latest"
privileged = true
disable_cache = false
volumes = ["/cache"]
cache_dir = "/cache/docker/"
(I have experimented with different values for cache_dir, docker_cache_dir and disable_cache, all with the same result: no caching whatsoever)
I suppose there's no simple answer to your question. Before adding some details, I strongly suggest to read this blog article from the maintainer of DinD, which was originally named "do not use Docker in Docker for CI".
What you might try is declaring /var/lib/docker as a volume for your GitLab runner. But be warned, depending on your file-system drivers you may use AUFS in the container on an AUFS filesystem on your host, which is very likely to cause problems.
What I'd suggest to you is creating a separate Docker-VM, only for the runner(s), and bind-mount docker.sock from the VM into your runner-container.
We are using this setup with GitLab with great success (>27.000 builds in about 12 months).
You can take a look at our runner with docker-compose support which is actually based on the shell-executor of GitLab's runner.
Currently you cannot cache intermediate layers in GitLab Docker-in-Docker. Altough there are plans to add that (that are mentioned in the link below). What you can do today to speed up your DinD build is to use the overlay filesystem. To do this you need to be running a liunx kernel >=3.18 and make sure you load the overlay kernel module. Then you set this variable in your gitlab-ci.yml:
variables:
DOCKER_DRIVER: overlay
For more information see this issue and in particular this comment on "The state of optimising Docker Builds!", see the "Using docker executor with dind" section.
https://gitlab.com/gitlab-org/gitlab-ce/issues/17861#note_12991518
For build dependencies that do not change so ofter you can do kinda manual caching with gitlab image registry.
In CI script you do not explicitely call docker build but rather wrap it in a shell script
# cat build_dependencies.sh
registry=registry.example.com
project=group/project
imagebase=$registry/$project/linux
docker pull $imagebase/devbase:1.0
if [ $? -ne 0 ]; then
docker build -f devbase.dockerfile -t $imagebase/devbase:1.0 .
docker push $imagebase/devbase:1.0
fi
...
and call that script in your CI
...
script:
- ./build_dependencies.sh
The downside to this is that when your devbase.dockerfile is updated this would get unnoticed by CI, so you need to force build and push of a new image. So for dynamicly changing images this does not work well, but for your use case this seems like a possible way to go.