I created a task definition on Amazon ECS and want to run in with Fargate. I set up my task, network mode is awsvpc. I created a new container with a docker image (simple "Hello world" project) on Amazon ECR. Run the task - everything works fine. Now I need to run a docker container from hub.docker.com as a part of the task
Dockerfile
FROM ubuntu
RUN apt-get update && apt-install ...
ADD script.sh /script.sh
RUN chmod +x /script.sh
ENTRYPOINT ["/script.sh"]
script.sh
#!/bin/bash
...prepare data
docker run -rm some_container_from_docker_hub
...continue process data
Initially, I got "command not found" error. OK, I installed docker into my image. Now I've got "Cannot connect to the Docker daemon".
My question: is there any way to run a docker container inside of another docker container on Amazon Fargate?
You can't run a container from another container using Fargate.
Running a container from another one, like in your case, would mean that you could have access to the docker daemon. Accessing the docker daemon means root access to the host machine. This breaks the docker container isolation and is unsafe.
Depending on your usage, I suggest you use an EC2 instance, use CodeBuild or build an operator that is able to talk with the api to span containers.
[Edit]: It seems that there is an open issue on this topic [ECS,Fargate]: Support for building Docker containers #95
Related
How could I create an Airflow task where I would start a Docker container that is using GPU. When running from the terminal I would just use --gpus all flag. I can`t do that using DockerOperator, because it does not support device_requests parameter, which is used underneath when calling docker run with --gpus all flag.
Okay, for anyone in the future - I figured it out. First, you need to mount the docker daemon socket inside an airflow docker container. Do this by changing docker-compose file by adding in the volumes section of airflow:
- /var/run/docker.sock:/var/run/docker.sock
Then you need to create a new docker image based on airflow docker image and install docker python SDK, eg.:
# syntax=docker/dockerfile:1
FROM apache/airflow:2.2.0-python3.7
RUN pip install docker
then you can create tasks based on PythonOperator, where you use the docker library to create new containers. Example task (output is not pretty)
def start_gpu_container(**kwargs):
client = docker.from_env()
response = client.containers.run(
'tensorflow/tensorflow:latest-gpu',
'nvidia-smi',
device_requests=[
docker.types.DeviceRequest(count=-1, capabilities=[['gpu']])
]
)
return str(response)
I'm building an app that makes api calls to run code inside docker containers
I want to run a docker container that has docker running inside it.
I want to create a docker file that pulls other docker images inside it and then waits for api calls (on port 2376) to create, run and delete containers based on the docker images that i pulled into the dockerfile
This is the dockerfile I'm trying to create right now.
FROM docker:stable
RUN docker pull python
EXPOSE 23788
CMD tail -f /dev/null
However when the RUN command is issued i get this error message:
docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
I don't really know how to start docker inside a docker container.
The reason i need this kind of a docker file is so that i can then use kubernetes to scale this part of my application
There's a special image for this, docker:dind. See the bit about "Docker in Docker" in https://hub.docker.com/_/docker.
I have a master container instance (Node.js) that runs some tasks in a temporary worker docker container.
The base image used is node:8-alpine and the entrypoint command executes with user node (non-root user).
I tried running my container with the following command:
docker run \
-v /tmp/box:/tmp/box \
-v /var/run/docker.sock:/var/run/docker.sock \
ifaisalalam/ide-taskmaster
But when the nodejs app tries running a docker container, permission denied error is thrown - the app can't read /var/run/docker.sock file.
Accessing this container through sh and running ls -lha /var/run/docker.sh, I see that the file is owned by root:412. That's why my node user can't run docker container.
The /var/run/docker.sh file on host machine is owned by root:docker, so I guess the 412 inside the container is the docker group ID of the host machine.
I'd be glad if someone could provide me an workaround to run docker from docker container in Container-optimized OS on GCE.
The source Git repository link of the image I'm trying to run is - https://github.com/ifaisalalam/ide-taskmaster
Adding the following command into my start-up script of the host machine solves the problem:
sudo chmod 666 /var/run/docker.sock
I am just not sure if this would be a secure workaround for an app running in production.
EDIT:
This answer suggests another approach that might also work - https://stackoverflow.com/a/47272481/11826776
Also, you may read this article - https://denibertovic.com/posts/handling-permissions-with-docker-volumes/
I'd like to package Selenium grid exrtas into a docker image.
This service being run without using docker container can reboot the OS it's running in. I wonder if I can setup the container to restart by Selemiun grid extras service running inside the container.
I am not familiar with Selenium Grid, but as a general idea: you could mount a folder from the host as data volume, then let Selenium write information to there, like a flag file.
On the host, you have a scheduled task / cronjob running on the host that would check for this flag in the shared folder and if it has a certain status, you would invoke a docker restart from there.
Not sure if there are other more elegant solutions for this, but this is what came to my mind adhoc.
Update:
I just found this on the Docker forum:
https://forums.docker.com/t/how-can-i-run-docker-command-inside-a-docker-container/337
I'm not sure about CoreOS but normally you can manage your host
containers from within a container by mounting the Docker socket.
Such as
docker run -it -v /var/run/docker.sock:/var/run/docker.sock ubuntu:latest sh -c "apt-get update ; apt-get install docker.io -y ;
bash"
or
https://registry.hub.docker.com/u/abh1nav/dockerui/
I'm trying to run Docker inside a Jenkins container that is also running in Docker (i.e. Docker in Docker). What I want to know is how to properly start the Docker service when booting Jenkins. The only solution I've found today is to build my own Jenkins image based on the official Jenkins image but change the jenkins script loaded by the entry point to also start up Docker:
# I've added this line just before Jenkins is started from the script:
sudo service docker start
# I've also removed "exec" from the original file which used "exec java $JAVA_TOPS ..." but that didn't work
java $JAVA_OPTS -jar /usr/share/jenkins/jenkins.war $JENKINS_OPTS "$#"
This works when I run (using docker run) a new container but the problem is that if I do (docker start) on stopped container the Docker service is not started.
I strongly suspect that this is not the right way to start my Docker service. My plan is to perhaps use supervisord to start Jenkins and Docker separately (I suppose container linking is out of the question since Docker should be executed as a service on the same container that Jenkins is running on?). My concern with this approach is that I'm going to lose the EntryPoint specified in the Jenkins Dockerfile which allows me to pass arguments to the Jenkins container when starting the container, for example:
docker run -p 8080:8080 -v /your/home:/var/jenkins_home jenkins -- <jenkins_arguments>
Does anyone have any recommendations on a good way to solve this preferably by not forking the official Jenkins image?
I'm pretty you cannot do that.
Docker in Docker doesn't mean you have to run docker inside docker with 3 level : host > First level container > Second Level Container
In fact, you just need to share docker with host, and this is your host who will run others containers.
To do that, you have to mount volume with -v parameter
-v /var/run/docker.sock:/var/run/docker.sock
with this command, when you will docker run inside you jenkins container, the docker client will communicate with docker deamon from your host in order to run new container.
To do that, you should run your jenkins container with privileged
--privileged
To resume, here is the full command line
docker run -d -v /var/run/docker.sock:/var/run/docker.sock --privileged myimage
And you you don't need to create a new jenkins image for that.
Hoping to have helped you
http://container-solutions.com/running-docker-in-jenkins-in-docker/