Any idea why I can't run docker using Airflow?
The same docker command runs fine from terminal using the user that runs airflow. So there is no permission issue on the Linux side.
But when put it into Airflow dag, it complains
docker: Got permission denied while trying to connect to the Docker
daemon socket at unix:///var/run/docker.sock: Post
http://%2Fvar%2Frun%2Fdocker.sock/v1.39/containers/create: dial unix
/var/run/docker.sock: connect: permission denied.
The command is simply docker run --rm -v /data:/data:ro docker_image mycommand. It is wrapped inside a python subprocess, so I can't use docker operator.
Note this is not the same issue with the question: How to fix "dial unix /var/run/docker.sock: connect: permission denied" when group permissions seem correct?
There it is a linux permission issue as it cannot run the docker run command. Here the problem is more with Airflow, I think.
Faced a similar issue .
Got it fixed by changing the default_owner(default=airflow) in airflow.cfg to the user having permission to access docker(i.e username belonging to docker group).
Interestingly passing owner in 'default_args' in the DAG definition didn't seem to work
Looks like the reason is in access rights to /var/run/docker.sock.
For dev environment, you can do:
docker exec -ti -u root <container> bash
sudo chmod 777 /var/run/docker.sock
Related
I have a self hosted linux agent, where I can docker, npm, nodeJs installed.
I want to execute a simple container job, where I fetch the publicly available image 'ubuntu:18.04' and just echo a line on to the terminal.
But I end up facing this issue:
Starting: Initialize containers
/usr/bin/docker version --format '{{.Server.APIVersion}}'
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/version": dial unix /var/run/docker.sock: connect: permission denied
'
##[error]Exit code 1 returned from process: file name '/usr/bin/docker', arguments 'version --format '{{.Server.APIVersion}}''.
Finishing: Initialize containers
I researched a bit on to this issue, and added my user to the docker group, restarted the daemon, but still no success.
Any suggestions?
When using buildpacks to build my spring boot application on Fedora I get the following error during the execution of the spring-boot-plugin:build-image goal:
[INFO] [creator] ERROR: initializing analyzer: getting previous image: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/info": dial unix /var/run/docker.sock: connect: permission denied
After digging into the goal and buildpacks, I found the following command in the buildpack.io docs (by selecting "Linux" and "Container"):
docker run \
-v /var/run/docker.sock:/var/run/docker.sock \
-v $PWD:/workspace -w /workspace \
buildpacksio/pack build <my-image> --builder <builder-image>
AFAICT this command should be equivalent to what happens inside of maven and it exhibits the same error.
My previous assumption was that the use in the buildpacksio/pack image doesn't have the access permissions to my docker socket. (The socket had 660 permissions and root:docker owner).
UPDATE: Even after updating to 666 permissions the issue still persists.
I don't really want to allow anyone to interact with the docker socket so setting it to 666 seems unwise. Is this the only option or can I also add the user in the container to the docker group?
The solution was that the Fedora docker package is no longer the most up-to-date way to install Docker. See the official Docker documentation
They both provide the same version number, but their build hash is different.
While I could not fully diagnose the difference between the two, I can report that it works with docker-ce and doesn't with docker.
I have access to a server by ssh with docker version 1.13.1 and I'm
just trying to load a local image using docker load -i
and I'm receiving this error message:
docker load -i docker.img
Error processing tar file(exit status 1): permission denied
And by the way:
docker image import docker.img
Error response from daemon: ApplyLayer exit status 1 stdout: stderr: permission denied
The img file has all the permissions:
> ls -l
> -rwxrwxrwx 1 myuser myuser 9278464 Mar 22 19:12 docker.img*
And docker seems to work rigth:
> docker images
> REPOSITORY TAG IMAGE ID CREATED SIZE
The image works perfectly fine in my local machine...
Any idea about what can be happening here ? The host is running ubuntu 16.04, i was looking for an answer about 1 hour...
=======
I could figure it out, the problem was that I was accessing a proxmox container
not fully virtualized, so, docker requires kernel capabilities that I had not. I searched for the correct proxmox configuration and I solve the issue.
You are trying to execute docker commands as which user?, Standard account or as root user?
Note that the docker daemon binds to a Unix socket instead of a TCP port. By default that Unix socket is owned by the user root and other users can only access it using sudo. The docker daemon always runs as the root user.
If you don’t want to use sudo when you use the docker command, create a Unix group called docker and add users to it. When the docker daemon starts, it makes the ownership of the Unix socket read/writable by the docker group.
First, check if docker group exists on your server. If not, then add it
$ sudo groupadd docker
Add your user to the docker group.
$ sudo usermod -aG docker $USER
Log out and log back in so that your group membership is re-evaluated
I am trying to setup remote host configuration for docker. After setting up certificates i ran dockerd command which is giving error:
dockerd --tlsverify --tlscacert=ca.pem --tlscert=server-cert.pem --tlskey=server-key.pem -H=0.0.0.0:2376
>>> unable to configure the Docker daemon with file /etc/docker/daemon.json: open /etc/docker/daemon.json: permission denied
I am running from non root user and I've already added my user as part of Docker group. Docker version i am using is:
Docker version 17.12.0-ce, build c97c6d6
I have tried below but still getting same error:
1. the /etc/docker/daemon.json file is having {}
2. I also removed the /etc/docker/daemon.json
3. I also changed ownership but same issue.
Permissions of daemon.json were: -rw-r--r--
The dockerd daemon must be run as root. It is creating networking namespaces, mounting filesystems, and other tasks that cannot be done with a user account. You'll need to run these commands with something like sudo.
The docker socket (/var/run/docker.sock) is configured to allow the docker client to access the api by users in the docker group. This is the client, not the daemon, so you can't get away with running the daemon as a user.
I have two questions.
1) I get an error when I run:
karl#karl-laptop:~o docker pull node
2015/04/29 01:39:41 Post http:///var/run/docker.sock/images/create?fromImage=banode&tag=: dial unix /var/run/docker.sock: permission denied
2) When I pull down images, where are they located?
The permission error could be due to the Docker socket being owned by root and thus either you have to use sudo or following the instructions in this link to Create a docker group
For the default docker storage driver, aufs, the images are found in /var/lib/docker/aufs/diff/, assuming the Root Dir for Docker is /var/lib/docker/aufs. You can find this out by running docker info.
Add yourself to the docker group by running
sudo usermod -a -G docker $USER
and log out and back in