OCI runtime exec failed - docker

since today i have got an error message with various docker commands. Unfortunately I don't really know what to do with it. Does anyone have any idea what the problem could be and how I fix it?
Error:
OCI runtime exec failed: exec failed: container_linux.go:370: starting container process caused: process_linux.go:95: starting setns process caused: fork/exec /proc/self/exe: resource temporarily unavailable: unknown
Another Error:
ERROR: for hosting_mail_1 Cannot start service mail: OCI runtime create failed: unable to retrieve OCI runtime error (open /run/containerd/io.containerd.runtime.v2.task/moby/5fabf9edf67fbd6455bdc955c56c063683aa78e8e31514660661799aaa867391/log.json: no such file or directory): runc did not terminate successfully: unknown
ERROR: for mail Cannot start service mail: OCI runtime create failed: unable to retrieve OCI runtime error (open /run/containerd/io.containerd.runtime.v2.task/moby/5fabf9edf67fbd6455bdc955c56c063683aa78e8e31514660661799aaa867391/log.json: no such file or directory): runc did not terminate successfully: unknown
ERROR: Encountered errors while bringing up the project.

I don't know if you solved your problem finally, but this really looks like bad file system authorization that may have been corrupted from an update on the file systems.
Regarding the error : container_linux.go:370:,/run/containerd/io.containerd.runtime.v2.task/moby/5fabf9edf67fbd6455bdc955c56c063683aa78e8e31514660661799aaa867391/log.json
I can see that :
docker manage to initiate a volume ID
did not manage to mount that volume on the disk
0/ Check docker basic command
docker ps
docker images
docker pull ubuntu:latest
If one of these commands is failing, you are up to review docker installation, seems that maybe docker is not installed properly.
1/
To check if you need to completely re-install docker, you may try the following basic command
docker run --name checkDocker -it ubuntu:latest bash
If this is not displaying any docker shell, then you have a problem on running a container, not necessarly docker installation.
2/
Check your docker volumes and rights, I don't have your installation setup, but It seems you are using docker-compose and maybe there is some conflicts when mounting the volume of your containers with specific rights and the host's rights and user id
3/
If you are ending up here you should follow that work around of re-installation, which would be the fatest solution to restore your application if you have backup (hope you have )

Related

Docker error failed to create shim: OCI runtime create failed: container_linux.go:380:

ERROR: for app_web
Cannot start service app_web: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: rootfs_linux.go:75: mounting "/host_mnt/Users/akanwar/Documents/c/cbax-config" to rootfs at "/cbax-apply-platform/node_modules/#c/cbax-config" caused: mkdir /var/lib/docker/overlay2/a3bccebb167966c795860d95a5a758f244ae5da780f962333f0d51d2d8b2def7/merged/cbax-apply-platform/node_modules/#c/cbax-config: operation not permitted: unknown
STEPS TAKEN TO RESOLVE
deleted the docker data ran everything.
From Docker widget Clean / Purge data
From Docker widget reset to factory settings
docker system prune --all
https://github.com/docker/for-mac/issues/1396 I followed this link as well and tried the solutions mentioned in it , but still none of it worked for me.
None of the solutions worked for me
You need to remove this from all services:
- ${CBAX_PATH}/cbax-apply-platform:/cbax-apply-platform:rw
You cannot mount a folder and then also sub-folders. This is wrong:
- ${CBAX_PATH}/cbax-apply-platform:/cbax-apply-platform:rw
- ${CBAX_PATH}/cbax-application-pages:/cbax-apply-platform/app/assets/components/cbax-application-pages:rw
Also you mount a lot of host folders in multiple service with read-write access. I think that makes a mess of your files on the host if all services start modifying files. It's better to mount them read-only.
This happens when the docker-compose file was looking for the folder cbax-config path on my machine on mac OS. MacOS doesn't have an cbax-config therefore it was not able to mount.

problem with start and execute a docker container

I am new to docker
I am using ssh to connect to Nvidia jetson NX and use the docker on nvidia
the system for nvidia is a modifed ubuntu 18.04 called tegra.
What I want to do is to exec the container named tetraai_service_nx.
It worked fine until yesterday by command "docker exec -it tetraai_service_nx /bin/bash
"
however, today when I tried the same command, i see"Error response from daemon: Container 0feeb9be5a251bb9ce45ed9a05d24a86e5a77ef9d93439c717fdef9f7a6560d4 is not running
"
0feeb9be5a251bb9ce45ed9a05d24a86e5a77ef9d93439c717fdef9f7a6560d4 is the right id for the container I want which is correct.
Then i tried as in the following picture and got the error
This is the error i see
Error response from daemon: Cannot restart container tetraai_service_nx: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: Running hook #0:: error running hook: exit status 1, stdout: , stderr: Auto-detected mode as 'csv'
invoking the NVIDIA Container Runtime Hook directly (e.g. specifying the docker --gpus flag) is not supported. Please use the NVIDIA Container Runtime instead.: unknown
Could anyone explain what this message mean exactly and what might the problem be? because I saw similar errors in different places
some command result for me
some command result for me
some command result for me
I have the same issue. If you install the previous version, 1.9 it should work until they can fix 1.10. For more information, check out this: https://aur.archlinux.org/packages/nvidia-container-runtime and read the comments.

Docker (in Azure App Service) Running out of space

A docker container running jsReport is running out of sapce and the container is failing.
Error in logfile
2021-07-13T07: 37: 25.469Z ERROR - Container start failed for xxxx
with System.AggregateException, One or more errors occurred. (Docker
API responded with status code=InternalServerError, response= {
"message": "OCI runtime create failed: container_linux.go:370: starting container process caused: process_linux.go:459: container
init caused: rootfs_linux.go:59: mounting
"/var/LWASFiles/Sites/xxxx/appsvctmp" to rootfs at
"/mnt/data/docker/images/231072.231072/aufs/mnt/7c0065c198b3881806bb293ccee0ceef29b8b619bb47a7512f9b5485557811d5/appsvctmp"
caused: mkdir
/mnt/data/docker/images/231072.231072/aufs/mnt/7c0065c198b3881806bb293ccee0ceef29b8b619bb47a7512f9b5485557811d5/appsvctmp:
no space left on device: unknown" }
Problem started when I was updating to service where each dowload was about 200MB+.
I could probably fix this by removing the service and create a new one OR up the service plan which adds more space.
However I would think there must be a way to use docker commands to clean up some of these upgrade files. Can anyone point me in the right direction?
you can use docker system df command to see the reclaimable space as shown in below image
you can use the docker prune command to remove the unused objects such as images, containers, volumes, and networks
https://docs.docker.com/config/pruning/

docker: Error response from daemon: OCI runtime create failed: unable to retrieve OCI runtime error

my docker starts giving error after a recent update. Existing containers works but I can not build or run any image.
A simple
docker run hello-world
gives me an error:
docker: Error response from daemon: OCI runtime create failed: unable to retrieve OCI runtime error (open /run/containerd/io.containerd.runtime.v1.linux/moby/881b53be5cfe91d19577414c2f4a52dd06804624fe1d2189d06c1c3c13f2b4d1/log.json: no such file or directory): runc did not terminate successfully: unknown.
I tried all the tips that are suggested on the internet regarding this issue as restarting, building links, reinstall.....
My current docker version is 18.09.5, Ubuntu 19.04
Does anyone had a similar issue and solved it?
To check the issue by run docker in the debug mode.
stop docker with systemctl stop docker
run docker in debug mode dockerd --debug
start container with docker start container_name
Then check the output in docker debug console in 2.
In my case, it shows
ERRO[2020-07-07T23:15:02Z] stream copy error: reading from a closed fifo
ERRO[2020-07-07T23:15:02Z] stream copy error: reading from a closed fifo
And solve it by removing the container state folder with rm -rf /var/run/docker/runtime-runc/moby/docker_id
Then start your container.
Refer to: Solving Error response from daemon OCI runtime create failed container with id exists
Check the output of docker version and see if the client version and daemon version have gone out of sync.
Check the output of following commands which runc and which docker-runc. If the docker daemon version is 18.09, you should be having runc or else docker-runc.
I use debian 11 with docker 20.10.5, When I run docker run hello-world I got:
docker: Error response from daemon: OCI runtime create failed: unable
to retrieve OCI runtime error (open
/run/containerd/io.containerd.runtime.v2.task/moby/2957ad06a6bc7a4f7c7f3fca6b43bde1d6b27600df774f0e8052f4c736300759/log.json:
no such file or directory): runc did not terminate successfully: exit
status 139: unknown. ERRO[0010] error waiting for container: context
canceled
when I run runc command , I got "segmentation fault".
I solved it by apt reinstall runc.
I was trying to run docker as docker run ... my_external_script.py. Because my_external_script.py wasn't w/in the docker image I was getting the error. What I did was to set the scripts argument w/in my setup.py as scripts=["path/to/my_external_script.py"] and rebuild docker image. That solved the problem.

Docker Image running correctly on one machine but failing on another

When Trying to install Istio 1.2.3 on my cluster using Helm, I encountered an issue with the istio/kubectl image being used in the istio-init jobs with the following error:
container_linux.go:295: starting container process caused "exec: \"kubectl\": executable file not found in $PATH"
docker: Error response from daemon: oci runtime error: container_linux.go:295: starting container process caused "exec: \"kubectl\": executable file not found in $PATH".
Running the kubectl command in my local docker also gives the same error, however on another machine it works correctly
docker run <istio/kubectl-imageid> kubectl
What could cause this issue? And what would I need to change to overcome it?
It is definitely the same docker image and from my understanding a docker image should work identically in different environments assuming the same cpu architecture.
Turns out when I copied the image across machines, I did a
docker import istio-kubectl.1.2.3.tar
instead of a
docker load istio-kubectl.1.2.3.tar
The difference according to the documentation is:
docker load: Load an image from a tar archive or STDIN
docker import: Import the contents from a tarball to create a filesystem image
Loading the image instead of importing corrected the observed issue.

Resources