Im trying to use buildctl tool with Artifactory registry running on my localhost.
I'm using the following command.
buildctl build \
--frontend=dockerfile.v0 \
--local context=. \
--local dockerfile=. \
--output type=image,name=192.168.0.110:8082/docker-local/test,push=true,registry.insecure=true \
--export-cache type=registry,ref=192.168.0.110:8082/docker-local/test,mode=max,push=true,registry.insecure=true \
--import-cache type=registry,ref=192.168.0.110:8082/docker-local/test,registry.insecure=true
I added the flag "registry.insecure=true" as stated in the documentation.
but, still getting the following error:
> exporting content cache:
------
error: failed to solve: error writing layer blob: failed to do request: Head "https://192.168.0.110:8082/v2/docker-local/test/blobs/sha256:03d1cdba14f373b9dbca6b5fe65f8eca1e9852aaaf9060450b27f924a56a1b3c": remote error: tls: unrecognized name
Seems like it's trying to reach the local repo with HTTPS.
How can I make it work with HTTP?
Using version: buildctl github.com/moby/buildkit 0.11.1
The buildkit daemon needs to be run with a configuration file that specifies the registry is http instead of https. See the documentation on buildkitd.toml:
[registry."192.168.0.110:8082"]
http = true
The file path is /etc/buildkit/buildkitd.toml for rootful mode, ~/.config/buildkit/buildkitd.toml for rootless mode.
Related
I'm facing the same problem as here - I have set up a private Docker Registry with TLS certification (certificates generated via Certbot), and I can interact with it directly via curl etc. (thus proving that the certificate is correct), but the Docker Plugin in my Drone flow gives an error x509: certificate signed by unknown authority.
As per this StackOverflow answer, I believe that putting the certificate at /etc/docker/certs.d/<my_registry_address:port>/ca.crt should fix this problem, but it doesn't appear to (neither does adding the certificate into the standard /etc/ssl/certs/ca-certificates.crt location)
Demonstration that the certificates work as-expected, having already built the Docker Drone Plugin locally as per https://github.com/drone-plugins/drone-docker:
$ docker run --rm -v <path_to_directory_containing_pems>:/custom-certs -it --entrypoint /bin/sh plugins/docker
/ # ls /custom-certs
accounts archive csr keys live renewal renewal-hooks
/ # apk add curl
...
OK: 28 MiB in 56 packages
/ # curl https://docker-registry.scubbo.org:8843/v2/_catalog
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
/ # curl https://docker-registry.scubbo.org:8843/v2/_catalog --cacert /custom-certs/live/docker-registry.scubbo.org/fullchain.pem
{"repositories":[...]}
/ # cat /custom-certs/live/docker-registry.scubbo.org/fullchain.pem >> /etc/ssl/certs/ca-certificates.crt
/ # curl https://docker-registry.scubbo.org:8843/v2/_catalog
{"repositories":[...]}
Here's my .drone.yml, for a Runner instantiated with --env=DRONE_RUNNER_VOLUMES=/var/run/docker.sock:/var/run/docker.sock,<path_to_directory_containing_pems>:/custom-certs:
kind: pipeline
name: hello-world
type: docker
platform:
os: linux
arch: arm64
steps:
- name: copy-cert-into-place
image: busybox
volumes:
- name: docker-cert-persistence
path: /etc/docker/certs.d/
commands:
# https://stackoverflow.com/a/56410355/1040915
# Note that we need to mount the whole `custom-certs` directory into the workflow and then copy the file to `/etc/...`,
# rather than mounting the file directly into `/etc/...`, because the original file is a symlink and it's not possible (AFAIK)
# to instruct Docker to "mount the eventual-target-of this symlink into <location>"
- mkdir -p /etc/docker/certs.d/docker-registry.scubbo.org:8843
- cp -L /custom-certs/live/docker-registry.scubbo.org/fullchain.pem /etc/docker/certs.d/docker-registry.scubbo.org:8843/ca.crt
- name: check-cert-persists-between-stages
image: alpine
volumes:
- name: docker-cert-persistence
path: /etc/docker/certs.d/
commands:
- apk add curl
# The command below would fail if the cert was unavailable or invalid
- curl https://docker-registry.scubbo.org:8843/v2/_catalog --cacert /etc/docker/certs.d/docker-registry.scubbo.org:8843/ca.crt
- name: build-image
# ...contents irrelevant to this question...
- name: push-built-image
image: plugins/docker
volumes:
- name: docker-cert-persistence
path: /etc/docker/certs.d/
settings:
repo: docker-registry.scubbo.org:8843/scubbo/blog_nginx
tags: built_in_ci
debug: true
launch_debug: true
volumes:
- name: docker-cert-persistence
temp: {}
giving these logs from push-built-image step - ending in...
+ /usr/local/bin/docker tag 472d41d9c03ee60fe9c1965ad9cfd36a1cdb6cbf docker-registry.scubbo.org:8843/scubbo/blog_nginx:built_in_ci
+ /usr/local/bin/docker push docker-registry.scubbo.org:8843/scubbo/blog_nginx:built_in_ci
The push refers to repository [docker-registry.scubbo.org:8843/scubbo/blog_nginx]
Get "https://docker-registry.scubbo.org:8843/v2/": x509: certificate signed by unknown authority
exit status 1
How should I go about providing the CA Certificate to my Drone Docker Plugin step to permit it to communicate over TLS with a secure Docker registry? This answer suggests simply reverting to insecure integration, which works but is unsatisfactory.
EDIT: After re-reading this documentation, I extended the copy-cert-into-place commands to copy all 3 certificate-related files:
commands:
- mkdir -p /etc/docker/certs.d/docker-registry.scubbo.org:8843
- cp -L /custom-certs/live/docker-registry.scubbo.org/fullchain.pem /etc/docker/certs.d/docker-registry.scubbo.org:8843/ca.crt
- cp -L /custom-certs/live/docker-registry.scubbo.org/privkey.pem /etc/docker/certs.d/docker-registry.scubbo.org:8843/client.key
- cp -L /custom-certs/live/docker-registry.scubbo.org/cert.pem /etc/docker/certs.d/docker-registry.scubbo.org:8843/client.cert
but that did not resolve the problem - same x509: certificate signed by unknown authority error.
EDIT2: I directly confirmed (directly on a host, outside the context of a plugin or docker container) that adding the certificate to the path used above is sufficient to permit interaction with the registry:
$ docker pull docker-registry.scubbo.org:8843/scubbo/blog_nginx:built_in_ci
Error response from daemon: Get "https://docker-registry.scubbo.org:8843/v2/": x509: certificate signed by unknown authority
$ sudo cp -L <path_to_directory_containing_pems>/live/docker-registry.scubbo.org/chain.pem /etc/docker/certs.d/docker-registry.scubbo.org\:8843/ca.crt
$ docker pull docker-registry.scubbo.org:8843/scubbo/blog_nginx:built_in_ci
built_in_ci: Pulling from scubbo/blog_nginx
Digest: sha256:3a17f86f23050303d94443f24318b49fb1a5e2d0cc9228270678c8aa55b4d2c2
Status: Image is up to date for docker-registry.scubbo.org:8843/scubbo/blog_nginx:built_in_ci
docker-registry.scubbo.org:8843/scubbo/blog_nginx:built_in_ci
This isn't a complete answer, but I was able to get secure registry access working by switching from mounting a directory, to mounting the file directly:
I changed the docker run option to --env=DRONE_RUNNER_VOLUMES=/var/run/docker.sock:/var/run/docker.sock,$(readlink -f <path_to_directory_containing_pems>/live/docker-registry.scubbo.org/chain.pem):/registry_cert.crt
I changed the commands in copy-cert-into-place to:
- mkdir -p /etc/docker/certs.d/docker-registry.scubbo.org:8843
- cp /registry_cert.crt /etc/docker/certs.d/docker-registry.scubbo.org:8843/ca.crt
I don't consider this a complete answer (and would love further input or advice!), because:
I don't know why copying the file out of the mounted directory into /etc/docker/... (as in the original question) didn't work, but mounting the file directly from the host filesystem worked. (Note that the check-cert-persists-between-stages stage confirms that the certificate is correct, so it's not a mistake of copying a wrong or empty file)
I don't know how to mount the file directly into an in-stage path that contains a colon - this answer indicates how to mount a path containing a colon directly into a container, but in this case we're passing the path to DRONE_RUNNER_VOLUMES
I tried to connect on remote server with docker context
when tried docker ps from client(local mac) , I got this error message.
error during connect: Get "http://docker.example.com/v1.24/containers/json": command [ssh -l wwww -- tane-dev-0.ccn docker system dial-stdio] has exited with exit status 255, please make sure the URL is valid, and Docker 18.09 or later is installed on the remote host: stderr=wwww#xxx-dev-0.xxx: Permission denied (publickey,gssapi-keyex,gssapi-with-mic)
After googled about this problem, there's few things to checkup
docker version later 18.09 ✅
client: 20.10.14
remote: 20.10.17
register ssh key with ssh-agent ✅
ssh-add ~/.ssh/id_rsa
Identity added: /Users/ma_kyeongwook/.ssh/id_rsa (ma_kyeongwook#xxx.xxx)
ssh-add -l
4096 SHA256:~~~ ma_kyeongwook#xxx.xxx (RSA)
ssh connect test ✅
also checked cat ~/.ssh/authorized_keys contains client's pub key
Did i missed something?
I am having issues when trying to connect to a docker-compose network from inside of a container. These are the files I am working with. The whole thing runs when I ./run.sh.
Dockerfile:
FROM docker/compose:latest
WORKDIR .
# EXPOSE 8228
RUN apk update
RUN apk add py-pip
RUN apk add jq
RUN pip install anchorecli
COPY dockertest.sh ./dockertest.sh
COPY docker-compose.yaml docker-compose.yaml
CMD ["./dockertest.sh"]
docker-compose.yaml
services:
# The primary API endpoint service
engine-api:
image: anchore/anchore-engine:v0.6.0
depends_on:
- anchore-db
- engine-catalog
#volumes:
#- ./config-engine.yaml:/config/config.yaml:z
ports:
- "8228:8228"
..................
## A NUMBER OF OTHER CONTAINERS THAT ANCHORE-ENGINE USES ##
..................
networks:
default:
external:
name: anchore-net
dockertest.sh
echo "------------- INSTALL ANCHORE CLI ---------------------"
engineid=`docker ps | grep engine-api | cut -f 1 -d ' '`
engine_ip=`docker inspect $engineid | jq -r '.[0].NetworkSettings.Networks."cws-anchore-net".IPAddress'`
export ANCHORE_CLI_URL=http://$engine_ip:8228/v1
export ANCHORE_CLI_USER='user'
export ANCHORE_CLI_PASS='pass'
echo "System status"
anchore-cli --debug system status #This line throws error (see below)
run.sh:
#!/bin/bash
docker build . -t anchore-runner
docker network create anchore-net
docker-compose up -d
docker run --network="anchore-net" -v //var/run/docker.sock:/var/run/docker.sock anchore-runner
#docker network rm anchore-net
Error Message:
System status
INFO:anchorecli.clients.apiexternal:As Account = None
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): 172.19.0.6:8228
Error: could not access anchore service (user=user url=http://172.19.0.6:8228/v1): HTTPConnectionPool(host='172.19.0.6', port=8228): Max retries exceeded with url: /v1
(Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused',))
Steps:
run.sh builds container image and creates network anchore-net
the container has an entrypoint script, which does multiple things
firstly, it brings up the docker-compose network as detached FROM inside the container
secondly, nstalls anchore-cli so I can run commands against container network
lastly, attempts to get a system status of the anchore-engine (d.c network) but thats where I am running into HTTP request connection issues.
I am dynamically getting the IP of the api endpoint container of anchore-engine and setting the URL of the request to do that. I have also tried passing those variables from command line such as:
anchore-cli --u user --p pass --url http://$engine_ip/8228/v1 system status but that throws the same error.
For those of you who took the time to read through this, I highly appreciate any input you can give me as to where the issue may be lying. Thank you very much.
I've been battling an issue with a corporate proxy when trying to run docker-compose up -d nginx mysql
I'm attempting to run the Laradock container on OSX but keep running into errors when composer attempts to install dependencies. I've updated my docker settings to notify it about my corporate proxy:
Before adding the proxy information, I was receiving this error:
[Composer\Downloader\TransportException]
The "https://packagist.org/packages.json" file could not be downloaded: SSL operation failed with code 1. OpenSSL Error messages:
error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed
Since updating the proxy details, I am now receiving this error:
Step 27/183 : RUN if [ ${COMPOSER_GLOBAL_INSTALL} = true ]; then composer global install ;fi
---> Running in a7699d4ecebd
Changed current directory to /home/laradock/.composer
Loading composer repositories with package information
[Composer\Downloader\TransportException]
The "https://packagist.org/packages.json" file could not be downloaded: SSL: Success
Failed to enable crypto
failed to open stream: operation failed
I'm an experienced dev, but new to Docker. I think that the error is being caused because PHP is running inside the docker container but for some reason does not have access to my local certificates?
I was trying to run apt-get update in a docker container.
I got these errors:
W: Failed to fetch
http://ftp.osuosl.org/pub/mariadb/repo/10.2/debian/dists/jessie/Release.gpg
Cannot initiate the connection to 8000:80 (0.0.31.64).
- connect (22: Invalid argument)
W: Failed to fetch
http://security.debian.org/dists/jessie/updates/Release.gpg
Cannot initiate the connection to 8000:80 (0.0.31.64).
- connect (22: Invalid argument)
I googled around, and some problems related to docker apt-get are related to proxy settings or DNS settings. I think I have addressed both but I am still getting the above error. Any ideas?
proxy settings
Error messages would be this -- I am NOT seeing these errors anymore.
W: Failed to fetch
http://ftp.osuosl.org/pub/mariadb/repo/10.2/debian/dists/jessie/Release.gpg
Cannot initiate the connection to ftp.osuosl.org:80 (2600:3404:200:237::2).
- connect (101: Network is unreachable) [IP: 2600:3404:200:237::2 80]
W: Failed to fetch
https://repo.percona.com/apt/dists/jessie/main/binary-amd64/Packages
Connection timed out after 120001 milliseconds
My solution has been putting lines like these in my dockerfile. Since
then the error messages have changed, so I believe this is the right fix
for the proxy problem.
ENV http_proxy <myCorpProxy>:8000
ENV https_proxy <myCorpProxy>:8000
dns settings
The error would be this -- I am NOT seeing these errors anymore.
W: Failed to fetch
http://archive.ubuntu.com/ubuntu/dists/trusty/Release.gpg
Could not resolve 'my.proxy.net'
W: Failed to fetch
http://archive.ubuntu.com/ubuntu/dists/trusty-updates/Release.gpg
Could not resolve 'my.proxy.net'
W: Failed to fetch
http://archive.ubuntu.com/ubuntu/dists/trusty-security/Release.gpg
Could not resolve 'my.proxy.net'
Solutions: Fix Docker's networking DNS config
Other forums
I found a discussion thread on forums.docker.com but could not
reach a solution yet
https://forums.docker.com/t/cannot-run-apt-get-update-successfully-behind-proxy-beta-13-1/14170
Update with correct syntax of proxy settings that solved the problem!
Thanks for Matt's answer. I realized that the syntax that
I was using was wrong. They CANNOT be
ENV http_proxy <myCorpProxy.domain.name>:8000
ENV https_proxy <myCorpProxy.domain.name>:8000
but has to be
ENV http_proxy http://<myCorpProxy.domain.name>:8000
ENV https_proxy http://<myCorpProxy.domain.name>:8000
Once I changed that, my apt-get started to work. Thanks a lot!
It looks like your proxy settings are invalid. Running the following command produces the same error message:
docker run -ti --rm \
-e https_proxy=http://8000:80 \
-e http_proxy=http://8000:80 \
debian apt-get update
You need a valid http://hostname:port that you can connect to as a proxy
docker run -ti --rm \
-e https_proxy=http://10.8.8.8:3142 \
-e http_proxy=http://10.8.8.8:3142 \
debian apt-get update
You should be able to get some form of response from the proxy address
→ curl -v http://10.8.8.8:3142
* Rebuilt URL to: http://10.8.8.8:3142/
* Trying 10.8.8.8...
* Connected to 10.8.8.8 (10.8.8.8) port 3142 (#0)
> GET / HTTP/1.1
> Host: 10.8.8.8:3142
> User-Agent: curl/7.43.0
> Accept: */*
>
< HTTP/1.1 406 Usage Information
< Connection: close
< Transfer-Encoding: chunked
< Content-Type: text/html
<
I also met this problem on Mac OSX. I tried the solution from #Matt. I use docker run -it -e http_proxy=http://127.0.0.1:1235 -e https_proxy=http://127.0.0.1:1235 ubuntu to set the proxy. I got Could not connect to 127.0.0.1:1235 (127.0.0.1). - connect (111: Connection refused).
I use docker inspect <container_id> to get the proxy information of that container. I got "HTTP_PROXY=docker.for.mac.localhost:1235", "HTTPS_PROXY=docker.for.mac.localhost:1235". I change the command to docker run -it -e http_proxy=http://docker.for.mac.localhost:1235 -e https_proxy=http://docker.for.mac.localhost:1235 ubuntu. It works.