I have a jenkins job and we build our docker containers using the docker build and publish plugin.
Most of the containers are built and published to our private repos in dockerhub without any issues. They inherit images from public repositories and work fine.
However we have noticed that dockerfiles that contains
FROM private_repo:tag
fail because docker cannot find the image in our private repo. We have provided credentials via the docker registry credentials option but doesn't work. It simply says
"image not found"
at the end of the build.
If I login to our jenkins server and do docker login and then pull the image that works fine. This leads me to believe something is wrong with the credentials when it is pulling from the private repo. Has anyone encountered this or know the issue?
I have always run docker login -u $DOCKER_USERNAME -p $DOCKER_PASSWORD at the beginning of my docker build / docker push Jenkins jobs. Save the username/password as a Jenkins credential and you won't have to share it anywhere.
Related
I was wondering is it possible to use cached docker images in gitlab registry for gitlab-ci?
for example, I want to use node:16.3.0-alpine docker image, can I cache it in my gitlab registry and pull it from that and speed up my gitlab ci instead of pulling it from docker hub?
Yes, GitLab's dependency proxy features allow you to configure GitLab as a "pull through cache". This is also beneficial for working around rate limits of upstream sources like dockerhub.
It should be faster in most cases to use the dependency proxy, but not necessarily so. It's possible that dockerhub can be more performant than a small self-hosted server, for example. GitLab runners are also remote with respect to the registry and not necessarily any "closer" to the GitLab registry than any other registry over the internet. So, keep that in mind.
As a side note, the absolute fastest way to retrieve cached images is to self-host your GitLab runners and hold images directly on the host. That way, when jobs start, if the image already exists on the host, the job will start immediately because it does not need to pull the image (depending on your pull configuration). (that is, assuming you're using images in the image: declaration for your job)
I'm using a corporate Gitlab instance where for some reason the Dependency Proxy feature has been disabled. The other option you have is to create a new Docker image on your local machine, then push it into the Container Registry of your personal Gitlab project.
# First create a one-line Dockerfile containing "FROM node:16.3.0-alpine"
docker pull node:16.3.0-alpine
docker build . -t registry.example.com/group/project/image
docker login registry.example.com -u <username> -p <token>
docker push registry.example.com/group/project/image
where the image tag should be constructed based on the example given on your project's private Container Registry page.
Now in your CI job, you just change image: node:16.3.0-alpine to image: registry.example.com/group/project/image. You may have to run the docker login command (using a deploy token for credentials, see Settings -> Repository) in the before_script section -- I think maybe newer versions of Gitlab will have the runner authenticate to the private Container Registry using system credentials, but that could vary depending on how it's configured.
I am trying to pull and push images between Docker Desktop and Azure and Visual Studio 2019.
currently I can push from VS2019 by Publish option and I can push to Docker and Azure Container Registry.
How do I pull from Azure to Docker? I believe there is an issue with security accounts between the 2 systems. After all, my Docker account is not my Azure account. I came across this article
https://learn.microsoft.com/en-us/azure/container-registry/container-registry-auth-service-principal
which contains a script. Is this the right article to solve my problem? I made a copy of the script but I am struggling to run it. If I save it to assignpermissions.sh file and run wsl ./assignpermissions.sh it complains that az does not exist.
So
Is that the right article to help me (eventually) pull and push between Azure and Docker?
How do I run the script when calling az is causing an error?
Any other things I need to watch out for in the next step?
Log in to a registry
There are several ways to authenticate to your private container registry.
Azure CLI
The recommended method when working in a command line is with the Azure CLI command az acr login. For example, to log in to a registry named myregistry, log into the Azure CLI and then authenticate to your registry:
az login
az acr login --name myregistry
Azure PowerShell
The recommended method when working in PowerShell is with the Azure PowerShell cmdlet Connect-AzContainerRegistry. For example, to log in to a registry named myregistry, log into Azure and then authenticate to your registry:
Connect-AzAccount
Connect-AzContainerRegistry -Name myregistry
You can also log in with docker login. For example, you might have assigned a service principal to your registry for an automation scenario. When you run the following command, interactively provide the service principal appID (username) and password when prompted. For best practices to manage login credentials, see the docker login command reference:
docker login myregistry.azurecr.io
Both commands return Login Succeeded once completed.
Note: You might want to use Visual Studio Code with Docker extension for a faster and more convenient login.
Tip: Always specify the fully qualified registry name (all lowercase) when you use docker login and when you tag images for pushing to your registry. In the examples in this article, the fully qualified name is myregistry.azurecr.io.
Push the image to your registry
Now that you've tagged the image with the fully qualified path to your private registry, you can push it to the registry with docker push:
docker push myregistry.azurecr.io/samples/nginx
Pull the image from your registry
Use the docker pull command to pull the image from your registry:
docker pull myregistry.azurecr.io/samples/nginx
I setup docker automated build via Github and I can build an image successfully from a Dockerfile in my GitHub repository
I have followed the thread here and here
According to the documentation, if I do my build via an automated build connected to Github, all I need is to have my Dockerfile present and it will be added.
The builds succeeds, so the automation process works.
I can even see the Readme from GitHub in the registry. But for some reason, I can't see my Dockerfile in Docker hub.
Here is an example Github repo and here is the repository in Docker hub
Is there any more configuration necessary to get the Dockerfile to docker hub?
welcome Jungo!
TL;DR
You can learn more information about why on the Docker forum.
I need to pull all images from an openshift template file, in my case it's openwhisk.
I'm trying to deploy this project on a private network so I don't have access to docker's official repository from there and thus need to push the images myself.
I was hoping there is a script/tool to automate this process.
There is no such available tool/script but you can write small shell script to do it.
If public dockerhub registry not allowed then either use private separate registry
or
Pull the image in your local laptop then tag it and push to openshift registry.
After pushing all the image to openshift, import your openshift template to deploy your application.
Below is the steps for single image. you can define list of image and loop it over the list.
docker pull imagename
oc login https://127.0.0.1:8443 --token=<hidden_token> #copy from https://your_openshift_server:port/console/command-line
oc project test
oc create imagestream imagename
docker login -u `oc whoami` -p `oc whoami --show-token` your_openshift_server:port
docker tag imagename your_openshift_server:port/openshift_projectname/imagename:tag
docker push your_openshift_server:port/openshift_projectname/imagename:tag
you can get more details on page suggested by graham-dumpleton
.
Graham Dumpleton's book talks about this. You create a list (JSON) of all the images used and import that into the openshift namespace. Since your OpenShift is offline/disconnected, you'll also change any remote registry to the URL of the internal, hosted registry.
Example that imports all JBoss images: https://github.com/projectatomic/adb-utils/blob/master/services/openshift/templates/common/jboss-image-streams.json
My actual requirement is pull docker image from GitHub and build a docker image in ec2 instance and push that image to ecr. So, am just trying to clear my first step by asking help to pull image from git, very new to all this.
Let's walk through each step you're asking about in your requirements:
Pull from GitHub - You won't pull a docker image from here, however you may pull a Dockerfile from here, which would be used to build an image. The command to do this would be just like cloning any other repository: git clone <repository url>
Build the image on ec2 - First you will need to have docker installed on the ec2 instance. Assuming you're running Ubuntu on your ec2 instance, follow the good instructions on Docker's page (https://docs.docker.com/install/linux/docker-ce/ubuntu/) miror. Once docker is installed, navigate to the directory that has your Dockerfile in it (cloned from git) and type docker build . --tag mytag
Push the image to ecr - To do this, you need to have the amazon CLI installed on your box, and you need an ACCESS_KEY_ID and SECRET_ACCESS_KEY from AWS IAM. Once you have these, configure your connection by storing them as environment variables, or by typing aws configure and entering them. Once your credentials are configured, log into ECR by typing aws ecr get-login --no-include-email, and then copy/pasting the command it gives you. (you can also put ` around it to skip the copying step). This will allow you to push to ecr using docker push.
To clarify some of the points:
Github: It is a web-based hosting service for version control using git. So you can not pull docker image from Github.
To build a Docker image, you need Dockerfile. So you can fork the GitHub project which has this Dockerfile.
Then to build it on ec2, you can check out the project containing Dockerfile on ec2 server and build it using:
https://docs.docker.com/engine/reference/commandline/build/
and then you can push it to any registry using:
https://docs.docker.com/engine/reference/commandline/push/