I've a simple Jenkins pipeline which will pull a docker image (from a private registry)and execute command, I'm failing to understand as despite providing credential ID, pipeline fails with error message ERROR: Could not find credentials matching docker-cred
Pipeline:
pipeline
{
agent
{
docker {
image "xxxxxxx/dotnet:latest"
registryUrl 'xxxxxxx'
registryCredentialsId "docker-cred"
reuseNode true
}
}
stages
{
stage('Test')
{
steps
{
sh 'dotnet --version'
}
}
}
}
Credentials added:
According to the documentation you need to use "Username/Password" credentials type:
For a Docker Registry which requires authentication, add a "Username/Password" Credentials item from the Jenkins home page and use the Credentials ID as a second argument to withRegistry():
...
Related
I am in multi-branch option with my jenkins and I have a problem of authentication to Gitlab. Here is my jenkins file :
pipeline {
agent any
environment {
registry = "*****#gmail.com/test"
registryCredential = 'test'
dockerImage = ''
}
stages {
stage('Cloning our Git') {
steps{
git 'https://gitlab.com/**********/*************/************.git'
}
}
stage('Build docker image') {
steps {
script {
dockerImage = docker.build registry + ":$BUILD_NUMBER"
}
}
}
stage('Deploy our image') {
steps{
script {
docker.withRegistry( '', registryCredential ){
dockerImage.push()
}
}
}
}
stage('Cleaning up') {
steps{
sh "docker rmi $registry:$BUILD_NUMBER"
}
}
}
}
This is the error I got:
Caused by: hudson.plugins.git.GitException: Command "git fetch --tags --force --progress -- https://gitlab.com/************/*******/***************.git +refs/heads/:refs/remotes/origin/" returned status code 128:
stdout:
stderr: remote: HTTP Basic: Access denied. The provided password or token is incorrect or your account has 2FA enabled and you must use a personal access token instead of a password. See https://gitlab.com/help/topics/git/troubleshooting_git#error-on-git-fetch-http-basic-access-denied
I would like to know how to authenticate with the jenkinsfile to gitlab or if you have a better solution for me I am interested. Thanks
If you follow the link provided in the error message, you end up here:
https://docs.gitlab.com/ee/user/profile/account/two_factor_authentication.html#troubleshooting
You need to create a Personal Access Token which is kind of a special ID to delegate access to parts of your account rights.
The documentation for PAT is here:
https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html
In the Gitlab repository interface, it is under Settings > Access Tokens.
As you try to read an HTTPS repository, it seems you need to create a token with rights read_repository.
Then you should be able to access the repository with:
https://<my-user-id>:<my-pat>#gitlab.com/<my-account>/<my-project-name>.git
I'm trying to write a JenkinsFile that automatically will reach to a git repo via ssh and perform some actions but I want to make the repo and ssh key use variables with the ssh id stored in Jenkins but I seem to be missing the Jenkins documentation for how to pass down variables to Jenkins Files as I'm not able to pass values down into the credentials key. The variables being passed down to the sh commands resolve perfectly fine though...
Example Pipeline Below:
pipeline {
parameters {
string(name: 'SSH_priv', defaultValue: 'd4f19e34-7828-4215-8304-a2d1f87a2fba', description: 'SSH Credential with the private key added to Jenkins and the public key to the username stored in Git Server, this id can be found in the credential section of Jenkins post its creation.')
string(name: 'REPO', defaultValue: 'git#--------------------')
}
stages {
stage ('Output Variables'){
// checks I can get these variables
steps{
sh("echo ${params.SSH_priv}")
sh("echo ${params.REPO}")
}
}
stage('Do Something') {
steps {
// this below commented line, does not work.
// sshagent (credentials: ['${params.SSH_priv}']){
// this line does work
sshagent (credentials: ['d4f19e34-7828-4215-8304-a2d1f87a2fba']){
sh("git clone --mirror ${params.REPO} temp")
dir("temp"){
// start doing fancy stuff ...
....
....
}
}
}
}
The aim is a Pipeline that my fellow developers could call and will work with their own repos and own ssh id's that I'm not using. When I try to run this with the SSH_priv parameter passing down the value I get the below failure in Jenkins.
The JenkinsFile works perfectly fine with the credential id hard-coded- as shown below:
So after testing different things a friend solved this in sub 5 minutes. Quotation mark types matter in Groovy Script
Changing
sshagent (credentials: ['${params.SSH_lower}']){
To
sshagent (credentials: ["${params.SSH_lower}"]){
Solved the issue.
Better to use environment step in pipeline.
pipeline {
agent any
environment {
AN_ACCESS_KEY = credentials('an_access_key_id')
}
stages {
stage('Example') {
steps {
sh 'printenv'
}
}
}
}
And credentials should exist in jenkins with id an_access_key_id
Take a look at official documentation here
I am able to connect to both private registries from Jenkins and I can pull the image I want to, however I don't know how to push that same image to a different repo.
Note, I am using scripted pipeline syntax since declarative syntax doesn't support pushing/pulling or custom registries as far as I know. I'm also not familiar with Groovy syntax.
Here's what I've got so far for my Jenkinsfile:
node {
checkout scm
docker.withRegistry('https://private-registry-1', 'credentials-1') {
def image = docker.image('my-image:tag')
image.pull()
docker.withRegistry('https://private-registry-2', 'credentials-2') {
image.push()
}
}
}
I put the second "withRegistry()" method within the first so that I could use the defined "image" variable.
I successfully connect to the first registry and pull the latest image. From Jenkins console output:
Login Succeeded
[Pipeline] {
[Pipeline] sh
+ docker pull private-registry-1/my-image:tag
tag: Pulling from my-image
Digest: sha256:XXXXX
Status: Image is up to date for private-registry-1/my-image:tag
However, here's the relevant error snippet after connecting to the second registry:
...
Login Succeeded
[Pipeline] {
[Pipeline] sh
+ docker tag my-image:tag private-registry-2/my-image:tag
Error response from daemon: No such image: my-image:tag
...
I am using a Jenkins container on my local Windows machine. It's connected to Docker for Windows through my Ubuntu terminal (Windows Subsystem for Linux).
The solution was to tag the image before pushing it, final code:
node {
checkout scm
stage 'Pull latest image from private-registry-1'
def image
docker.withRegistry('https://private-registry-1', 'credentials-1') {
image = docker.image('my-image:tag')
image.pull()
}
stage 'Push image to private-registry-2'
// SOLUTION START
sh 'docker tag private-registry-1/my-image:tag private-registry-2/my-image:tag'
image = docker.image('private-registry-2/my-image:tag')
// SOLUTION END
docker.withRegistry('https://private-registry-2', 'credentials-2') {
image.push()
}
}
I don't like how the tagging is done manually through "sh" but I couldn't find a way to do it through the built-in Docker syntax. I will also need to parameterize the image name and tag (my-image:tag) for future use.
For the declarative syntax the following worked for me:
pipeline {
agent {
docker {
label 'service'
alwaysPull false
registryUrl "${my-private-docker-registry_url_with_https}"
registryCredentialsId "${jenkins_credential_id_for_login}"
image 'lambci/lambda:build-python3.7'
args '-v /var/run/docker.sock:/var/run/docker.sock --network host'
}
}
Thanks to the answer from VictoryShoe!
One thing is important and took a long time for me to find out the mistake:
"docker.withRegistry('https://private-registry-x', 'credentials-x')" -> Do not forget to add the "https://" at the front of the registryURL
In command line "sh 'docker tag private-registry-1/my-image:tag private-registry-2/my-image:tag'", please do NOT add any "https://" in the front of the registryURL
The following jenkinsfile has worked for me:
PS: in my use case, i pull a source-image from DockerHub, tag this image, then push this target-image to a private company image registry.
def registryCredentials1 = "cridentialIdOfJenkinsForRegistry1"
def registryCredentials2 = "cridentialIdOfJenkinsForRegistry2"
def protocol = "https://"
def registryURL1 = "registry.hub.docker.com"
def registryURL2= "harbor.mycompany.xx.yy.com"
pipeline {
agent any
parameters {
string(name: 'sourceImageName', defaultValue: '', description: 'Source-Image-Name, name-schema is like user/foo, e.g. jenkins/jenkins')
string(name: 'sourceImageTag', defaultValue: '', description: 'Source-Image-Tag, e.g. lts')
string(name: 'targetImageName', defaultValue: '', description: 'Target-Image-Name, name-schema is like user/foo, e.g. jenkins/jenkins')
string(name: 'targetImageTag', defaultValue: '', description: 'Target-Image-Tag, e.g. lts')
}
stages {
stage('Pull source-image from Registry 1 & tag the image') {
steps {
script {
//pull source-image from registry 1
docker.withRegistry(protocol + registryURL1, registryCredentials1) {
docker.image("${params.sourceImageName}:${params.sourceImageTag}").pull()
}
//tag the image
sh "docker tag ${registryURL1}/${params.sourceImageName}:${params.sourceImageTag} ${registryURL2}/${params.targetImageName}:${params.targetImageTag}"
}
}
}
stage('push target-image to Registry 2') {
steps {
script {
//push target-image to registry 2
docker.withRegistry(protocol + registryURL2, registryCredentials2) {
sh "docker push ${registryURL2}/${params.targetImageName}:${params.targetImageTag}"
}
}
}
}
}
}
I have a private GitHub Rust project that depends on another private GitHub Rust project and I want to build the main one with Jenkins. I have called the organization Organization and the dependency package subcrate in the below code.
My Jenkinsfile looks something like
pipeline {
agent {
docker {
image 'rust:latest'
}
}
stages {
stage('Build') {
steps {
sh "cargo build"
}
}
etc...
}
}
I have tried the following in Cargo.toml to reference the dependency, it works fine on my machine
[dependencies]
subcrate = { git = "ssh://git#ssh.github.com/Organization/subcrate.git", tag = "0.1.0" }
When Jenkins runs I get the following error
+ cargo build
Updating registry `https://github.com/rust-lang/crates.io-index`
Updating git repository `ssh://git#github.com/Organization/subcrate.git`
error: failed to load source for a dependency on `subcrate`
Caused by:
Unable to update ssh://git#github.com/Organization/subcrate.git?tag=0.1.0#0623c097
Caused by:
failed to clone into: /usr/local/cargo/git/db/subcrate-3e391025a927594e
Caused by:
failed to authenticate when downloading repository
attempted ssh-agent authentication, but none of the usernames `git` succeeded
Caused by:
error authenticating: no auth sock variable; class=Ssh (23)
script returned exit code 101
How can I get Cargo to access this GitHub repository? Do I need to inject the GitHub credentials onto the slave? If so, how can I do this? Is it possible to use the same credentials Jenkins uses to checkout the main crate in the first place?
I installed the ssh-agent plugin and updated my Jenkinsfile to look like this
pipeline {
agent {
docker {
image 'rust:latest'
}
}
stages {
stage('Build') {
steps {
sshagent(credentials: ['id-of-github-credentials']) {
sh "ssh -vvv -T git#github.com"
sh "cargo build"
}
}
}
etc...
}
}
I get the error
+ ssh -vvv -T git#github.com
No user exists for uid 113
script returned exit code 255
Okay, I figured it out, No user exists for uid error is because of a mismatch between the users in the host /etc/passwd and the container /etc/passwd. This can be fixed by mounting /etc/passwd.
agent {
docker {
image 'rust:latest'
args '-v /etc/passwd:/etc/passwd'
}
}
Then
sshagent(credentials: ['id-of-github-credentials']) {
sh "cargo build"
}
Works just fine
I'm new to using Jenkins....
I'm trying to automate the production of an image (to be stashed in a repo) using a declarative Jenkinsfile. I find the documentation to be confusing (at best). Simply put, how can I convert the following scripted example (from the docs)
node {
checkout scm
def customImage = docker.build("my-image:${env.BUILD_ID}")
customImage.push()
}
to a declarative Jenkinsfile....
You can use scripted pipeline blocks in a declarative pipeline as a workaround
pipeline {
agent any
stages {
stage('Build image') {
steps {
echo 'Starting to build docker image'
script {
def customImage = docker.build("my-image:${env.BUILD_ID}")
customImage.push()
}
}
}
}
}
I'm using following approach:
steps {
withDockerRegistry([ credentialsId: "<CREDENTIALS_ID>", url: "<PRIVATE_REGISTRY_URL>" ]) {
// following commands will be executed within logged docker registry
sh 'docker push <image>'
}
}
Where:
CREDENTIALS_ID stands for key in Jenkis under which you store credentials to your docker registry.
PRIVATE_REGISTRY_URL stands for url of your private docker registry. If you are using docker hub then it should be empty.
I cannot recommend the declarative syntax for building a Docker image bcos it seems that every important step requires falling back to the old scripting syntax. But if you must, a hybrid approach seems to work.
First a detail about the scm step: when I defined the Jenkins "Pipeline script from SCM" project that fetches my Jenkinsfile with a declarative pipline from git, Jenkins cloned the repo as the first step in the pipeline even tho I did not define a scm step.
For the build and push steps, I can only find solutions that are a hybrid of old-style scripted pipeline steps inside the new-style declarative syntax. For example see gustavoapolinario's work at Medium:
https://medium.com/#gustavo.guss/jenkins-building-docker-image-and-sending-to-registry-64b84ea45ee9
which has this hybrid pipeline definition:
pipeline {
environment {
registry = "gustavoapolinario/docker-test"
registryCredential = 'dockerhub'
dockerImage = ''
}
agent any
stages {
stage('Cloning Git') {
steps {
git 'https://github.com/gustavoapolinario/microservices-node-example-todo-frontend.git'
}
}
stage('Building image') {
steps{
script {
dockerImage = docker.build registry + ":$BUILD_NUMBER"
}
}
}
stage('Deploy Image') {
steps{
script {
docker.withRegistry( '', registryCredential ) {
dockerImage.push()
}
}
}
}
stage('Remove Unused docker image') {
steps{
sh "docker rmi $registry:$BUILD_NUMBER"
}
}
}
}
Because the first step here is a clone, I think he built this example as a standalone pipeline project in Jenkins (not a Pipeline script from SCM project).