Checkout from Gitlab is getting fail in Jenkins pipeline - jenkins

I have read lots of documentation but not able to fix this issue.
I used below code for checkout from gitlab in jenkinfile
checkout changelog: true, poll: true, scm: [
$class: 'GitSCM',
branches: [[name: "origin/${env.gitlabSourceBranch}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [[
$class: 'PreBuildMerge',
options: [
fastForwardMode: 'FF',
mergeRemote: 'origin',
mergeStrategy: 'default',
mergeTarget: "${env.gitlabTargetBranch}"
]
]],
submoduleCfg: [],
userRemoteConfigs: [[
credentialsId: 'gitlab-jenkins-user-credentials',
name: 'origin',
url: "${env.gitlabSourceRepoHttpUrl}"
]]
]
I even tried
checkout changelog: false, poll: false, scm: [$class: 'GitSCM', branches: [[name: '*/master']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'gitlab-jenkins-user-credentials', url: 'http://jenkins-master/testproject/devops_artifacts/test_devops.git']]]
But I am keep getting this error:
Warning: CredentialId "gitlab-jenkins-user-credentials" could not be found.
Cloning the remote Git repository
Cloning repository http://gitlab-master.com/testproject/devops_artifacts/test_devops.git
> C:\Program Files\Git\cmd\git.exe init C:\jenkins-docker\workspace\wildfly_gitlab # timeout=10
Fetching upstream changes from http://gitlab-master.com/testproject/devops_artifacts/test_devops.git
> C:\Program Files\Git\cmd\git.exe --version # timeout=10
> C:\Program Files\Git\cmd\git.exe fetch --tags --force --progress http://gitlab-master.com/testproject/devops_artifacts/test_devops.git +refs/heads/*:refs/remotes/origin/*
ERROR: Error cloning remote repo 'origin'
hudson.plugins.git.GitException: Command "C:\Program Files\Git\cmd\git.exe fetch --tags --force --progress http://gitlab-master.com/testproject/devops_artifacts/test_devops.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout:
stderr: Logon failed, use ctrl+c to cancel basic credential prompt.
remote: HTTP Basic: Access denied
fatal: Authentication failed for 'http://gitlab-master.com/testproject/devops_artifacts/test_devops.git/'
I have set webhook in this form
http://username:c4cc893d00cfc8865fc3#10.50.9.XXX:8080/project/wildfly_gitlab
And I tested the connection it is working. Same way I configured Gitlab in Jenkins as well
And connection is working as well. I don't know why I am still getting the error.

I assume your "gitlab-jenkins-user-credentials" is a user credential (credential scoped to a specific user).
You can circumvent by getting the user credential with withCredentials and then using the https with credentials url in the userRemoteConfigs config:
parameters {
credentials(credentialType: 'com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl', defaultValue: '', description: 'Jenkins wants to use the "gitlab-jenkins-user-credentials" credentials. Select your credential to allow this.', name: 'gitlab-jenkins-user-credentials', required: true)
}
...
withCredentials([usernamePassword(credentialsId: "gitlab-jenkins-user-credentials", passwordVariable: 'GITLAB_PASSWORD', usernameVariable: 'GITLAB_USER')]) {
checkout([$class: 'GitSCM', branches: [[name: 'refs/tags/mytag']], userRemoteConfigs: [[url: "https://${GITLAB_USER}:${GITLAB_PASSWORD}#gitlab.com/myproject.git"]]])
}
gitlab-jenkins-user-credentials is a user credential of type " Username with password" credential created with:
id: gitlab-jenkins-user-credentials
username = gitlab token name
password = gitlab token
When running the job, the user will be prompted to select it's user credential.
However note that groovy interpolation of secrets is not a good practice on a security point of view (see https://www.jenkins.io/doc/book/pipeline/jenkinsfile/#string-interpolation).

Related

How to write 'Git LFS pull after checkout' settings in the Jenkins pipeline

I created a configuration file with Jenkins file to configure the Jenkins pipeline.
An error occurred when doing git pull.
The cause was that the groovy file did not have the Git LFS pull after checkout setting.
I do not know how to write Git LFS pull after checkout setting to groovy.
git(
 url: git#...,
 branch: "master",
 credentialsId:"abcdefg"
)
// Git LFS pull after checkout setting??
Here's how I was able to use the Git plugin in the pipeline. Refer to the documentation here for more info:
checkout([ $class: "GitSCM",
branches: [[name: "refs/heads/${your branch name}"]],
extensions: [
[$class: "GitLFSPull"]
],
userRemoteConfigs: [
[credentialsId: "${your git credential ID}",
url: "${your git URL}"]
]
])
Considering you are trying to download a large file, you may want to increase the time out limit too (default is set to 10 minutes):
checkout([ $class: 'GitSCM',
branches: [[name: 'refs/heads/'+"${branch_or_tag}"]],
extensions: [[$class: 'GitLFSPull']]
+[[$class: 'CloneOption', timeout: 30]],
userRemoteConfigs: [
[credentialsId: "${your git credential ID}",
url: "${your git URL}"]
]
])

Jenkins Sshagent plugin in slave not working as expected

I am trying to use the sshagent and credentials plugin to execute couple of git commands on a remote repo in a jenkins pipeline script .
I am trying to do this checkout some files from a large repo . This runs on a docker slave .
sshagent(['a5f11347-dff6-4586-ae77-34adeffb0063']) {
script {
sh 'git archive --remote=ssh://git#git.comp.com/something.git HEAD Jenkinsfile | tar -x'
sh 'git archive --remote=ssh://git#git.comp.com/something.git HEAD:pipeline/ | tar -x;'
}
}
But it fails with a hostkey verification failed error. Any insight on why this is failing .
If i do a
checkout([$class: 'GitSCM', branches: [[name: "${BRANCH_NAME}"]],
doGenerateSubmoduleConfigurations: false,
extensions: [[$class: 'CloneOption', depth: 0, noTags: false, reference: '', shallow: true],
[$class: 'CloneOption', noTags: true, reference: '', shallow: true],
[$class: 'SparseCheckoutPaths', sparseCheckoutPaths: [[path: 'Jenkinsfile'], [path: 'pipeline/*']]]],
submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'a5f11347-dff6-4586-ae77-34adeffb0063',
url: 'ssh://git#git.comp.com/something.git']]])
it works fine ,but the large repo is fetched ,which i want to avoid
Regarding the host key verification problem, you need to add the host key for git.comp.com to your SSH known_hosts file. Try integrating the following command in your Jenkins script before the sshagent block:
ssh-keyscan git.comp.com >> ~/.ssh/known_hosts
This worked for me.
See Jenkins ssh-agent starts and then stops immediately in pipeline build for more info.

Jenkins pipeline polling multiple scm issue

I have a Jenkins pipeline script, which is supposed get 2 repositories from GitLab (triggered by Gitlab webhook), if there are changes on "master" branch in any of the repositories (and so more stuff afterwards).
My job DSL config:
pipelineJob('pipline') {
// Enable but do not use SCM polling. This needs to be enabled for
// notifyCommit to work but we don't want to do any actual polling.
configure { project ->
project / 'triggers' << 'hudson.triggers.SCMTrigger' {
spec('')
}
}
definition {
cps {
sandbox(true)
script ('''node('Foo') {
//repo abc
checkout changelog: false,
poll: true,
scm: [$class: 'GitSCM',
branches: [[name: 'refs/heads/master']],
doGenerateSubmoduleConfigurations: false,
extensions: [
[$class: 'SubmoduleOption',
disableSubmodules: true,
parentCredentials: false,
recursiveSubmodules: false,
reference: '',
trackingSubmodules: false],
[$class: 'RelativeTargetDirectory',
relativeTargetDir: "abc"]
],
submoduleCfg: [],
userRemoteConfigs: [
[url: '<repository abc>'],
[refspec: '+refs/heads/master:refs/remotes/origin/master'],
[name: 'origin']
]]
//repo def
checkout changelog: false,
poll: true,
scm: [$class: 'GitSCM',
branches: [[name: 'refs/heads/master']],
doGenerateSubmoduleConfigurations: false,
extensions: [
[$class: 'CheckoutOption', timeout: 20],
[$class: 'SubmoduleOption',
disableSubmodules: true,
parentCredentials: false,
recursiveSubmodules: false,
reference: '',
trackingSubmodules: false],
[$class: 'RelativeTargetDirectory',
relativeTargetDir: "def"],
[$class: 'PruneStaleBranch']],
submoduleCfg: [],
userRemoteConfigs: [
[url: '<repository def>'],
[refspec: '+refs/heads/master:refs/remotes/origin/master'],
[name: 'origin']
]]
...
do more things
''')
}
}
}
The polling working fine for the first repository, but when I have changes in a seconds repository not in a master branch, I still get the whole thing started. It looks that it takes last build revision SHA only from the first repository and then uses it to compare in all other repositories.
here's how the git polling log looks like:
Started on Feb 15, 2017 12:10:09 PM
Using strategy: Default
[poll] Last Built Revision: Revision <SHA from last commit in repository abc on master branch > (refs/remotes/origin1/master)
> git ls-remote -h <repository abc> # timeout=10
Found 4 remote heads on <repository abc>
[poll] Latest remote head revision on refs/heads/master is: <SHA from last commit in repository abc on master branch > - already built by 76
Using strategy: Default
[poll] Last Built Revision: Revision <SHA from last commit in repository abc on master branch > (refs/remotes/origin1/master)
> git ls-remote -h <repository def> # timeout=10
Found 2 remote heads on <repository def>
[poll] Latest remote head revision on refs/heads/master is: <SHA from last commit in repository def on master branch >
Done. Took 1.1 sec
Changes found
I was using buildFlowJob before with same functionality and it worked fine.

Jenkins groovy pipeline - git checkout error

I have defined a simple Jenkins pipeline using the Groovy DSL. My intention is that it will simply checkout my git repository. However I am getting an error.
Given my groovy DSL defintion:
stage 'build'
node
{
git branch: '*/mybranch', credentialsId: 'my credentials', url: 'git#git.servername:pathto/myrepo.git'
}
I'm expecting the Jenkins pipeline to simply checkout my git repository.
However I'm getting the following error:
ERROR: Couldn't find any revision to build. Verify the repository and branch configuration for this job.
Finished: FAILURE
Can anyone help me in resolving this issue please?
The more verbose version works:
checkout([$class: 'GitSCM', branches: [[name: '*/mybranch']], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'myCredentials', url: 'git#git.servername:pathto/myrepo.git']]])
However it would be nice to use the short-hand version.
Without the */ on the branch specifier works for me.
Try this:
git branch: 'mybranch', credentialsId: 'my credentials', url: 'git#git.servername:pathto/myrepo.git'

Change username in submodule checkout in Jenkins Pipeline DSL

I have a project that spins up EC2 instances as build slaves, the username is "ec2-user".
I am checking out a repository with submodules like this:
checkout([
$class: 'GitSCM',
branches: [[name: 'deadbeefdeadbeefcafebabe']],
doGenerateSubmoduleConfigurations: false,
extensions: [
[$class: 'SubmoduleOption',
disableSubmodules: false,
parentCredentials: true,
recursiveSubmodules: true,
reference: '',
trackingSubmodules: false]
], submoduleCfg: [],
userRemoteConfigs: [
[credentialsId: 'deadbeef-cafe-babe-cafe-babebeef1234',
refspec: 'refs/changes/12/34567/89',
url: 'ssh://user#host:29418/some/project']
]
])
Please note that the URL has a username.
Now, the .gitmodules file looks like this:
[submodule "path/in/project"]
path = path/in/project
url = ssh://host:29418/other/project
branch = somebranch
The submodule description does not have a username specified. When it is time for the submodule clone, the clone is denied with invalid credentials. Manually editing the .gitmodules file and adding the user# in front of the URL works, but this is a bad workaround at best. Changing the URL on disk and then calling checkout again does also not work.
How can I make Jenkins use the username it used in the parent checkout? If that is not possible, what is a workaround which does not involve changing the source repo?

Resources