I am trying to link a gitlab instance with a Jenkins instance. Due to being run in a DC/OS environment I cannot use the standard SSH port 22, so I am running gitlab on SSH port 16122 instead.
Gitlab displays SSH clone URLs in the format [git#git.company.de:16122]:user/test.git, which a normal git clone command understands. Also, Jenkins accepts this URI as SCM URI in the configuration screen (as evidenced by the error messages "invalid credentials" vanishing once I set the SSH key as credential that is configured as deploy-key in Gitlab.
The problem arises during actually building the job: jenkins for some unknown reason decides to double-urlencode the [ at the front of the URI and thus the build breaks:
Cloning repository %255bgit#git.company.de:16122]:user/test.git
> git init /var/jenkins_home/workspace/test # timeout=10
Fetching upstream changes from %25255bgit#git.company.de:16122]:user/test.git
> git --version # timeout=10
> git fetch --tags --progress %25255bgit#git.company.de:16122]:user/test.git +refs/heads/*:refs/remotes/origin/*
ERROR: Error cloning remote repo 'origin'
hudson.plugins.git.GitException: Command "git fetch --tags --progress %25255bgit#git.company.de:16122]:user/test.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
When I look at the raw XML of the job using xxd on the shell, there are no "invisible" ASCII chars or similar in front of the repo URL.
What causes this, and is there a way to fix this without having to manually rewrite every URL to ssh://git#git.company.de:16122/user/test.git?
Related
I'm trying to integrate Jenkins (ver. 2.337) for our project on our gitlab.doman.com (GitLab Community Version 13.0.3), and it is deployed and running on Linux Centos on AWS EC2.
When I try to test using my GitLab Access Token, it gives me error stating Connection Timed Out.
When I try with Gitlab webhooks and SSH Key as credentials instead of Access Token, webhooks execute successfully, but the job on jenkins fails with the below errors.
I am quite new to Jenkins, and CI/CD. So, any guidance or help resolving this issue is greatly appreciated!
Jenkins Console Log:
Started by GitLab push by j****
Running as SYSTEM
Building in workspace /var/lib/jenkins/workspace/xx_xxxxx
The recommended git tool is: NONE
using credential 7c284997-927a-40d9-b1cf-02axxxxxxxxx
> /usr/bin/git rev-parse --resolve-git-dir /var/lib/jenkins/workspace/xx_xxxxx/.git # timeout=10
Fetching changes from the remote Git repository
> /usr/bin/git config remote.origin.url git#gitlab.sampledomain.co.kr:xx_xxxxxDetection/xx_xxxxx_detection.git # timeout=10
Fetching upstream changes from git#gitlab.sampledomain.co.kr:xx_xxxxxDetection/xx_xxxxx_detection.git
> /usr/bin/git --version # timeout=10
> git --version # 'git version 1.8.3.1'
using GIT_SSH to set credentials GitLab_SSH_Key
> /usr/bin/git fetch --tags --progress git#gitlab.sampledomain.co.kr:xx_xxxxxDetection/xx_xxxxx_detection.git +refs/heads/*:refs/remotes/origin/* # timeout=10
ERROR: Error fetching remote repo 'origin'
hudson.plugins.git.GitException: Failed to fetch from git#gitlab.sampledomain.co.kr:xx_xxxxxDetection/xx_xxxxx_detection.git
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:1001)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1242)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1302)
at hudson.scm.SCM.checkout(SCM.java:540)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1215)
at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:645)
at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:85)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:517)
at hudson.model.Run.execute(Run.java:1896)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44)
at hudson.model.ResourceController.execute(ResourceController.java:101)
at hudson.model.Executor.run(Executor.java:442)
Caused by: hudson.plugins.git.GitException: Command "/usr/bin/git fetch --tags --progress git#gitlab.sampledomain.co.kr:xx_xxxxxDetection/xx_xxxxx_detection.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout:
stderr: ssh: connect to host gitlab.sampledomain.co.kr port 22: Connection timed out
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2671)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:2096)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$500(CliGitAPIImpl.java:84)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:618)
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:999)
... 11 more
ERROR: Error fetching remote repo 'origin'
Finished: FAILURE
A GitLab Access Token would work only for HTTPS URL (replacing your GitLab account password)
If you see Connection timed out both for HTTPS URL and SSH URL (git#gitlab.sampledomain.co.kr:... as in your question), it means gitlab.sampledomain.co.kr is not reachable from the Jenkins controller server which tries to contact your Git repository hosting server.
You need to make sure such a server can do a simple curl -v telnet://gitlab.sampledomain.co.kr:22 or curl -v telnet://gitlab.sampledomain.co.kr:443, before being able to run a service like Jenkins.
I have configured a jenkins test job just to clone a git repo and provided the "Repository URL" like below
https://oauth2:glpat-Vn4Vgvft7-2W4P1m4b4Z#gitlab.com/test/batch/test.git
but while running the job I am getting the console error output as below
Cloning the remote Git repository
Cloning repository https://oauth2:glpat-Vn4Vgvft7-2W4P1m4b4Z#gitlab.com/test/batch/test.git
> git init /jenkins/jobs/test/workspace # timeout=10
Fetching upstream changes from https://oauth2#gitlab.com/test/batch/test.git
> git --version # timeout=10
Setting http proxy: 10.203.195.26:8080
> git fetch --tags --progress https://oauth2#gitlab.com/test/batch/test.git
+refs/heads/*:refs/remotes/origin/*
ERROR: Error cloning remote repo 'origin'
The question is, while it is cloning it is showing the URL as https://oauth2:pass#gitlab.com
but why the password is not coming while fetching?
I have created a Jenkins' job to clone, build and deploy the code on tomcat server. But every time I run the job it ends exactly after 10 mins with same error
10:57:59 ERROR: Error cloning remote repo 'origin'
10:57:59 [DeployPublisher][INFO] Build failed, project not deployed
10:57:59 Finished: FAILURE
I know the root cause which is timeout is set to 10 min by default while jenkins job triggers the clone command
10:47:57 > git.exe --version # timeout=10
10:47:58 > git.exe fetch --tags --force --progress -- repository-url +refs/heads/*:refs/remotes/origin/* # timeout=10
10:57:58 ERROR: Timeout after 10 minutes
10:57:59 ERROR: Error cloning remote repo 'origin'
What I couldn't figure out is how to increase this default timeout.
Followings are the list of Git plugins that I have been using
Plugins used
Any help would be appreciated. Thanks in Advance
I have looked into Jenkins documentation and found few Keywords and searching with them, ends up to the answer of my question.
Please refer to this for answer
I want to trigger a pipeline on my OpenShift when an event occurs on bitbucket (push for example). I configured a webhook correctly following the instructions on Openshift documentation pages. Although I had to change my Openshift template of my pipeline which generated some conflicts.
The BuildConfig looks like this:
- apiVersion: "v1"
kind: "BuildConfig"
metadata:
name: "${SERVICE_NAME}-pipeline"
spec:
source:
contextDir: '${APPLICATION_GIT_JENKINSFILE_REPO_CONTEXT_DIR}'
git:
ref: master
uri: '${APPLICATION_GIT_JENKINSFILE_REPO}'
sourceSecret:
name: git-secret
type: Git
strategy:
jenkinsPipelineStrategy:
jenkinsfilePath: Jenkinsfile
triggers:
type: "Bitbucket"
bitbucket:
secretReference:
name: "mysecret"
So, on the 'source' component I reference a git repository where my Jenkinsfile is located. This way I can have many pipelines with only a single Jenkinsfile centralized. Note that this repo is completly different from the location of the api where I'm configuring the webhook.
This approach although fails on an automatic trigger due to the fact that the payload sent to the Openshift has the commit id of the changes of the respective api repository. Openshift (I don't know why) tries to associate that commit with the repo that is present on this template (Jenkinsfile repo).
The logs are the following:
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url http://jenkinsfile-repo.git # timeout=10
Fetching upstream changes from http://jenkinsfile-repo.git
> git --version # timeout=10
using GIT_ASKPASS to set credentials git-secret
> git fetch --tags --progress http://jenkinsfile-repo.git +refs/heads/*:refs/remotes/origin/*
> git rev-parse 79370e4fa88f19c693d85d82fbdbed77620d048b^{commit} # timeout=10
hudson.plugins.git.GitException: Command "git rev-parse 79370e4fa88f19c693d85d82fbdbed77620d048b^{commit}" returned status code 128:
stdout: 79370e4fa88f19c693d85d82fbdbed77620d048b^{commit}
stderr: fatal: ambiguous argument '79370e4fa88f19c693d85d82fbdbed77620d048b^{commit}': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2016)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1984)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1980)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1612)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1624)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.revParse(CliGitAPIImpl.java:809)
at hudson.plugins.git.GitAPI.revParse(GitAPI.java:316)
at hudson.plugins.git.RevisionParameterAction.toRevision(RevisionParameterAction.java:98)
at hudson.plugins.git.GitSCM.determineRevisionToBuild(GitSCM.java:1070)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1187)
at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:113)
at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:144)
at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:67)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:303)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE
There we can see the behavior I tried to explain, the '79370e4fa88f19c693d85d82fbdbed77620d048b' was the commit id on the api repo, which OpenShift is trying to associate with the jenkinsfile repo.
If I could for example ignore the payload the problem wouldn't exist.
Thanks for the help.
I don't think you require put type: git(duplicate) and also try using https:// bitbucket url,
source:
contextDir: '${APPLICATION_GIT_JENKINSFILE_REPO_CONTEXT_DIR}'
git:
ref: master
uri: '${APPLICATION_GIT_JENKINSFILE_REPO}'
sourceSecret:
name: git-secret
type: Git *****remove and try ?
I managed to implement a workaround, although I still don't understand why the behavior I specified previously occurs.
Basically the current solution looks like this:
The openshift template references the GIT repository of the respective API, and that repository has a Jenkinsfile of his own, which is the same for every API. Although, the only thing this Jenkinsfile does is to call a groovy script that is centralized in a separate GIT repository and is declared as a shared library in Jenkins.
This way if we must change something, a stage of a pipeline for example, we only need to change in a single location, which was the objective from the begginning.
I've set up some credentials in Jenkins for bitbucket and double-checked the Credentials settings (e.g. logging in manually) however when I try it in Jenkins it just spins forever giving this output:
> git config remote.origin.url <bitbucket url> # timeout=10
Fetching upstream changes from <bitbucket url>
> git --version # timeout=10
using GIT_ASKPASS to set credentials <bitbucket account email> Bitbucket
> git fetch --tags --progress <bitbucket url> +refs/heads/*:refs/remotes/origin/*
> git fetch --tags --progress <bitbucket url> +refs/heads/*:refs/remotes/origin/*
Note that the URL is fine when public. But when set to Private it simply fails with no output.
Is there anyway to debug this in a bit more detail?
I had a similar issue, with Jenkins on a Windows server. I installed git with credentials manager and whenever it tried to checkout a private repository, it would wait for me to input credentials manually in the server. Disabling the git credential manager fixed it for me.
I already had an option to input credentials in the git plugin so didn't need a separate credentials manager.
This is on MacOSX.
I changed the Jenkins setting on Git path to /usr/local/git as well as unset the credential.helper using git config, both don't work.
Finally, the problem was resolved by creating a default keychain file for jenkins in ~jenkins/Library/Keychains folder. Herewith is the steps...
sudo su jenkins
mkdir ~jenkins/Library/Keychains
cd ~jenkins/Library/Keychains
security create-keychain -p [pwd] ./Login.keychain
security login-keychain -d user -s ./Login.keychain
check default keychain setup properly
security default-keychain
git fetch --tags --progress https://github.com/....git +refs/heads/:refs/remotes/origin/
After that, the github userid/password is stored in jenkins default keychain and it will be used on jenkins build.
I had this problem on OSX. My issue was that Jenkins was using the wrong Git executable (I verified this by disabling the checkout step and adding which git before anything else).
I ran which git in terminal and copy-pasted the path into Manage Jenkins -> Global Tool Configuration -> Git -> Path to Git executable. It worked after that.
I had a similar issue, with Jenkins on a Windows server. I installed git with credentials manager and whenever it tried to checkout a private repository, it would wait for me to input credentials manually in the server. Disabling the git credential manager fixed it for me.
Actually, that should now (Q1 2021) work without having to disable the credential helper with Git 2.30.
"git credential(man)' didn't honor the core.askPass configuration variable (among other things), which has been corrected with Git 2.30 (Q1 2021).
See commit 567ad2c (15 Oct 2020) by Thomas Koutcher (koutcher).
(Merged by Junio C Hamano -- gitster -- in commit e0f6ad2, 02 Nov 2020)
credential: load default config
Signed-off-by: Thomas Koutcher
[jk: added test]
Signed-off-by: Jeff King peff#peff.net
Signed-off-by: Junio C Hamano gitster#pobox.com
Make git credential fill(man) honour the core.askPass variable.
As noted by kymikoloco in the comments, upgrading to the latest (Nov. 2021) Git for Windows 2.34 seems enough to solve the issue.
I had such problem with Jenkins on Windows 10. It always showed failure with permission denied error, code 128 returned by git.exe after ...GIT_ASKPASS for credentials step. Changing credential.helper and env.variable GIT_ASKPASS did not help me at all, the same behaviour.
Then i checked up which git.exe in cmd where git and was suprised, it was SmartGit's internal one.
I changed it to JGit in Jenkins global settings and it works now.
https://i.stack.imgur.com/IBfIi.png
Its works for me!
Goto Dashboard --> configuration
Scroll down to find Git plugin
input Global Config user.name Value: ex: jenkins
input Global Config user.email Value ex: youremail#gmail.com
you need to generate an SSH key from Git and add it to Bitbucket