Bitbucket webhook generating build on the wrong GIT repository - jenkins

I want to trigger a pipeline on my OpenShift when an event occurs on bitbucket (push for example). I configured a webhook correctly following the instructions on Openshift documentation pages. Although I had to change my Openshift template of my pipeline which generated some conflicts.
The BuildConfig looks like this:
- apiVersion: "v1"
kind: "BuildConfig"
metadata:
name: "${SERVICE_NAME}-pipeline"
spec:
source:
contextDir: '${APPLICATION_GIT_JENKINSFILE_REPO_CONTEXT_DIR}'
git:
ref: master
uri: '${APPLICATION_GIT_JENKINSFILE_REPO}'
sourceSecret:
name: git-secret
type: Git
strategy:
jenkinsPipelineStrategy:
jenkinsfilePath: Jenkinsfile
triggers:
type: "Bitbucket"
bitbucket:
secretReference:
name: "mysecret"
So, on the 'source' component I reference a git repository where my Jenkinsfile is located. This way I can have many pipelines with only a single Jenkinsfile centralized. Note that this repo is completly different from the location of the api where I'm configuring the webhook.
This approach although fails on an automatic trigger due to the fact that the payload sent to the Openshift has the commit id of the changes of the respective api repository. Openshift (I don't know why) tries to associate that commit with the repo that is present on this template (Jenkinsfile repo).
The logs are the following:
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url http://jenkinsfile-repo.git # timeout=10
Fetching upstream changes from http://jenkinsfile-repo.git
> git --version # timeout=10
using GIT_ASKPASS to set credentials git-secret
> git fetch --tags --progress http://jenkinsfile-repo.git +refs/heads/*:refs/remotes/origin/*
> git rev-parse 79370e4fa88f19c693d85d82fbdbed77620d048b^{commit} # timeout=10
hudson.plugins.git.GitException: Command "git rev-parse 79370e4fa88f19c693d85d82fbdbed77620d048b^{commit}" returned status code 128:
stdout: 79370e4fa88f19c693d85d82fbdbed77620d048b^{commit}
stderr: fatal: ambiguous argument '79370e4fa88f19c693d85d82fbdbed77620d048b^{commit}': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:2016)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1984)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1980)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1612)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommand(CliGitAPIImpl.java:1624)
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.revParse(CliGitAPIImpl.java:809)
at hudson.plugins.git.GitAPI.revParse(GitAPI.java:316)
at hudson.plugins.git.RevisionParameterAction.toRevision(RevisionParameterAction.java:98)
at hudson.plugins.git.GitSCM.determineRevisionToBuild(GitSCM.java:1070)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1187)
at org.jenkinsci.plugins.workflow.steps.scm.SCMStep.checkout(SCMStep.java:113)
at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:144)
at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:67)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:303)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE
There we can see the behavior I tried to explain, the '79370e4fa88f19c693d85d82fbdbed77620d048b' was the commit id on the api repo, which OpenShift is trying to associate with the jenkinsfile repo.
If I could for example ignore the payload the problem wouldn't exist.
Thanks for the help.

I don't think you require put type: git(duplicate) and also try using https:// bitbucket url,
source:
contextDir: '${APPLICATION_GIT_JENKINSFILE_REPO_CONTEXT_DIR}'
git:
ref: master
uri: '${APPLICATION_GIT_JENKINSFILE_REPO}'
sourceSecret:
name: git-secret
type: Git *****remove and try ?

I managed to implement a workaround, although I still don't understand why the behavior I specified previously occurs.
Basically the current solution looks like this:
The openshift template references the GIT repository of the respective API, and that repository has a Jenkinsfile of his own, which is the same for every API. Although, the only thing this Jenkinsfile does is to call a groovy script that is centralized in a separate GIT repository and is declared as a shared library in Jenkins.
This way if we must change something, a stage of a pipeline for example, we only need to change in a single location, which was the objective from the begginning.

Related

How to poll scm using Jenkins for release tags

I need a help to poll scm using Jenkins for release tags. Webhook trigger is not working as the Jenkins resides behind the firewall.
Jenkins build has to be triggered if there is a release/release tag is committed/pushed under https://github.com/<user>/<repo>/releases
I have configured the below under Source Code Management tab in Jenkins.
Git ->
Under Repositories, given Repository URL and Credentials. Also under "Advanced" for "Refspec" configured the values as:
+refs/tags/*:refs/remotes/origin/tags/*
Also for Branches to build -> /tags/
Configured SCM poll to run for every 5 minutes. Under "Git Polling Log", its polling every 5 minutes and bringing the release details based on the tag. But the Jenkins job is not triggered.Can anyone please help me to address this issue?
From Git Polling Log:
Started on 13 Oct, 2020 4:42:00 PM
Polling SCM changes on master
Using strategy: Default
[poll] Last Built Revision: Revision 2f3a23888bdce0b4ac63044c00f71adedc0aa175 (origin/tags/fb-bin1, origin/tags/15thRelease, origin/tags/9thRelease, origin/tags/17thRelease, origin/tags/14thRelease, origin/tags/11thRelease, origin/tags/20thRelease, origin/tags/8thRelease, origin/tags/13thRelease, origin/tags/10thRelease, origin/tags/21stRelease, origin/tags/19thRelease, origin/tags/12thRelease, origin/tags/16thRelease, refs/tags/fb-bin1, refs/tags/15thRelease, refs/tags/13thRelease, refs/tags/8thRelease, refs/tags/19thRelease, refs/tags/16thRelease, refs/tags/12thRelease, refs/tags/17thRelease, refs/tags/9thRelease, refs/tags/21stRelease, refs/tags/11thRelease, refs/tags/10thRelease, refs/tags/20thRelease, refs/tags/14thRelease)
using credential 7cbf657e-010a-40fd-ad76-0813e675c188
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repositories
> git config remote.origin.url https://github.com/<user>/repo.git # timeout=10
Fetching upstream changes from https://github.com/<user>/repo.git
> git --version # timeout=10
using GIT_SSH to set credentials
> git fetch --tags --progress https://github.com/<user>/repo.git +refs/tags/*:refs/remotes/origin/tags/* # timeout=10
Polling for changes in
Seen branch in repository origin/bin
Seen branch in repository origin/master
Seen branch in repository origin/tags/10thRelease
Seen branch in repository origin/tags/11thRelease
Seen branch in repository origin/tags/12thRelease
Seen branch in repository origin/tags/13thRelease
Seen branch in repository origin/tags/14thRelease
Seen branch in repository origin/tags/15thRelease
Seen branch in repository origin/tags/16thRelease
Seen branch in repository origin/tags/17thRelease
Seen branch in repository origin/tags/19thRelease
Seen branch in repository origin/tags/20thRelease
Seen branch in repository origin/tags/21stRelease
Seen branch in repository origin/tags/22ndRelease
Seen branch in repository origin/tags/23rdRelease
Seen branch in repository origin/tags/24thRelease
Seen branch in repository origin/tags/2ndLatest
Seen branch in repository origin/tags/3rdRelease
Seen branch in repository origin/tags/4thRelease
Seen branch in repository origin/tags/5thRelease
Seen branch in repository origin/tags/6thRelease
Seen branch in repository origin/tags/7thRelease
Seen branch in repository origin/tags/8thRelease
Seen branch in repository origin/tags/9thRelease
Seen branch in repository origin/tags/Latest
Seen branch in repository origin/tags/fb-bin1
Seen 26 remote branches
> git show-ref --tags -d # timeout=10
Done. Took 1.6 sec
No changes
Thanks in advance!

Jenkins #Library resolution failing due to git.exe not existing, but where?

I have this command at the top of my pipeline file, and my build failed immediately in Jenkins while trying to resolve it:
#Library('MyLibrary') _
pipeline{
...
I also have this library setup in my global jenkins config. Triple checked it.
The Jenkins log is below... the internet says this happens when git.exe is misconfigured or not found. We have 100% verified git is installed on all the slaves. However, I suspect that this failing git clone is running somewhere else since the pipeline has not officially started yet (we don't see RUNNING ON: blah in the log).
So... Does it run this on the master? We have master set to 0 executors, and git is not installed there.
Is this problem solved as easily as installing git on the master? Or is there some other magical place where these #Library includes get cloned?
Here is the Jenkins log output, with names changed to protect the innocent:
Started by user Me
15:28:23 Connecting to my github url
Obtained Jenkinsfile from blah
Running in Durability level: MAX_SURVIVABILITY
Loading library MyLibrary#master
Examining <my github repo>
Attempting to resolve master as a branch
Resolved master as branch master at revision blah
No credentials specified
Cloning the remote Git repository
Cloning with configured refspecs honoured and without tags
Cloning repository <my repo url>.git
> git init /home/afolder/jenkins/workspace/<git repo name>#libs/MyLibrary # timeout=10
ERROR: Error cloning remote repo 'origin'
hudson.plugins.git.GitException: Could not init /home/afolder/jenkins/workspace/<git repo name>#libs/MyLibrary
at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$5.execute(CliGitAPIImpl.java:813)
...
Caused by: java.io.IOException: Cannot run program "git" (in directory
"/home/afolder/jenkins/workspace/<git repo name>#libs/MyLibrary"):
error=2, No such file or directory
Yes, by default it will run on the Jenkins master.
Please follow the link for more info.
You can read library file and give the file to writeFile pipeline step
writeFile(file:"foo.groovy", text: libraryResource("bar.groovy"))
"groovy foo.groovy"

Getting Issues while VSTS Integration with Jenkins using Service Hooks

[As part of CI,I want to integrate VSTS with Jenkins using Service Hooks,Here are the Steps i perform.
1.After selecting Jenkins Service option, I am entering the Jenkins details as per below image.
2.Jenkins base URL, User name, User API Token are entered but Build Textbox is not allowing me enter any value.
Build Error Log using Build and Release:-
2017-06-23T12:46:02.5702001Z ##[section]Starting: Build
2017-06-23T12:46:02.5721999Z Current agent version: '2.119.1'
2017-06-23T12:46:03.0032004Z ##[section]Starting: Initialize Job
2017-06-23T12:46:03.0122005Z Prepare build directory.
2017-06-23T12:46:03.0431992Z Set build variables.
2017-06-23T12:46:03.0471996Z Download all required tasks.
2017-06-23T12:46:03.0601994Z Downloading task: JenkinsQueueJob
2017-06-23T12:46:03.7031992Z Downloading task: JenkinsDownloadArtifacts
2017-06-23T12:46:05.7861997Z Downloading task: PublishTestResults
2017-06-23T12:46:05.9391998Z Downloading task: PublishBuildArtifacts
2017-06-23T12:46:06.1041998Z ##[section]Finishing: Initialize Job
2017-06-23T12:46:06.1351993Z ##[section]Starting: Get Sources
2017-06-23T12:46:06.1711981Z Syncing repository: MyFirstProject (TfsGit)
2017-06-23T12:46:06.1761997Z Prepending Path environment variable with directory containing 'git.exe'.
2017-06-23T12:46:06.1842000Z ##[command]git version
2017-06-23T12:46:06.4442001Z git version 2.12.0.windows.1
2017-06-23T12:46:06.4561989Z ##[command]git init "d:\a\1\s"
2017-06-23T12:46:06.5122094Z Initialized empty Git repository in d:/a/1/s/.git/
2017-06-23T12:46:06.5151998Z ##[command]git remote add origin https://ramandogra.visualstudio.com/_git/MyFirstProject
2017-06-23T12:46:06.5371993Z ##[command]git config gc.auto 0
2017-06-23T12:46:06.5571999Z ##[command]git config --get-all http.https://ramandogra.visualstudio.com/_git/MyFirstProject.extraheader
2017-06-23T12:46:06.5761994Z ##[command]git config --get-all http.proxy
2017-06-23T12:46:06.6061997Z ##[command]git -c http.extraheader="AUTHORIZATION: bearer ********" fetch --tags --prune --progress origin
2017-06-23T12:46:07.4221999Z ##[command]git checkout --progress --force refs/remotes/origin/master
2017-06-23T12:46:07.4431999Z error: pathspec 'refs/remotes/origin/master' did not match any file(s) known to git.
2017-06-23T12:46:07.6841986Z ##[error]Git checkout failed with exit code: 1
2017-06-23T12:46:07.6881995Z ##[section]Finishing: Get Sources
2017-06-23T12:46:07.6901995Z ##[section]Starting: Post Job Cleanup
2017-06-23T12:46:07.6951999Z Cleaning any cached credential from repository: MyFirstProject (Git)
2017-06-23T12:46:07.6992000Z ##[command]git remote set-url origin https://ramandogra.visualstudio.com/_git/MyFirstProject
2017-06-23T12:46:07.7211998Z ##[command]git remote set-url --push origin https://ramandogra.visualstudio.com/_git/MyFirstProject
2017-06-23T12:46:07.7401991Z ##[section]Finishing: Post Job Cleanup
2017-06-23T12:46:07.7461997Z ##[section]Finishing: Build
PS: I am first time user of VSTS.
This is mainly caused your jenkins base URL is a local address, such as http://computername:8080 (or http://ip:8080). But the URL can’t be reached by others.
Do below things to check if local URL can be used:
In http://localhost:8080/configure, use the Jenkins URL as Jenkins base URL for service hook.
Turn off firewall for your local machine.

Jenkins and git with custom port

I am trying to link a gitlab instance with a Jenkins instance. Due to being run in a DC/OS environment I cannot use the standard SSH port 22, so I am running gitlab on SSH port 16122 instead.
Gitlab displays SSH clone URLs in the format [git#git.company.de:16122]:user/test.git, which a normal git clone command understands. Also, Jenkins accepts this URI as SCM URI in the configuration screen (as evidenced by the error messages "invalid credentials" vanishing once I set the SSH key as credential that is configured as deploy-key in Gitlab.
The problem arises during actually building the job: jenkins for some unknown reason decides to double-urlencode the [ at the front of the URI and thus the build breaks:
Cloning repository %255bgit#git.company.de:16122]:user/test.git
> git init /var/jenkins_home/workspace/test # timeout=10
Fetching upstream changes from %25255bgit#git.company.de:16122]:user/test.git
> git --version # timeout=10
> git fetch --tags --progress %25255bgit#git.company.de:16122]:user/test.git +refs/heads/*:refs/remotes/origin/*
ERROR: Error cloning remote repo 'origin'
hudson.plugins.git.GitException: Command "git fetch --tags --progress %25255bgit#git.company.de:16122]:user/test.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
When I look at the raw XML of the job using xxd on the shell, there are no "invisible" ASCII chars or similar in front of the repo URL.
What causes this, and is there a way to fix this without having to manually rewrite every URL to ssh://git#git.company.de:16122/user/test.git?

Where to put ${library.pipeline-utils.version} when using Git SCM to setup Global Pipeline Library

I am trying to setup a global pipeline library that will be explicitly loaded using #Library() per these instruction. I have this working fine with respect to loading the default library version from master branch.
What I am unable to do is figure out how to configure things so that it can load different branches upon request using the Library('lib-name#branch-name') syntax.
The instructions say:
"When using Legacy SCM, you will need to include ${library.pipeline-utils.version} in the SCM configuration somewhere."
I have tried using ${library.pipeline-utils.version} as the value for the Branch Specifier which is the only logical place I can think to put it. But that just break things. Attempts to load the library fail with:
Fetching upstream changes from ssh://git#stash:7999/jen/pipeline-utils.git
> git --version # timeout=10
> git fetch --tags --progress ssh://git#stash:7999/jen/pipeline-utils.git +refs/heads/*:refs/remotes/origin/*
> git rev-parse refs/remotes/origin/${library.pipeline-utils.version}^{commit} # timeout=10
> git rev-parse refs/remotes/origin/origin/${library.pipeline-utils.version}^{commit} # timeout=10
> git rev-parse origin/${library.pipeline-utils.version}^{commit} # timeout=10
ERROR: Couldn't find any revision to build. Verify the repository and branch configuration for this job.
This makes me think I put the variable in the right place but Jenkins is not interpolating it. (Bug?)
This is using Jenkins 2.7.1 and Pipeline version 2.4
This is how I set it, using version variable ${library.CpsGlobal.version} and Branches to build / Branch Specifier :
ps. Please remove "-" character from library name as it's not supported by Groovy.
Work for me:
Library name: testlib
Default version: branchname
Branches to build: ${library.testlib.version}
There is finally now a 'modern scm' implementation for Git that works as we are using that.

Resources