I have a project that uses Travis-CI to build and rsync a static website. I use the following to ensure that this only occurs in master.
branches:
only:
- master
However, when someone makes a pull request on the repository, Travis-CI prospectively merges that branch into master and does the build and rsync, meaning that anyone could replace the contents of my website with anything by submitting a pull request.
Is there a way to prevent Travis-CI from attempting to build pull requests?
You can find out if Travis is checking a pull request by checking the environment variable TRAVIS_PULL_REQUEST. It contains:
The pull request number if the current job is a pull request, "false" if it's not a pull request.
See also the docs.
You can change your command to check this and only build on non-pull requests with
if [[ $TRAVIS_PULL_REQUEST == 'false' ]]; then your-command; fi
To avoid the scenario described by #ruslo, you can (and should!) use encrypted environment variables for building the rsync connection. These are not available with pull requests (at least with those that come from a fork), so that everything's safe here:
Please note that secure env variables are not available for pull requests from forks. This is done due to the security risk of exposing such information in submitted code. Everyone can submit a pull request and if an unencrypted variable is available there, it could be easily displayed.
(The reasons stated in the docs are different, but the mechanism would work here as well.)
Can't you disable PULL request builds? At least, at the time of this writing, I see that as a choice on a per repo setting:
Related
Brief background. My team uses Jenkins for CI but (among other things) we pass our output to Azure Devops Release pipelines, to be used for downstream testing. We currently have a working system for this but it uses the Jenkins Team Foundation Server/TFS plugin (to trigger the Azure Devops release) and specific features of azure-pipelines-tasks (to pull the artifacts from Jenkins). The issue is that the former is deliberately disabled in recent versions of Jenkins (for licencing and security issues) and the latter similarly has a bug talking to Jenkins. Basically we are stuck on Jenkins 2.263.1 with no sign that this will be rectified. It would seem prudent to use a completely different approach.
Intuitively we need to be able to programmatically trigger the release pipeline. Additionally I need to transfer the artifact - whether Jenkins would push or ADevops would pull, I don't know - guess the latter as closer to current. Whatever, I am wondering if somebody already has instructions on how to do this to avoid us re-inventing the wheel.
Install the TEE-CLC and invoke the CLI commands from a shell?
That seems to be MS suggestion for their Azure plugins which they have formally announced abandoning.
Of course, now they have quietly announced they won't maintain that either, so I guess the implied message is "go use github?"
After going through the Azure REST API, chiefly here, and some experimentation, I can report what I got to work.
Because the original job was Traditional/Freestyle (the plugin did not support pipelines), I kept the job but replaced the plugin step with a bash script, run at after the "archive" step I had in there anyway. The key statement in this script is:
curl -X POST -f \
-H "Authorization: Basic ${SERVICE_TOKEN_B64}" \
-H "Content-Type: application/json" \
-d "{\"definitionId\":${RELEASE_DEFINITION_ID},\"description\":\"Triggered by ${BUILD_TAG}\",\"isDraft\":false,\"artifacts\":[{\"alias\":\"_${JOB_BASE_NAME}\",\"instanceReference\":{\"id\":\"${BUILD_NUMBER}\",\"name\":\"${BUILD_DISPLAY_NAME}\"}}]" \
-o output.json \
"${AZURE_URL}/${ORGANISATION}/${PROJECT_NAME}/_apis/release/releases?api-version=5.0"
The various macros are mainly set in an "Inject Environment Variables" above.
AZURE_URL here is "https://vsrm.dev.azure.com", but could be something else I guess if you had your own Enterprise instance of Azure. ORGANISATION and PROJECT_NAME should be obvious.
Less obvious is RELEASE_DEFINITION_ID. You can go through and enquire this from the REST API, given then name of the release (pipeline) you are dealing with - essentially, release definition in the API equates to release pipeline in the Web UI. However, the simplest approach is to navigate in the Web UI to the release pipeline you are interested in (not an individual release job/instance). If you look at the URL for this, you will see definitionId=XXX as a parameter. You want the XXX value.
SERVICE_TOKEN_B64 is a suitable PAT manipulated into a different format. What you actually want is to take the output of:
printf "%s"":$PAT" | base64
and save that as secret text in the Jenkins credentials system. Note that the plugin can do this on the fly (using the username and PAT from the credentials system), but if you do that in a bash script it will (typically) show this modified token in the log - which sort of defeats the object.
This leaves the artifacts. It took me quite a while to discover i) you need the artifacts and ii) the payload is quite important. We have what I assume is a fairly standard setup where there is a "Jenkins" Artifact on the Azure release pipeline. We have "Specify at time of release creation" for the Default Version - I didn't set the Azure release pipeline up, but I suspect all fairly standard too. With this, the artifact part is crucial and without it (or setting to an empty array) I was getting 400 errors which baffled me - I raised a parallel question here. Looking back the scenario is fairly straightforward. The artifacts payload itself is:
[{"alias":"_${JOB_BASE_NAME}","instanceReference":{"id":"${BUILD_NUMBER}","name":"${BUILD_DISPLAY_NAME}"}}]
The alias value is that of the Artifact name in the Azure release pipeline - not sure if just prefixing the job with "_" is standard or just the convention used by the guy who set the pipeline up. The other values could I think be treated as magic, but my guess is that the "id" is the actual value used of the Jenkins job instance from where the artifact itself is pulled (the Jenkins job name comes from the Azure pipeline setting) and the "name" is used for displaying/logging. As said, this works for our purposes but your usage may differ.
I have a repository that I can create a release for. I have jenkins setup and since the jenkins is hosted inside a firewall that restricts any communication from outside the network, github-webhook doesnt work. Also getting the reverse proxy to work is a bit of a challenge for me. I understand that the github webhook sends out a json payload and I can qualify it based on release. But as I previously mentioned, this won't work because jenkins and github cannot talk with each other.
Therefore, I tried this solution; Filtering the branches or tags that the jenkins will build on. The following are the things I tried and they all didnt work. Everytime I run a build, jenkins just builds it.
I also tried the below mentioned regex,
:refs\/tags\/(\d+\.\d+\.\d+)
I also tried [0-9] instead of d. It build it every single time.
Am I missing something ? Or is that how jenkins work ? Even though we qualify the builds to run only on certain tags or releases, if we click on build now, it just runs it every single time ?
My requirement is very simple. I want jenkins build to run only on the release I created even if the release is 'n' commits behind the master. How can I achieve this ?
I'm setting up some tests to be run via Travis CI. I have some secret encrypted environment variables containing AWS credentials. According to Travis's documentation, encrypted environment variables aren't available in untrusted builds such as GitHub Pull Requests. I'd like to make it so that if/when an un-trusted build is run, the testing scripts do something different.
The Environment Variables doc page says Travis provides TRAVIS_PULL_REQUEST in order to say when a build is triggered by a pull request. However, it's not clear to me whether there are other circumstances in which Travis may withhold encrypted environment variables. As it earlier refers to (emphasis mine):
Encrypted variables are not available to untrusted builds such as pull requests coming from another repository.
This suggests to me that there are other builds that Travis would consider untrusted, besides simply Pull Requests.
TRAVIS_SECURE_ENV_VARS seems like a better fit, but it seems to be set to true when secure variables are defined, regardless of whether they are used or not. Will this resolve to false if it's done by an untrusted build?
The obvious solution is to of course just check to see if the secret environment variables are defined at all when the code runs. But that's not ideal for my use-case, though it's probably what I'll end up doing if I can't find anything.
I try to configure Jenkins. I want a simple behavior: trigger a build on new pull request.
I cannot understand what I missed...
Jenkins version: 2.89.2
At https://ci.mysite.fr/configure :
Still no build triggered:
At https://ci.mysite.fr/job/test-back/configure :
On Github, Webhook is sent and well received by Jenkins:
Nginx Log says the same:
Help please!
Some things to check when debugging this sort of thing:
Check your Jenkins logs to see whether or not Jenkins is receiving the hook and deciding not to take action for some reason.
Check Jenkins security by clicking Manage Jenkins -> Configure Global Security. Open things up as much as you're comfortable doing and see if that changes anything.
Ensure that you're pushing changes to the master branch. For simplification, consider using ** as your branch specifier while you're getting this to work.
Ensure Git is properly configured on your Jenkins host by clicking Manage Jenkins -> Global Tool Configuration
Make sure the user whose credentials you provided can manage hooks and pull from the repo you're interested in.
Run the job manually in Jenkins, ensure that it works.
After you run the job, it should show up as an option in Protected Branches/Required Statuses. In your repo, click on Settings->Branches, select your branch in the Branches section, click Require Status Check to Pass before merging option, and your job should show up in the list which appears.
Webhooks are arguably the most difficult Jenkins feature to test without prior experience, because of gotchas like these (probably their list is incomplete):
New git commit / git push must be made for each pipeline build (repeating a previous one won't trigger a new build even if webhooks are already set up correctly - see below).
First build made after setting up webhook correctly must be manual (no bootstrap from the webhook itself is possible).
First build made after setting webhook correctly must succeed completely for the changes to take effect and for webhooks to start working. This will also cause Jenkins to miss all incoming requests made during the first build of a newly created pipeline.
More info
Please be warned that it is not possible to trigger a build using the same build conditions again (at least using a webhook). Therefore you might have a correct webhook setup already, but not find out that it works unless you create a new git commit and push it to the remote repo on Github. If your try to repeat some old push over and over again, by simply pressing the "Redeliver" button in the Recent deliveries section on Github's Webhooks / Manage webhook page, Jenkins will never move beyond the "poke" repo stage, as it requires SCM changes to be detected in order to trigger a new build:
Received PushEvent for https://github.com/mirekphd/<REPO_NAME> from <GITHUB_IP> ⇒ <JENKINS_URL>/github-webhook/
Apr 16, 2021 9:42:12 PM INFO org.jenkinsci.plugins.github.webhook.subscriber.DefaultPushGHEventSubscriber$1 run
Poked <REPO_NAME>
Apr 16, 2021 9:42:13 PM INFO com.cloudbees.jenkins.GitHubPushTrigger$1 run
SCM changes detected in <REPO_NAME>. Triggering #236
For further info on points 2) and 3): see original source.
I'd like to configure bitbutcket to trigger a jenkins build.
I've spent some time researching this and all the answers are from a few years ago, and have not found any guides because things seem to have changed since.
What I'm trying to do:
A bitbucket push to a particular branch triggers a build.
What I've got:
Bitbucket web hooks which fires HTTP request to Jenkins on a push to any branch. I've also installed the Bitbucket plugin on Jenkins which adds a check box in the job config Build when a change is pushed to BitBucket. This checkbox doesnt seem to work (maybe I set it up wrong? minimal docs for this), despite me pushing to the configured branch in the SCM section.
Problem 1: Bitbucket does not fire a GET, but another request which causes a 403. I tested with postman, and it works with a GET, but not a POST.
Problem 2: This HTTP build request is fired on pushes to any branch. While the build is still restricted to a particular branch, it seems unnecessary to be rebuilding all the time.
How do i address these issues? Bitbucket does not seem to be very flexible in customizing this. The Jenkins plugin for bitbucket has a lot of 'bad' reviews. How are developers currently doing this?
SPECIFIC solution for Jenkins CI server--Webhook to Jenkins for Bitbucket plugin has been commercialized in latest version of Bit-Bucket and the current price is around $4800 which was earlier a free offering, because of this, guys who want to save their bucks, can go to the alternative solution by using webhooks feature of bit-bucket:-
Steps to create a webhook:-
BitBucket Side
1) Go to your bitbucket repo, click on Repository Setting, under WORKFLOW got for WEBHOOKS option and create a webhook.
a) creation of webhook:- URL https://JenkinsserverURL/git/notifyCommit?url=https://bitbucket.repository-link/repository.git
b) In the name tab, give any name of your choice
c) click on TEST CONNECTION before saving it. Make sure you get http status 200
d) View details your logs, check your request and response is correct.
Things to take care of from
Jenkins Side:-
1) Make sure repository mentioned in bitbucket webhook is used in Jenkins job.
2) In SCM option, activate/select Poll SCM option, don't mention anything in the schedule, leave it blank.
3) configure rest job,
Whenever your git repo observes any change an automatic build will get triggered in Jenkins. By default push trigger is activated and if you want to activate other action, please select those events while creating webhook.
***to specify the branch in repository webhook:-
http://yourserver/git/notifyCommit?url=<URL of the Git repository>[&branches=branch1[,branch2]*][&sha1=<commit ID>]
Cheers,
Is your Jenkins URL accessible from your bitbucket server? If yes that it should be fairly simple to do it. You add the webhook in your repository as http://<url-of-jenkins>/git/notifyCommit?url=<url-of-repository>. When jenkins receives this POST, it automatically triggers builds on those jobs that use this git repo with that URL you give in webhook.
But you also need to make sure your Build Schedule is set to empty for those jobs. otherwise it wont get triggered. You can specify a branch in webhook URL too
See the Push Notification from repository here
https://wiki.jenkins.io/display/JENKINS/Git+Plugin
For anyone here after July 2022, here are the simple steps I followed to make it work.
Create a live Jenkins URL
First, create a tunnel from a live URL to your local Jenkins URL using ngrok because using locahost:8080 directly as your webhook URL on bitbucket will simply not work as bitbucket does not recognize your local computer.
ps: ngrok claims to be the fastest way to put anything on the internet and I agree,
you can use it beyond Jenkins once you know the trick,
such as quickly handling out your localhost react app for testing by your friends
out of your local network
To do this is simple. For Linux:
Install ngrok snap install ngrok
Add authtoken ngrok config add-authtoken <token>
Don't have an auth token, sign up
Start a tunnel on your Jenkins port eg ngrok http 8080
To know more and for other OS, check ngrok download page
You will then get a response like
ngrok (Ctrl+C to quit)
Hello World! https://ngrok.com/next-generation
Session Status online
Account <your email>#<domain>.com (Plan: <plan type>)
Version 3.0.6
Region Europe (eu)
Latency 162ms
Web Interface <web interface url>
Forwarding https://<your-assigned-host>.ngrok.io -> http://localhost:8080
Basically, the web interface URL on click gives you a web interface to inspect all the requests being tunnelled from your ngrok live URL to your local host.
Forwarding URL is basically a proxy to your localhost, so when you want to configure webhook, instead of using locahost:8080, you replace it with ngrok URL eg https://syue-162-34-12-01.eu.ngrok.io and all requests get tunnelled to localhost:8080
Hook up the URL on bitbucket cloud
Secondly, configure your Bitbucket repository with a Webhook, using URL JENKINS_URL/bitbucket-hook/ (no need for credentials but do remember the trailing slash) eg https://syue-162-34-12-01.eu.ngrok.io/bitbucket-hook/
If you are using bitbucket server and not cloud or you want to know more, the bitbucket plugin documentation for Jenkins is pretty straightforward and easily understandable, see bitbucket plugin
then you can inspect all your webhook requests on the web interface URL or via your terminal as well as check your build logs on Jenkins via your localhost port or ngrok live url.
Disclaimer: I have not figured out how to enable build only when a specific branch change but you can configure jenkins to only build a specific branch or any branch created as your need may demand, check Source Code Management and Build Triggers