Problem Statement
I am using Jenkins for CI/CD automation. I installed the GitHub Pull Request Builder through Jenkin's Plugin Manager.
However I cannot figure out how to do this with a Jenkins Pipeline
Attempts/Documentation
I have skimmed through this exhaustive list of pipeline steps and declarative documentation but cannot find any documentation on how to accomplish this?
Installing Plugin via Jenkinsfile
Firstly, I would question why exactly you want to do this. This is a fairly odd requirement that I have not seen in my 10 years of Jenkins & Hudson experience. With that said, anything that can be done in the Jenkins web interface, and a billion other things that cannot, can be done in either a Pipeline or in the Script Console because these two constructs have access to the entire SDK.
node {
// 'ghprb' is this plugins short name value
GITHUB_PULL_REQUEST_BUILDER = 'ghprb'
for (plugin in [GITHUB_PULL_REQUEST_BUILDER]) {
e = Hudson.instance.updateCenter.getPlugin(plugin).deploy().get().getError()
if (e != null)
println e.message
}
}
Security
Anything you see on the Jenkins GUI can be done in the script console or a pipeline via the Jenkins SDK. Jenkins has implemented a Sandbox to prevent nefarious actors from executing malicious scripts. Your requirement must use the Jenkins/Hudson SDK and therefore you have two options:
Disable the Sandbox for this pipeline
Authorize the scripts and/or methods being used
If you try and run the aforementioned script without solving these security constraints the following exception will be thrown:
org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use method jenkins.model.Jenkins getUpdateCenter
Sandbox
To disable the Sandbox simply uncheck the "Use Groovy Sandbox" checkbox. This is the easiest solution.
Script Approval
If your administrator does not authorize you to outright disable the Sandbox for that job then you can approve the method signatures used in your script or approve the script itself.
The following message signatures would need to be approved for this script
staticMethod hudson.model.Hudson getInstance
method jenkins.model.Jenkins getUpdateCenter
method hudson.model.UpdateSite$Plugin deploy
method java.util.concurrent.Future get
method hudson.model.UpdateCenter$UpdateCenterJob getError
Additional Information
Most likely the plugin will not be available until a restart has occurred. You can automate a restart a variety of ways. You can hit the following endpoints
http://<jenkins.server>/restart
http://<jenkins.server>/safeRestart
You could invoke the jenkins-cli.jar tool. If you're feeling exceptionally risky you could even do something like Jenkins.instance.restart()
Note: don't use the SDK to restart unless you really know what you're doing
Related
I am running Jenkins version 2.85 on Kubernetes as pod(Affinity set to one workernode). I am creating Jobs using Salt Jenkins module by passing XML to this module.
I am using Jenkins Global Library for preforming job execution.
My Job config looks like this
I am calling GobalLibrary with my parameters like repoURL, componet etc..,
Things goes well for weeks and now I landed to a weird situation where my job configurations(config.xml) gets updated/revert automatically.
Intermittently my "Build with parameter" options disappears and I can see only "Build now" in Jenkins GUI. Initially I thought someone is doing this, so to track the config changes I installed Job config history plugin in Jenkins and what I find is strange. Someone with "SYSTEM" username is making/reverting changes.
This is how it looks
and what I find is SYSTEM user revert only JOB config changes, not the PIPELINE.
I am not sure what's going wrong behind the scenes and how to stop or fix this. This is my Production instance so I am more worried.
I can see a SYSTEM user in my Jenkins
but I can not delete that user
Few relevant Question I find for this but with no answers
Configuration of Jobs getting updated by System user on Jenkins
Jenkins SYSTEM user removes custom workspace configuration
I am not sure if this Jenkins Bug or some plugin is playing with my soul.
Need help! :(
Okay I find the answer to this problem.
I have used properties in my Jekins Global Library something like this
// Disable concurrent builds
//properties([disableConcurrentBuilds()])
which overrides my external job configuration(done via salt).
Hint I get from this blog:
https://st-g.de/2016/12/parametrized-jenkins-pipelines
I also had this problem. For me it was solved when I changed the Build triggers -> Build Periodically settings from 'H 23 * * *' to '00 23 * * *'. (As I want my build to execute every night at 23:00.) Where H lets Jenkins decide when to run the job somewhere between 23:00 and 23:59 to spread load evenly. It seems Jenkins sometimes decided that it would be best to run my job on a different server and changed the parameters automatically.
In my case the issue was that the Jenkinsfile was removing the parameters I added to the pipeline from Jenkins console. Adding the same parameters in the JenkinSfile (stage -> script -> properties -> parameters) solved the issue.
In a nutshell, make sure that your Pipeline script is using the same configuration that your pipeline uses.
Jenkins documentation on parameters: https://www.jenkins.io/doc/book/pipeline/syntax/#parameters
I am trying to write the test cases to validate the Jenkinsfile, But the load script function not working expecting the extension to be provided and throwing ResourceException exception loadScript("Jenkinsfile")
Is their better way to test the Jenkinsfile
The problem is that there are not enough tools for the development of pipelines. Pipelines is DSL and it imposes a restrictions.
There is an interesting approach to using flags. For example, test which defines outside pipeline(in job). If test=true, a pipeline change some "production" logic to "test" - select another agent, load artifacts into another repository, run another command and so on.
But recently appeared Pipeline Unit Testing Framework. It allows you to unit test Pipelines and Shared Libraries before running them in full. It provides a mock execution environment where real Pipeline steps are replaced with mock objects that you can use to check for expected behavior.
Useful links:
Jenkins World 2017: JenkinsPipelineUnit: Test your Continuous Delivery Pipeline
Pipeline Development Tools
You can validate your Declarative Pipeline locally thanks to Jenkins built-in features.This can be done using a Jenkins CLI command or by making an HTTP POST request with appropriate parameters.
The command is the following:
curl -s -X POST -F "jenkinsfile=<YourJenkinsfile" \
https://user:password#jenkins.example.com/pipeline-model-converter/validate
For a practical example follow this guide:
https://pillsfromtheweb.blogspot.com/2020/10/validate-jenkinsfile.html
How can I add/edit new code to my Jenkins instance that would be accesible in a DSL script? Context follows
I've inherited a Jenkins instance. Part of this inheritance includes spending the night in a haunted house writing some new automation in groovy via the Jobs DSL plugin. Since I'm fearful of ruining our jenkins instance, my first step is setting up a local development instance.
I'm having trouble running one of our existing DSL Scripts on my local development instance -- my builds on the local server fail with the following in the Jenkins error console.
Processing DSL script jobs.groovy
ERROR: startup failed:
jobs.groovy: 1: unable to resolve class thecompanysname.jenkins.extensions
The script in question starts off like this.
import thecompanysname.jenkins.extensions
use(extensions) {
def org = 'project-name'
def project = 'test-jenkins-repo'
def _email = 'foo#example.com'
So, as near I can tell, it seems like a predecesor has written some custom Groovy code that they're importing
import thecompanysname.jenkins.extensions
What's not clear to me is
Where this code lives
How I can find it in our real Jenkins instance
How I can add to to my local instance
Specific answers are welcome, as our here's how you can learn to fish answers.
While there may be other ways to accomplish this, after a bit of poking around I discovered
The Jenkins instance I've installed has an older version of the Jobs DSL plugin installed.
This version of the Jobs DSL plugin allowed you to set an additional classpath in your Process DSL Builds job section that pointed to additional jar files.
These jar files can give you access to additional classes in your groovy scripts (i.e. thecompanysname.jenkins.extensions)
Unfortunately, more recent versions of the Jobs DSL plugin have removed this option, and it's not clear if it's possible to add it back. That, however, is another question.
Configure Global Security -> uncheck "Enable script security for Job DSL
scripts".
works for me
I have GitLab Community Edition 8.15.2 successfully trigger pipeline projects in Jenkins 2.32.1 using a webhook. I want the gitlab push to trigger a build with parameters but the parameter value is null when it comes through so the build fails.
The gitlab webhook looks like:
http://jenkins.server:8080/project/project-a/buildWithParameters?MYPARAM=foo
In my pipeline project I echo the parameter value out with
echo "MYPARAM: ${MYPARAM}"
and it's not set to anything. Any ideas on where I've gone wrong?
UPDATE
The actual code I'm using in the pipeline is:
node {
try {
echo "VM_HOST: ${VM_HOST}"
echo "VM_NAME: ${VM_NAME}"
stage('checkout') {
deleteDir()
git 'http://git-server/project/automated-build.git'
}
stage('build') {
bat 'powershell -nologo -file Remove-MyVM.ps1 -VMHostName %VM_HOST% -VMName "%VM_NAME%" -Verbose'
}
...
}
}
The parameter VM_HOST has a default value but VM_NAME doesn't. In my Console output in Jenkins I can see:
[Pipeline] echo
VM_HOST: HyperVHost
[Pipeline] echo
VM_NAME:
I have been struggling with this for weeks. I had it working once, but I couldn't get it to work again, untill today. And the solution was mindblowingly obvious ofcourse...
Automatically for each pipeline job I ticked the following box:
Build when a change is pushed to GitLab. GitLab CI Service URL:
http://jenkins.dev:8080/project/MyProject
Then from GitLab I used the webhook to trigger the above.
Like you I tried to add /buildWithParameters and tried many other things that didn't work.
The problem was, I ticked the wrong checkbox!
Since I trigger the build from a GitLab webhook, the above checkbox (build when a...) does not have to be checked at all.
What needs to be checked is:
Trigger builds remotely (e.g., from scripts)
That checkbox provides you with a new URL:
Use the following URL to trigger build remotely:
JENKINS_URL/job/MyProject/build?token=TOKEN_NAME or
/buildWithParameters?token=TOKEN_NAME
Like all the documentation I came along states and as you can see, the URL now no longer starts with /project, but with /job instead!
So tick that box and change your URL accordingly:
http://jenkins.server:8080/**job**/project-a/buildWithParameters?token=TOKEN_NAME&MYPARAM=foo
Least I want to mention the token:
In the GitLab webhook there is a seperate field for "token", which states:
Use this token to validate received payloads. It will be sent with the request in the X-Gitlab-Token HTTP header.
So, the token provided there will be sent along the request as a HTTP header.
This is the token which can be provided globally in the Jenkins setup.
The token you must provide in the Jenkins job when ticking the box Use the following URL to trigger build remotely must be send in the URL as GET parameter, just like the example shows.
Final note: personally I have never got this working completely, because I don't get the Jenkins CSRF protection off my back. Disabling it gives me another error. However, hopefully the above does fix the problem for you and others.
GitLab plugin does not allow you to pass arbitrary parameters. In their project there is an open issue for it that deserves to be upvoted.
My convoluted solution was to use the desired values for the push trigger as the default parameters of the job. Then I used the Parameterized Scheduler plugin to use other values in the scheduled executions.
The problem is that I got a bad usability for the job when it was manually run, since the default parameters were appropriate for the push hook.
I found the solution here https://www.jittagornp.me/blog/jenkins-gitlab-webhook/
I verified it with Jenkins 2.263.1 and GitLab Community Edition 13.6.1
Your webhook url will look like
https://hunter:11a403302a4f01b9b4975c0ac27441a5cc#jenkinsservername.com/job/yourjenkinsproject/buildWithParameters?token=Aju9ryHUu6t7W8wLSeCWtY2bWjzQduYNPyY7B3gs&yourparam=yourvalue
"hunter" ist your username in Jenkins.
The following is the Jenkins API Token you have to create in your Jenkins User Managment independent of the project.
The last Token is the one you specify in the jenkins project options under "Trigger builds remotely (e.g., from scripts)"
The last thing is to add your Parameter and value to the url with ¶m=value
e.g. I have the following errors in jenkins.err.log. How can I make Jenkins email this type of errors to me?
caused by: com.microsoft.tfs.core.ws.runtime.exceptions.UnauthorizedException: Authorization failure connecting to 'http://vstsprodapp:8080/Services/v1.0/Registration.asmx' (authenticating as domain\user)
This isn't a feature that's built into Jenkins.
You could use whatever system you use for monitoring your servers, and check the logs on the Jenkins server for error messages.
There is also an idea suggested on the Jenkins wiki, which allows you to use a Groovy hook script to add a further Java logging handler to Jenkins. But again, even if you do this, you would still have to implement a custom solution to send notifications based on the log's contents.
From https://wiki.jenkins-ci.org/display/JENKINS/Logging:
Put the following Groovy script into a file called $JENKINS_HOME/init.groovy.d/extra_logging.groovy:
import java.util.logging.ConsoleHandler
import java.util.logging.LogManager
def logger = LogManager.getLogManager().getLogger("hudson.WebAppMain")
logger.addHandler (new ConsoleHandler())
This will just copy all log records generated by Jenkins and any plugins to the console.