How can I add/edit new code to my Jenkins instance that would be accesible in a DSL script? Context follows
I've inherited a Jenkins instance. Part of this inheritance includes spending the night in a haunted house writing some new automation in groovy via the Jobs DSL plugin. Since I'm fearful of ruining our jenkins instance, my first step is setting up a local development instance.
I'm having trouble running one of our existing DSL Scripts on my local development instance -- my builds on the local server fail with the following in the Jenkins error console.
Processing DSL script jobs.groovy
ERROR: startup failed:
jobs.groovy: 1: unable to resolve class thecompanysname.jenkins.extensions
The script in question starts off like this.
import thecompanysname.jenkins.extensions
use(extensions) {
def org = 'project-name'
def project = 'test-jenkins-repo'
def _email = 'foo#example.com'
So, as near I can tell, it seems like a predecesor has written some custom Groovy code that they're importing
import thecompanysname.jenkins.extensions
What's not clear to me is
Where this code lives
How I can find it in our real Jenkins instance
How I can add to to my local instance
Specific answers are welcome, as our here's how you can learn to fish answers.
While there may be other ways to accomplish this, after a bit of poking around I discovered
The Jenkins instance I've installed has an older version of the Jobs DSL plugin installed.
This version of the Jobs DSL plugin allowed you to set an additional classpath in your Process DSL Builds job section that pointed to additional jar files.
These jar files can give you access to additional classes in your groovy scripts (i.e. thecompanysname.jenkins.extensions)
Unfortunately, more recent versions of the Jobs DSL plugin have removed this option, and it's not clear if it's possible to add it back. That, however, is another question.
Configure Global Security -> uncheck "Enable script security for Job DSL
scripts".
works for me
Related
I'm trying to wrap my head around how this declarative Jenkinsfile is Groovy. I want to write supporting code to execute this outside the Jenkins environment, in pure Groovy, if that's possible. I've been writing example groovy code but still am unsure what "pipeline", "agent", and "stages" are.
Any tips to understand this structure is appreciated
EDIT: I edited this question with simplified code below. I'm just wondering if there is a way that this can be turned into valid groovy code without the preprocessor/groovyshell environment that is utilized by Jenkins
pipeline {
stages {
// extra code here
}
}
No, you can't run Jenkinsfile as a standalone Groovy script. In short, Jenkins executes the pipeline code inside a pre-configured GroovyShell that knows how to evaluate things like pipeline, agent, stages, and so forth. However, there is a way to execute Jenkinsfie without the Jenkins server - you can use JenkinsPipelineUnit test library to write JUnit/Spock unit tests that will evaluate your Jenkinsfile and display the call stack tree. It uses mocks, so you can treat it as interaction-based testing, to see if a specific part of your pipeline gets executed. Plus, you can catch some code errors prior to running the pipeline on the server.
A simple unit test for the declarative pipeline can look like this:
import com.lesfurets.jenkins.unit.declarative.*
class TestExampleDeclarativeJob extends DeclarativePipelineTest {
#Test
void should_execute_without_errors() throws Exception {
def script = runScript("Jenkinsfile")
assertJobStatusSuccess()
printCallStack()
}
}
You can find more examples in the official README.md - https://github.com/jenkinsci/JenkinsPipelineUnit
Alternatively, you can try Jenkinsfile Runner command-line tool that can execute your Jenkinsfile outside of the Jenkins server - https://github.com/jenkinsci/jenkinsfile-runner
UPDATE
I edited this question with simplified code below. I'm just wondering if there is a way that this can be turned into valid groovy code without the preprocessor/groovyshell environment that is utilized by Jenkins.
Your pipeline code example looks like a valid Jenkinsfile, but you can't turned it into a Groovy code that can be run e.g. from the command-line as a regular Groovy script:
$ groovy Jenkinsfile
This won't work, because Groovy is not aware of the Jenkins Pipeline syntax. The syntax is added as a DSL via the Jenkins plugin, and it uses a dedicated GroovyShell that is pre-configured to interpret the pipeline syntax correctly.
If you are interested in checking if the syntax of the Jenkins Pipeline is correct, there are a few different options:
npm-groovy-lint (https://github.com/nvuillam/npm-groovy-lint) can validate (and even auto-fix) the syntax of your Jenkinsfile without connecting to the Jenkins server,
Command-Line Pipeline Linter (https://www.jenkins.io/doc/book/pipeline/development/#linter) can send your pipeline code to the Jenkins server and validate its syntax.
These are a few tools that can help you with catching up the syntax errors before you run the pipeline. But that's just a nice addon to your toolbox. The first step, as always, is to understand what the syntax means, and the official documentation (https://www.jenkins.io/doc/book/pipeline/syntax) is the best place to start.
Problem Statement
I am using Jenkins for CI/CD automation. I installed the GitHub Pull Request Builder through Jenkin's Plugin Manager.
However I cannot figure out how to do this with a Jenkins Pipeline
Attempts/Documentation
I have skimmed through this exhaustive list of pipeline steps and declarative documentation but cannot find any documentation on how to accomplish this?
Installing Plugin via Jenkinsfile
Firstly, I would question why exactly you want to do this. This is a fairly odd requirement that I have not seen in my 10 years of Jenkins & Hudson experience. With that said, anything that can be done in the Jenkins web interface, and a billion other things that cannot, can be done in either a Pipeline or in the Script Console because these two constructs have access to the entire SDK.
node {
// 'ghprb' is this plugins short name value
GITHUB_PULL_REQUEST_BUILDER = 'ghprb'
for (plugin in [GITHUB_PULL_REQUEST_BUILDER]) {
e = Hudson.instance.updateCenter.getPlugin(plugin).deploy().get().getError()
if (e != null)
println e.message
}
}
Security
Anything you see on the Jenkins GUI can be done in the script console or a pipeline via the Jenkins SDK. Jenkins has implemented a Sandbox to prevent nefarious actors from executing malicious scripts. Your requirement must use the Jenkins/Hudson SDK and therefore you have two options:
Disable the Sandbox for this pipeline
Authorize the scripts and/or methods being used
If you try and run the aforementioned script without solving these security constraints the following exception will be thrown:
org.jenkinsci.plugins.scriptsecurity.sandbox.RejectedAccessException: Scripts not permitted to use method jenkins.model.Jenkins getUpdateCenter
Sandbox
To disable the Sandbox simply uncheck the "Use Groovy Sandbox" checkbox. This is the easiest solution.
Script Approval
If your administrator does not authorize you to outright disable the Sandbox for that job then you can approve the method signatures used in your script or approve the script itself.
The following message signatures would need to be approved for this script
staticMethod hudson.model.Hudson getInstance
method jenkins.model.Jenkins getUpdateCenter
method hudson.model.UpdateSite$Plugin deploy
method java.util.concurrent.Future get
method hudson.model.UpdateCenter$UpdateCenterJob getError
Additional Information
Most likely the plugin will not be available until a restart has occurred. You can automate a restart a variety of ways. You can hit the following endpoints
http://<jenkins.server>/restart
http://<jenkins.server>/safeRestart
You could invoke the jenkins-cli.jar tool. If you're feeling exceptionally risky you could even do something like Jenkins.instance.restart()
Note: don't use the SDK to restart unless you really know what you're doing
I am running Jenkins version 2.85 on Kubernetes as pod(Affinity set to one workernode). I am creating Jobs using Salt Jenkins module by passing XML to this module.
I am using Jenkins Global Library for preforming job execution.
My Job config looks like this
I am calling GobalLibrary with my parameters like repoURL, componet etc..,
Things goes well for weeks and now I landed to a weird situation where my job configurations(config.xml) gets updated/revert automatically.
Intermittently my "Build with parameter" options disappears and I can see only "Build now" in Jenkins GUI. Initially I thought someone is doing this, so to track the config changes I installed Job config history plugin in Jenkins and what I find is strange. Someone with "SYSTEM" username is making/reverting changes.
This is how it looks
and what I find is SYSTEM user revert only JOB config changes, not the PIPELINE.
I am not sure what's going wrong behind the scenes and how to stop or fix this. This is my Production instance so I am more worried.
I can see a SYSTEM user in my Jenkins
but I can not delete that user
Few relevant Question I find for this but with no answers
Configuration of Jobs getting updated by System user on Jenkins
Jenkins SYSTEM user removes custom workspace configuration
I am not sure if this Jenkins Bug or some plugin is playing with my soul.
Need help! :(
Okay I find the answer to this problem.
I have used properties in my Jekins Global Library something like this
// Disable concurrent builds
//properties([disableConcurrentBuilds()])
which overrides my external job configuration(done via salt).
Hint I get from this blog:
https://st-g.de/2016/12/parametrized-jenkins-pipelines
I also had this problem. For me it was solved when I changed the Build triggers -> Build Periodically settings from 'H 23 * * *' to '00 23 * * *'. (As I want my build to execute every night at 23:00.) Where H lets Jenkins decide when to run the job somewhere between 23:00 and 23:59 to spread load evenly. It seems Jenkins sometimes decided that it would be best to run my job on a different server and changed the parameters automatically.
In my case the issue was that the Jenkinsfile was removing the parameters I added to the pipeline from Jenkins console. Adding the same parameters in the JenkinSfile (stage -> script -> properties -> parameters) solved the issue.
In a nutshell, make sure that your Pipeline script is using the same configuration that your pipeline uses.
Jenkins documentation on parameters: https://www.jenkins.io/doc/book/pipeline/syntax/#parameters
I want to setup a Jenkins from code to
Create one initial pipeline
Create the Job DSL seed job and executing it to configure jobs used in the pipeline
Configure Jenkins settings
Locales - set locale to EN
Access control - Lock down system
I read many tutorials and questions and found the following ideas
Using the Jenkins CLI
Some Job DSL interface for setting up a job as described here at the bottom
Using JenkinsSCI interface within a Groovy file located in init.groovy.d - see below
For testing I use Docker and have the following sample already running.
Dockerfile
# https://github.com/jenkinsci/docker/blob/master/README.md
FROM jenkins/jenkins:lts
USER root
COPY groovy/* /usr/share/jenkins/ref/init.groovy.d/
USER jenkins
EXPOSE 8080
ENTRYPOINT ["/bin/tini", "--", "/usr/local/bin/jenkins.sh"]
groovy/jobs/test1-basic.groovy
#!/usr/bin/env groovy
import hudson.model.*
import jenkins.model.Jenkins;
import hudson.tasks.Shell;
job = Jenkins.instance.createProject(FreeStyleProject, 'test1-basic')
job.buildersList.add(new Shell('echo hello world'))
job.save()
The sample sadly lacks the
configuration part, as I do not know how to access the locale plugin from within the groovy code
Job DSL integration, how to read the seed job and execute it ones
I really did an intensive research and could not find much about this initial setup part. It seems many people do this manually, or the legacy way copying XML files. Could you help me out solving this and making it a "best practice documentation" for other?
If you are familiar with configuration management tool like chef you can use it for configuring your jenkins instance. There is a jenkins community cookbook which can be utilized to write a wrapper to suite your needs.
jenkins_job resource in this cookbook lets you create any type of job be it pipeline, free style etc, you just need to supply the required job configuration. You can template this with variables so based on what you supplied, job will be created accordingly. Not just jobs, you can configure almost everything you do manually with chef using a resource corresponding to that.
One of the best part about using chef is you can source control it and update configuration based on requirements at any point of time.
If you are not planning to use a configuration management tool, you can check out the discussion here on how to achieve job creation with plugins
I want to create DSL extension for my Jenkins plugin (built using maven) just like in the example of Docker plugin for Jenkins. I see that the groovy file Docker.groovy is in: src/main/resources/org/jenkinsci/plugins/docker/workflow/Docker.groovy
Does this groovy file have to be within org.jenkinsci.plugin.docker.workflow, or can I just put it inside resources? What is the difference?
Also, If I define my DSL extension within the groovy file in this manner is the DSL extension available to call implicitly in the pipeline file?
In order to make a step available in the Pipeline DSL through your plugin, you need to define a subclass of Step that performs the needed task. This can be completely done within Java, and is the preferred method for adding expanding the Pipeline DSL within a Jenkins plugin.
The Docker example you linked is unusual in this instance, and doesn't define a typical Pipeline DSL step (the docker directive in Pipeline functions like a cross between an agent, a step and a context block). Furthermore, it appears to include a Java class that loads the Groovy script dynamically, which acts as the entry point into the directive.
Groovy can be used to expand the Pipeline DSL; however this is done within the context of a shared library, which is meant to be more of a boilerplate reducing tool to be used internally.