When I create a jenkins job via the API, I use a previous job's config.xml, make my modifications and then make the POST call to create the job.
My questions is, is there a way to generate this programmatically? I.e. is there a structure of a config.xml, what XML entities it should have, what values, etc so I can write a small module to generate one and send it to the jenkins API call?
I don't think there's any mandatory XML entities. Submitting an empty structure should result in a job that has default values for all settings.
What you want to do is exactly what's done by the Jenkins Job Builder. It provides a YAML-based framework for creating Job configuration XML files and submitting them to Jenkins. It's a common alternative to the Job DSL plugin. I wouldn't recommend to re-implement such a solution yourself -- handling all the plugin-specific XML configuration parts will be a nightmare.
We create our jobs using Job DSL plugin. You can try the playground http://job-dsl.herokuapp.com/.
At first, it seems that it is hard to learn, but after the first seed job, it is much better.
When we started writing our scripts we were afraid that there will not be suitable API methods for our needs. It turned out that we had one such case, which was solved using the configure block.
Get started guide here.
Related
Jenkins declarative pipeline is too powerful for us, often users can abuse it. We are thinking to use an opinionated YAML to describe CI/CD pipeline. And it seems there are two choices.
Write a plugin and consume YAML and dynamically create stage / steps.
Write a plugin to convert a YAML to Jenkins pipeline.
I am not expert on Jenkins, so I hope some expert can give some guidance and maybe an example.
using official plugin pipeline-as-yaml, but it has a fixed grammar.
using or customization wolox-ci
create your own shared libaray. However, they are easy from beginning but full grammer design is required when used widely. Here is a psudo code based on curry.
// create a file named yamlCompiler.groovy in shared library,
def call(str){
def rawMap = readYaml(text: str)
// consume yaml and get a lambda function
return {
stage{
steps.each{it ->
it."$type"(it)
}
}
}
}
Use yamlCompiler in your jenkinsfile code block.
#Library('your libs name')
def str =
'''
steps:
- type: sh
script: ls -la
- type: echo
message: xxx
'''
Closure closure = yamlCompiler(str)
closure.call()
I'm looking for a similar solution. We run hardened predefined pipelines for every project, but still want to allow dev teams to customise certain steps within the process —without allowing them the full power of a Jenkinsfile.
I'm also exploring the possibility of an —in your words— "opinionated YAML".
I've so far only found one example of such an implementation: Wolox-CI supports their own pre-defined build steps via YAML. You'll be able to see the steps they support here.
I'm thinking of parsing the YAML using Snake YAML. Here's an SO answer with an example on how to do it.
Two solutions:
create a shared library to abstract the actual pipeline and provide to your users some guidance on how to setup a shared library and a Jenkinsfile sample. Here is an example of embeded pipeline https://github.com/SAP/jenkins-library/blob/master/vars/piperPipeline.groovy
use another tool like https://drone.io/
If you're not an expert and don't want/have the time to become one, the second solution might be the best one.
Really? Is the only difference here when the plugin is executed?:
Write a plugin and consume YAML and dynamically create stage / steps.
Write a plugin to convert a YAML to Jenkins pipeline.
Forgive me, because I may be a little hardened, but abstracting a layer for the dynamic creation of a declarative, or scripted, Jenkinsfile written in the simple groovy lang syntax so that it can be pretty-printed in yml prevents users from updating your yml exactly how? It seems to me your abstraction only adds to the complexity with which you wish to implement usability.
One, all the current yml plugins for Jenkins do exactly that. Two, they don't actually have the full breadth of "features" (yes, I'm using that term loosely here) accessible by implementing the groovy/(java) classes already available in the Jenkins domain (referencing the DSL). Two solutions exist right now for this, and I've investigated both, and implemented both, extensively. One is wolox-ci, which is the better of the two, and the other is Pipeline-as-YAML. In my opinion, it's easy to use, but both lack the full breadth of implementation features simply using groovy provides. So why force it? Simply so your users can have a pretty-printed yml file, and not have to be concerned with simple syntax, which you claim hardens your infrastructure-as-code backend so that the same users can't screw it up? Sorry, I'm calling bull pucky on that assertion. What's to stop anyone from totally screwing up your builds by pushing a change to the yml file which breaks the integration with groovy, or worse, completely changes an algorithm you worked hard to customize?
Sorry, I just don't get it. Sure, making something more human readable is always a good thing. Doing it because of the reasons you've stipulated makes no sense, though. Also, unless you have a super simple defined algorithm in your CI/CD process, without any non-continuous-passing-style transform methods being implemented, then using the current iterations of the yml-as-Jenkinsfile-templates plugins is probably not the way you want to go.
Now, you could write your own plugin to do this, but what's the technical debt on that, versus just learning the groovy syntax? Also, it still doesn't prevent users from making code changes to your build infrastructure, then integrating those changes in a simple yml file.
I am using the scriptrunner plugin for Jira.
Is it possible to add a condition to a transition using scriptrunner?
Currently, my condition is in a script which I have manually added to the workflow.
But I was wondering if there is a way to do it automatically?
I was looking through documentation on: https://docs.atlassian.com/
I came across this method:
replaceConditionInTransition which is a method of WorkFlowManager.
But I'm unsure on how to use this.
Any help would be appreciated.
Conditions as any another scripts can be added from file system. You can store scripts in any VCS (bitbucket, github, gitlab, etc) and automatically deploy them to Jira server file system through any CI/CD system (teamcity, jenkins, bamboo, gitlab, etc).
So, as result process will be looks like. 1. commit changes in you script to vcs 2. wait a bit for auto deploy (e.g. triggered by commit) 3. done. As additional you can write any script/service/etc for commit these changes automatically if needed.
Also look at script roots it's helpful way which allows reuse any of script fragments through helpers classes.
It's rather conceptual answer basically because implementation is depends on environment, but I hope that you get at least one more point of view to solve this task.
I think that using the Java API to modify Jira workflows is pretty tough. You could dig around in the workflow editor to see how conditions are added there. Remember that you have to do this in a draft workflow and then publish it, which takes some time in large projects
I like the idea of replacing a script file as easier, if it can be done when no issues are transitioning
I recently managed to convert several manually-created jobs to DSL scripts (inlined into temporary 'seed' jobs), and was pleasantly surprised how straightforward it was. Now I'd like to get rid of the multiple seed jobs and try to structure things more cleanly.
To that end, I created a new jenkins-ci repo and committed all the Groovy DSL scripts to it. Then I created a job-generator Jenkins job that pulls from the jenkins-ci repo and has a single Process Job DSLs step. This step has the Look on Filesystem box ticked, with the DSL Scripts field set to jobs/*.groovy. With global push notifications already in place, this works more-or-less as intended: if I make a change to the jenkins-ci repo, the job-generator job automatically runs and regenerates all the jobs—awesome!
What I don't like about this solution is that it has poor locality of reference: the DSL scripts for the job live in a completely separate repository from the code. What I'd really like is to keep the job DSL scripts in each individual code repository, in a jenkins subfolder, and have a single seed job that processes them all. That way, changes to CI setup could be code-reviewed right alongside the code. To me, that just feels like an ideal setup.
Unfortunately, I don't have a clear idea about how to make this happen. If I could figure out a way to make the seed job watch multiple repos, such that a commit to any one of them would trigger it, perhaps I could inject another build step before the Process Job DSLs step and (somehow) script my way to victory, but... I'm unsure how to even get to that point. (I certainly don't want to do full clones of each repo in the generator job just to pull in the DSL scripts!)
I suspect I'm not the first person to wish they could put the Job DSL scripts alongside the code, though perhaps I'm over-estimating the benefits. Any advice on this topic would be much appreciated—thanks!
Unfortunately there is no direct way of solving this. Several feature requests have been opened (JENKINS-33275, JENKINS-37220), but AFAIK no one is working on any of them.
As a workaround you can use the Pipeline Multibranch Plugin and create a multibranch project for each of your repositories. You must then add a simple Jenkinsfile to each repo/branch and use the Jenkinsfile to execute your Job DSL scripts. See Use Job DSL in Pipeline scripts for details. This would require minimal coding, but I think each repo must be cloned for this to work because the Job DSL files must be available on the file system.
You can use Job DSL to create the multibranch jobs, see multibranchPipelineJob in the API viewer. This would be your "root" seed job.
If your repos are hosted on GitHub, you can also checkout the GitHub Organization Folder Plugin. With that plugin you must only create one job for each organization instead of multiple multibranch jobs.
I have about 200 jenkins, each of them has a long config page but actually most config are the same. Everytime when I need to update something in the common config, I write a groovy script to loop though those jobs and update them one by one. It's a pain because it takes about 5 minutes to update those jobs by the groovy script. I am wondering is there a jenkins plugin(or something else) that I can use to put the common config in one place? jenkins slicing plugin doesn't work well, I think it conflicts with another plugin.
Thanks
Sounds like a job for the job-dsl plugin
From the wiki page
The Jenkins job-dsl-plugin attempts to solve this problem by allowing
jobs to be defined with the absolute minimum necessary in a
programmatic form, with the help of templates that are synced with the
generated jobs. The goal is for your project to be able to define all
the jobs they want to be related to their project, declaring their
intent for the jobs, leaving the common stuff up to a template that
were defined earlier or hidden behind the DSL.
My Jenkins job runs many tests that create log files. In case of failure, I want to look at the log of the failed test. I'd rather use Jenkins web-server to do it, even have a link in the email it sends me.
Is there any plugin that can do it? Or maybe another way?
You provide few details in your question, so it is impossible to give specific advice. In a general level: this is already possible. When your test framework creates JUnit XML files with test results, the test output can be included between the <failure> and </failure> tags. Usually test frameworks should take care of this automatically, so you are probably not using a test framework and are manually generating the XML files containing test results?
I recommend you adopt some test framework. It is usually well worth the effort.