I am looking for help to create a Jenkins job which reads a value from an excel file and then use this as one of the build parameters for my job.
I have found the following documentation and I have downloaded the Pipeline Utility Steps plugin, but I am unsure where in the job config I can set this up:
https://www.jenkins.io/doc/pipeline/steps/pipeline-utility-steps/#readcsv-read-content-from-a-csv-file-in-the-workspace
I am relatively new to Jenkins so any help would be greatly appreciated. I have looked for a solution already on this page and others but have not been able to find anything that helps me. I have included a screenshot below of how my job is currently parameterised:
jenkins job screenshot
Jenkins declarative pipeline is too powerful for us, often users can abuse it. We are thinking to use an opinionated YAML to describe CI/CD pipeline. And it seems there are two choices.
Write a plugin and consume YAML and dynamically create stage / steps.
Write a plugin to convert a YAML to Jenkins pipeline.
I am not expert on Jenkins, so I hope some expert can give some guidance and maybe an example.
using official plugin pipeline-as-yaml, but it has a fixed grammar.
using or customization wolox-ci
create your own shared libaray. However, they are easy from beginning but full grammer design is required when used widely. Here is a psudo code based on curry.
// create a file named yamlCompiler.groovy in shared library,
def call(str){
def rawMap = readYaml(text: str)
// consume yaml and get a lambda function
return {
stage{
steps.each{it ->
it."$type"(it)
}
}
}
}
Use yamlCompiler in your jenkinsfile code block.
#Library('your libs name')
def str =
'''
steps:
- type: sh
script: ls -la
- type: echo
message: xxx
'''
Closure closure = yamlCompiler(str)
closure.call()
I'm looking for a similar solution. We run hardened predefined pipelines for every project, but still want to allow dev teams to customise certain steps within the process —without allowing them the full power of a Jenkinsfile.
I'm also exploring the possibility of an —in your words— "opinionated YAML".
I've so far only found one example of such an implementation: Wolox-CI supports their own pre-defined build steps via YAML. You'll be able to see the steps they support here.
I'm thinking of parsing the YAML using Snake YAML. Here's an SO answer with an example on how to do it.
Two solutions:
create a shared library to abstract the actual pipeline and provide to your users some guidance on how to setup a shared library and a Jenkinsfile sample. Here is an example of embeded pipeline https://github.com/SAP/jenkins-library/blob/master/vars/piperPipeline.groovy
use another tool like https://drone.io/
If you're not an expert and don't want/have the time to become one, the second solution might be the best one.
Really? Is the only difference here when the plugin is executed?:
Write a plugin and consume YAML and dynamically create stage / steps.
Write a plugin to convert a YAML to Jenkins pipeline.
Forgive me, because I may be a little hardened, but abstracting a layer for the dynamic creation of a declarative, or scripted, Jenkinsfile written in the simple groovy lang syntax so that it can be pretty-printed in yml prevents users from updating your yml exactly how? It seems to me your abstraction only adds to the complexity with which you wish to implement usability.
One, all the current yml plugins for Jenkins do exactly that. Two, they don't actually have the full breadth of "features" (yes, I'm using that term loosely here) accessible by implementing the groovy/(java) classes already available in the Jenkins domain (referencing the DSL). Two solutions exist right now for this, and I've investigated both, and implemented both, extensively. One is wolox-ci, which is the better of the two, and the other is Pipeline-as-YAML. In my opinion, it's easy to use, but both lack the full breadth of implementation features simply using groovy provides. So why force it? Simply so your users can have a pretty-printed yml file, and not have to be concerned with simple syntax, which you claim hardens your infrastructure-as-code backend so that the same users can't screw it up? Sorry, I'm calling bull pucky on that assertion. What's to stop anyone from totally screwing up your builds by pushing a change to the yml file which breaks the integration with groovy, or worse, completely changes an algorithm you worked hard to customize?
Sorry, I just don't get it. Sure, making something more human readable is always a good thing. Doing it because of the reasons you've stipulated makes no sense, though. Also, unless you have a super simple defined algorithm in your CI/CD process, without any non-continuous-passing-style transform methods being implemented, then using the current iterations of the yml-as-Jenkinsfile-templates plugins is probably not the way you want to go.
Now, you could write your own plugin to do this, but what's the technical debt on that, versus just learning the groovy syntax? Also, it still doesn't prevent users from making code changes to your build infrastructure, then integrating those changes in a simple yml file.
In Jenkins, we want to get the Pipeline stages information through API, e.g. a stage is success or fail. From another answer it seems we can achieve it through Pipeline REST API Plugin.
My question is:
Can Jenkinsapi and Python-Jenkins achieve the same thing? It seems they're designed for bare metal Jenkins, instead of the Pipeline plugin, right ? If that's the case, do we have similar Python library for Pipeline plugin?
Thanks!
You have all the info to answer your own question in the documentation you linked. Just to put it together:
Jenkins has a REST API
The pipeline REST API plugin injects new endpoints in the Jenkins REST API to give you access to several pipeline info
Python-Jenkins and jenkinsapi you mentioned above are both wrappers around the Jenkins REST API to help you develop scripts/apps targeting the Jenkins API in Python. Since those modules/libs are based most probably based on the core API specification, they most probably don't provide specific methods to target the pipeline endpoints (but you can probably extend that quite easily).
If you want to stay in Jenkinsapi, get_data() defined in JenkinsBase class could be used for querying Pipeline REST API directly. Not very sure if it's recommended or not.
Following codes works for me.
from jenkinsapi.jenkins import Jenkins
import requests
requests.packages.urllib3.disable_warnings()
# GET /job/:job-name/:run-id/wfapi/describe
url = 'https://localhost/job/anyjob/10/wfapi/describe'
jenkins = Jenkins(
'https://localhost',
username='username',
password='api_token',
ssl_verify=False,
lazy=True)
print(jenkins.get_data(url))
I find it hard to read the Jenkins javadoc. (I'll already here apologize for my java ignorance. I just have basic knowledge and are only able to write simple programs.)
When working in Jenkins and groovy I found the below useful example that shows how to find all Jenkins jobs and print some property from it:
def hi = hudson.model.Hudson.instance
hi.getItems(hudson.model.Project).each {project ->
println(project.lastBuild.result)
}
It shows how to get the lastBuild.result property of the project object. Reading the API docs on hudson.model.Hudson I find a method getItems that returns some List of an arbitrary Class. So I assume that project is of class hudson.model.Project.
Now to my problem. When I read the documentation on hudson.model.Project I can't find any lastBuild property. The only properties that are listed are the inherited ones. No listing of the class' own properties.
Where do I find such a list? Or what is it that I don't understand?
Thanks!
If I know a class name or field name in the Jenkins model for whatever I am dealing with, I often find it easier to go directly to the Jenkins source code on GitHub to understand it. So you could go to https://github.com/jenkinsci/jenkins and search for lastBuild there. If you are running Groovy script in Jenkins and you want to know what class some object x is, just do "println x. getClass(). getName()". Then you can find more about the class searching the GitHub repo. Of course you could pull down all the Jenkins source code locally and navigate and search through it with something like IntelliJ IDEA or another Java IDE. You need to be comfortable with reading Java code.
New to Jenkins and new to the way it is being used to my new workplace, I encountered the following in a job's page:
I would like to understand what I see, so a quick Google search yielded the following in regard to the use of 'Static' and '(blocking)' in Jenkins:
Static Code Analysis Plug-ins
Build Blocker Plugin
Are these the right places to look at for attempting to understand the meaning of these keywords in the context of Jenkins?
If not, what is the right place to find out about them?