Given a project like this
- project:
name: my-project
type:
- apple
and a job-template like this
- job-template:
project-type: freestyle
name: '{name}-build-{type}'
builders:
!include: "builders/{type}.yml"
I get the error
IOError: [Errno 2] No such file or directory: 'builders/{type}.yml'
which makes me thing that JJB isn't willing to do variable subtitution in the path of an included file.
Is there any way to do this?
Thanks.
Related
I'd need to get started with JJB however one issue I'm facing is that I'm not able to upload jobs in an existing Folder (I'm using Cloudbees Folder plugin).
I've arranged an example based on an existing question available on SO:
- project:
name: playground
- job:
name: sample_job
description: 'Automatically generated test'
project-type: freestyle
builders:
- shell: 'ls'
So the job should be loaded into the "playground" folder.
However the syntax seems uncorrect:
File "/usr/lib64/python2.7/site-packages/yaml/constructor.py", line 37, in get_single_data
node = self.get_single_node()
File "/usr/lib64/python2.7/site-packages/yaml/composer.py", line 36, in get_single_node
document = self.compose_document()
File "/home/francesco/.local/lib/python2.7/site-packages/jenkins_jobs/local_yaml.py", line 282, in compose_document
node = self.compose_node(None, None)
File "/usr/lib64/python2.7/site-packages/yaml/composer.py", line 82, in compose_node
node = self.compose_sequence_node(anchor)
File "/usr/lib64/python2.7/site-packages/yaml/composer.py", line 110, in compose_sequence_node
while not self.check_event(SequenceEndEvent):
File "/usr/lib64/python2.7/site-packages/yaml/parser.py", line 98, in check_event
self.current_event = self.state()
File "/usr/lib64/python2.7/site-packages/yaml/parser.py", line 393, in parse_block_sequence_entry
"expected <block end>, but found %r" % token.id, token.start_mark)
yaml.parser.ParserError: while parsing a block collection
in "/home/francesco/git/tests-jobs/jobs/test.yaml", line 1, column 1
expected <block end>, but found '?'
in "/home/francesco/git/tests-jobs/jobs/test.yaml", line 2, column 1
The existing answers provided so far on this question are related to folders created as part of JJB. In my case the folder has been already created ahead.
Any help ?
#francesco, Basically you have to use jobs, job template to create a jenkins job. I have tried this based on your question and able to execute without failures. Check if this answers your question.
---
- project:
name: playground
jobs:
- sample_job
- job-template:
name: sample_job
description: 'Automatically generated test'
project-type: freestyle
builders:
- shell: 'ls'
I am unable to define the new plugins while using jenkins job builder to build my jobs. The Jenkins UI has these plugins. I am new to JJB, hence I am not too sure how these new plugins get defined. This is the configuration used in my YAML:
- job:
name: Run_Pipeline
project-type: freestyle
defaults: global
description: Build to pass variables for a pipeline
disabled: false
node: Linux
logrotate:
numToKeep: 5
parameters:
- choice:
name: TYPE
choices: ['DEV','QA']
- reactive_choice:
name: ENV_NAME
description: "Select environment (depending on Type) where the build has to be deployed"
referencedParameters: TYPE
script: |
if (TYPE == 'DEV'){
return['dev1','dev2','dev3']
}else if(TYPE == 'QA'){
return['qa1','qa2','qa3','qa4']
}else{
return['Uknown Type']
}
choiceType: PT_SINGLE_SELECT
filterLength: 1
builders:
- shell: |
if [[ "$TYPE" == "QA" && "$PASSWORD" != "password" ]];then echo "Invalid Password for QA Deployment"; exit 1; fi
While running this yaml through jenkins-job builder, I am getting the error:
jenkins_jobs.errors.JenkinsJobsException: Unknown entry point or macro 'reactive_choice' for component type: 'parameter'.
I know this method is not defined in the module parameters.py; I found one in GITHub, but I am not sure how to integrate that method with the existing package.
https://github.com/ochirkov/jenkins-job-builder-active-choice-reactive-param
What am I missing? Any help is highly appreciated.
Thanks!
I got it, it's pretty straightforward. I had easy_install installed in my file path and I had to just run the package from github as follows:
easy_install jjb-reactive-choice-param-0.3.0
This link was quite helpful - http://setuptools.readthedocs.io/en/latest/easy_install.html
Simple enough!
Within JJB, you can define project-level variables like this:
- defaults:
name: global
git_url: "git#....."
- project
name: some-test
jobs:
- test-{name}
- job-template
name: test-{name}
scm:
- git:
url: "{git_url}"
branches:
- master
My question, must I hardcode the value of git_url at the default level or can I use some JJB mechanism to bring that in at job load/execution?
The reason I ask is that the yaml script that contains these JJB jobs can be used to define TEST, QA and PROD. It would be nice to just point at a properties file that contains the value for git_url and any other global variable values. I took a look at: http://docs.openstack.org/infra/jenkins-job-builder/definition.html?highlight=default#defaults and I did not see any mechanism.
If I understand your question correctly, there are two other approaches available within the context of a single yaml file
Approach 1: Set git_url at the project level
- project
name: some-test
git_url: "git#dogs.net:woof/bark.git"
jobs:
- test-{name}:
- job-template
name: test-{name}
scm:
- git:
url: "{git_url}"
branches:
- master
Here git_url is set at the project level. This approach allows you to define a second project with a different value for git_url, ie
- project
name: some-other-test
git_url: "git#cats.net:meow/meow.git"
jobs:
- test-{name}:
Approach 2: Set git_url at the job-template instance level
- project
name: some-test
jobs:
- test-{name}:
git_url: "git#....."
- job-template
name: test-{name}
scm:
- git:
url: "{git_url}"
branches:
- master
Here git_url is set on the actual instance of the job-template where it is specified. If your job-template had more than just {name} in its name, this would allow you to create multiple instances of it in the list of jobs at the project level, ie
- project
name: some-test
git_url: "git#....."
jobs:
- test-{name}-{type}:
type: 'cat'
- test-{name}-{type}:
type: 'dog'
- job-template
name: test-{name}-{type}
display-name: 'Test for {type} projects'
scm:
- git:
url: "{git_url}"
branches:
- master
Thoughts on TEST vs QA vs PROD
You also mentioned that you would like some kind of external properties file to differentiate between TEST, QA, and PROD environments. To address this let's consider four different files, project.yaml, defaults/TEST.yaml, defaults/QA.yaml, defaults/PROD.yaml whose contents are enumerated below.
project.yaml
- project
name: some-test
jobs:
- test-{name}:
defaults/TEST.yaml
- defaults:
name: global
git_url: "git#dogs.net:woof/test.git"
defaults/QA.yaml
- defaults:
name: global
git_url: "git#dogs.net:woof/qa.git"
defaults/PROD.yaml
- defaults:
name: global
git_url: "git#dogs.net:woof/prod.git"
Okay so these aren't great examples because you probably wouldn't have a different git repository for each environment, but I don't want to complicate things by straying too far from your original example.
With JJB you can specify more than one YAML file on the command line (I don't want to complicate the example or its explanation, but you can also specify directories full of JJB yaml). To differentiate between TEST, QA, and PROD deployments of your Jenkins job you can then do something like:
jenkins-jobs project.yaml:defaults/TEST.yaml
For your test environment.
jenkins-jobs project.yaml:defaults/QA.yaml
For your qa environment.
jenkins-jobs project.yaml:defaults/PROD.yaml
For your prod environment.
Hope that helps.
I am investigating using Jenkins Job-Builder (from OpenStack) as our means of managing jenkins job configurations. In doing so I am trying to figure out the right (best?) way to include a job-template from an external file using the !include custom tag.
In the current use case we will basically have one template that is going to be used by a LOT of job. Each job is going to need to exist in its own file for reason that are out of scope here.
So far I have gotten this to work:
job-template.yml
name: 'pre-build-{proj}-{repo}'
project-type: freestyle
... etc ...
job-1.yml
- job-template:
!include job-template.yml
- project:
name: job-1
proj: my-proj
repo: my-repo
jobs:
- 'build-config-{proj}-{repo}'
This seem wrong because the template definition gets split across both files and require needless duplication of the -job-template: line in every job file. I would like to get the following to work instead:
job-template.yml
- job-template:
name: 'pre-build-{proj}-{repo}'
project-type: freestyle
... etc ...
job-1.yml
!include job-template.yml
- project:
name: job-1
proj: my-proj
repo: my-repo
jobs:
- 'build-config-{proj}-{repo}'
The latter unfortunately results in a yaml parse error on the - project: line:
yaml.scanner.ScannerError: mapping values are not allowed here
in "job-1.yml", line 3, column 10
Is there way to get the entire template definition into the template file? This will become particularly annoying if ever we need to pull in multiple templates from multiple files.
Jenkins-jobs takes a path argument which can be a directory holding your files (job-template.yaml, job-1.yaml and job-2.yaml. It will assemble them as a single YAML document, so you do not need to use !include. So you can write:
job-template.yaml
- job-template:
name: 'pre-build-{proj}-{repo}'
builders:
- shell: 'echo building {proj} for {repo}'
job1.yaml
- project:
name: job-1
proj: my-proj
repo: my-repo
jobs:
- 'pre-build-{proj}-{repo}'
job2.yaml
- project:
name: job-2
proj: my-other-proj
repo: my-other-repo
jobs:
- 'pre-build-{proj}-{repo}'
That will generates two jobs with the following shell commands:
pre-build-my-other-proj-my-other-repo:
<command>echo building my-other-proj for my-other-repo</command>
pre-build-my-proj-my-repo:
<command>echo building my-proj for my-repo</command>
Assuming the files are in a directory config/ you can generate them all with:
jenkins-jobs test config/ -o /tmp/myjobs
Or use the name argument to filter the jobs that will be realized:
jenkins-jobs test config/ -o /tmp/myjobs '*my-proj*'
# Creates pre-build-my-proj-my-repo
# Skips pre-build-my-other-proj-my-other-repo
I am using jenkins-job-builder to create my pipeline project. But I have a problem with the variables values when I am trying to reuse or propagating.
It is my project configuration:
- project:
name: myproject
git_url: git#gitlabserver.cu:demos-products/myproject.git
jobs:
- '{name}-nfr-smoke-tests':
pipeline-next: '{name}-nfr-smoke-tests'
And here is my job-template:
- job-template:
name: "{name}-nfr-smoke-tests"
node: 'slave1'
scm:
- git:
skip-tag: false
url: 'git#gitlabserver.cu:test-products/{name}-nfr-tests.git'
branches:
- master
wipe-workspace: true
builders:
- shell: |
bundle install
bundle exec cucumber features/smoke.feature
publishers:
- trigger:
project: "{pipeline-next}"
threshold: SUCCESS
Ok, now when I run this configuration in jenkins and I check the job's construction, it says:
No such project ‘{name}-nfr-smoke-tests’. Did you mean ‘myproject-nfr-smoke-tests’?
Why the line: pipeline-next: '{name}-nfr-smoke-tests'doesn't propagates the value of variable name and just used it as a literal string? I am missing something.
You are missing 'name' under 'project' section in your job-template. Append the following lines:
- project:
name: project-name
The purpose of a project is to collect related jobs together, and provide values for the variables in a Job Template.
I found out that Jenkins Job Builder version 0.9.0-0.2 does not propagate the value, but for me version 1.3.0+2015.12.15.git136.959eb4b909-1 did. Perhaps updating your version of Jenkins Job Builder might help?