I'm trying to trigger one or more downstream jobs using a job template. A summary of my definition:
- job-template:
name: something-{app}-build
project-type: freestyle
defaults: global
block-downstream: true
scm:
- git:
url: url
branches:
- 'master'
excluded-users:
- Jenkins
builders:
!include: templates/build-and-publish.yml
publishers:
- postbuildscript:
builders:
!include: templates/docker-build-and-push-to-ecr.yml
script-only-if-succeeded: True
mark-unstable-if-failed: True
- trigger-parameterized-builds:
- project: 'deploy-dev-ecs-service'
condition: SUCCESS
predefined-parameters: |
service={app}
envparams={envparams}
- project:
name: release-to-ecr
type: app
envparams: ''
app:
- app-1
- app-2:
- app-3:
envparams: 'FOO=42'
jobs:
- 'something-{app}-build'
Now this works, but I need to trigger different downstream jobs based on the app. This means triggering deploy-dev-ecs service multiple times with multiple parameters. For example:
app:
- app-1:
- project: deploy-dev-ecs-service
service: 'app-1'
envparams: 'foo=bar'
- app-2:
- project: deploy-dev-ecs-service
service: 'app-2.2'
envparams: 'x=2'
- project: deploy-dev-ecs-service
service: 'app-2.3'
envparams: 'x=3'
Essentially, I need to control which downstream job(s) get triggered based on
the project parameters. Is there a way to do this?
Related
I have many specflow feature files in my solution and there are multiple UI test cases.
At the page level, I have defined a tag eg #Feature1 for the first file and #feature2 for the second file. They are passed onto as the parameter in the yaml file
I pass the tag to my pipeline yml. Now I am in a situation wherein I have few test cases marked as #ignore as well
So then the pipeline runs, these test cases are not exculded but eventually fail.
I want to skip the test cases marked with the #ignore attribute/tag.
Here is a snippet from my pipeline
parameters:
- name: 'featuresToRun'
type: object
default:
- Performance
- AutoComplete
- Benches
- CATMI
- Export
- GemIntegration
- Keyboard
- MainMenu
- NewVoyages
- ReferenceData
- Settings
- SimilarVoyages
- Validation
- Views
- VolumeConversion
- Voyages
- LaycanRanges
trigger: none
jobs:
- job: startVM
timeoutInMinutes: 10
pool:
vmImage: 'windows-latest'
steps:
- checkout: none
- job: runTests
timeoutInMinutes: 1800
dependsOn: startVM
condition: not(canceled())
pool:
name: 'UI Automation'
steps:
- task: ScreenResolutionUtility#1
inputs:
displaySettings: 'optimal'
- task: VisualStudioTestPlatformInstaller#1
inputs:
packageFeedSelector: 'nugetOrg'
versionSelector: 'latestStable'
- task: NuGetCommand#2
inputs:
command: 'restore'
restoreSolution: '**/*.sln'
feedsToUse: 'config'
- task: MSBuild#1
inputs:
solution: 'UIAutomation.sln'
msbuildArchitecture: 'x64'
clean: true
- ${{each feature in parameters.featuresToRun}}:
- task: VSTest#2
displayName: ${{feature}} Tests
inputs:
testSelector: 'testAssemblies'
testAssemblyVer2: |
UIAutomation.Specs\bin\Debug\UIAutomation.Specs.dll
!**\*TestAdapter.dll
!**\obj\**
searchFolder: '$(System.DefaultWorkingDirectory)'
uiTests: true
testRunTitle: '${{feature}}'
testFiltercriteria: 'Category=${{feature}}'
rerunFailedTests: true
rerunMaxAttempts: 2
rerunFailedThreshold: 80
codeCoverageEnabled: true
continueOnError: true
Modify the testFiltercriteria to exclude that category:
testFiltercriteria: 'Category=${{feature}}&Category!=ignore'
^^^^^^^^^^^^^^^^^
it appears the testFiltercriteria property translates to the --filter argument to dotnet test or the --testcasefilter:<Expression> argument to vstest.console.exe.
The #ignore tag in Gherkin gets translates to a [TestCategory(...)] attribute above the test methods (when using MS Test). Other unit test providers have a similar conversion.
More info:
https://github.com/Microsoft/vstest-docs/blob/main/docs/filter.md
https://learn.microsoft.com/en-us/visualstudio/test/vstest-console-options
This is my first post here. I am a newbie in YAML and a beginners in Jenkins. I have a requirement where I have to create a multijob project using Jenkins Job builders where project names I want to store as a variable.
so the intention is explained below:
Job definition of Multijob Project:
job:
name: test_job
project-type: multijob
builders:
- multijob:
name: PhaseOne
condition: SUCCESSFUL
projects:
- name: PhaseOneJobA
current-parameters: true
git-revision: true
- name: PhaseOneJobB
current-parameters: true
property-file: build.props
I want to store following part in a List variable let's say : JobsList
- name: PhaseOneJobA
current-parameters: true
- name: PhaseOneJobB
current-parameters: true
as
jobsList:
- name: {list}
current-parameters: true
where list is
list:
-JobA
-JobB
so that I can use JobsList in my multijob definition. Something like this:
job:
name: test_job
project-type: multijob
builders:
- multijob:
name: PhaseOne
condition: SUCCESSFUL
projects: JobsList
and it expands into
job:
name: test_job
project-type: multijob
builders:
- multijob:
name: PhaseOne
condition: SUCCESSFUL
projects:
- name: JobA
current-parameters: true
- name: JobB
current-parameters: true
Any suggestion , how to do this?
-Archna
I am almost sure you cannot do that. I am pretty noob to above too, but I only saw so far your extended version.
My advice is that if you would be able to implement a what you want, using variables, it would be too confusing and hard to follow. Just keep it basic and have a normal list:
projects:
- name: JobA
current-parameters: true
- name: JobB
current-parameters: true
I have a Jenkins JJB template, where I would like to customize triggers per job instance. This approach would have worked great if triggers was a string parameter, but it requires an object. How can I pass an object (in this case - an array) to the job template?
- job-template:
name: 'my-template-{proj}'
trigger_overrides: []
# The obj: syntax should have worked,
# but it doesn't handle the default from above
triggers: '{obj:trigger_overrides}'
- project:
name: my-project
jobs:
# for this job, customize triggers
- 'my-template':
proj: A
trigger_overrides:
- github
- timed: "#daily"
# this job should use the default triggers
- 'my-template':
proj: B
Note that in case triggers is always passed in, the empty triggers doesn't work. The global default overrides template parameter:
- defaults:
name: global
triggers:
- github
- job-template:
name: 'my-template-{proj}'
triggers: '{obj:trigger_overrides}'
- project:
name: my-project
jobs:
- 'my-template':
proj: A
trigger_overrides: []
Neither does this work (empty array is given as a default):
triggers: '{obj:trigger_overrides|[]}
P.S. This issue was also filed with jjb devs
You want to refer to how gerrit_release_trigger_file_paths is handled here in the releng/builder repository for passing a list:
https://github.com/opendaylight/releng-builder/blob/master/jjb/lf-infra/lf-infra-jobs.yaml#L120-L130
The original template is implemented in releng-global-jjb repository here: https://github.com/lfit/releng-global-jjb/blob/master/jjb/lf-docker-jobs.yaml#L229
JJB Best Practices:
https://docs.releng.linuxfoundation.org/projects/global-jjb/en/latest/best-practices.html
- project:
name: my-project
jobs:
- 'my-template':
proj: A
trigger_overrides:
- trigger_a
- trigger_b
- trigger_c
Within JJB, you can define project-level variables like this:
- defaults:
name: global
git_url: "git#....."
- project
name: some-test
jobs:
- test-{name}
- job-template
name: test-{name}
scm:
- git:
url: "{git_url}"
branches:
- master
My question, must I hardcode the value of git_url at the default level or can I use some JJB mechanism to bring that in at job load/execution?
The reason I ask is that the yaml script that contains these JJB jobs can be used to define TEST, QA and PROD. It would be nice to just point at a properties file that contains the value for git_url and any other global variable values. I took a look at: http://docs.openstack.org/infra/jenkins-job-builder/definition.html?highlight=default#defaults and I did not see any mechanism.
If I understand your question correctly, there are two other approaches available within the context of a single yaml file
Approach 1: Set git_url at the project level
- project
name: some-test
git_url: "git#dogs.net:woof/bark.git"
jobs:
- test-{name}:
- job-template
name: test-{name}
scm:
- git:
url: "{git_url}"
branches:
- master
Here git_url is set at the project level. This approach allows you to define a second project with a different value for git_url, ie
- project
name: some-other-test
git_url: "git#cats.net:meow/meow.git"
jobs:
- test-{name}:
Approach 2: Set git_url at the job-template instance level
- project
name: some-test
jobs:
- test-{name}:
git_url: "git#....."
- job-template
name: test-{name}
scm:
- git:
url: "{git_url}"
branches:
- master
Here git_url is set on the actual instance of the job-template where it is specified. If your job-template had more than just {name} in its name, this would allow you to create multiple instances of it in the list of jobs at the project level, ie
- project
name: some-test
git_url: "git#....."
jobs:
- test-{name}-{type}:
type: 'cat'
- test-{name}-{type}:
type: 'dog'
- job-template
name: test-{name}-{type}
display-name: 'Test for {type} projects'
scm:
- git:
url: "{git_url}"
branches:
- master
Thoughts on TEST vs QA vs PROD
You also mentioned that you would like some kind of external properties file to differentiate between TEST, QA, and PROD environments. To address this let's consider four different files, project.yaml, defaults/TEST.yaml, defaults/QA.yaml, defaults/PROD.yaml whose contents are enumerated below.
project.yaml
- project
name: some-test
jobs:
- test-{name}:
defaults/TEST.yaml
- defaults:
name: global
git_url: "git#dogs.net:woof/test.git"
defaults/QA.yaml
- defaults:
name: global
git_url: "git#dogs.net:woof/qa.git"
defaults/PROD.yaml
- defaults:
name: global
git_url: "git#dogs.net:woof/prod.git"
Okay so these aren't great examples because you probably wouldn't have a different git repository for each environment, but I don't want to complicate things by straying too far from your original example.
With JJB you can specify more than one YAML file on the command line (I don't want to complicate the example or its explanation, but you can also specify directories full of JJB yaml). To differentiate between TEST, QA, and PROD deployments of your Jenkins job you can then do something like:
jenkins-jobs project.yaml:defaults/TEST.yaml
For your test environment.
jenkins-jobs project.yaml:defaults/QA.yaml
For your qa environment.
jenkins-jobs project.yaml:defaults/PROD.yaml
For your prod environment.
Hope that helps.
We have got a Jenkins setup (at the moment not managed with the Jenkins Job Builder) where every web project has 5 very similar Jenkins jobs. This is a simplified Jenkins Job Builder yaml file which could work for our needs:
- defaults:
name: global
project-type: 'freestyle'
disabled: false
buildTargets: 'build'
- job-template:
name: '{jobNameFirstSegment}-{jobNameSecondSegment}'
defaults: global
builders:
- ant:
targets: '{buildTargets}'
buildfile: 'build.xml'
- project:
name: lorem
jobNameFirstSegment:
- foo:
displayNameFirstSegment: 'Foo'
urlGitRepository: 'git#bitbucket.org:mycompany/foo.git'
- bar:
displayNameFirstSegment: 'Bar'
urlGitRepository: 'git#bitbucket.org:mycompany/bar.git'
- baz:
displayNameFirstSegment: 'Baz'
urlGitRepository: 'git#bitbucket.org:mycompany/baz.git'
jobNameSecondSegment:
- lorem:
buildTargets: 'build-lorem'
- ipsum:
buildTargets: 'build-ipsum'
- dolor:
buildTargets: 'build-dolor'
- sit:
buildTargets: 'build-sit'
- amet:
buildTargets: 'build-amet'
jobs:
- '{jobNameFirstSegment}-{jobNameSecondSegment}'
This yaml file results in 15 Jenkins jobs - all are NOT disabled:
foo-lorem
foo-ipsum
foo-dolor
foo-sit
foo-amet
bar-lorem
bar-ipsum
bar-dolor
bar-sit
bar-amet
baz-lorem
baz-ipsum
baz-dolor
baz-sit
baz-amet
Is it possible with this kind of setup to disable a specific Jenkins job? I could not find out how to override the "disabled" attribute for e.g. the Jenkins job "foo-lorem". How can this be achieved?
Infact we would need to have the possibility to be able to override any attribute of any Jenkins job if needed. Is this doable? How?
In the job template, you want to add a disabled property which takes its value from a parameter. Parameters default value can be set at the defaults and project level. So in theory you could pass the disabled value at the job level:
- defaults:
name: global
disabled: false
- job-template:
name: 'build-{name}'
defaults: global
disabled: '{obj:disabled}'
- project:
name: myproject
jobs:
- 'build-{name}'
- project:
name: mydeadproject
jobs:
- 'build-{name}':
disabled: true
Note obj: prefix is to pass the boolean as is. Turns out there is an issue in JJB specific to the disabled parameter, so you need to use a different variable (I named it prevent_run below):
- defaults:
name: global
prevent_run: false
- job-template:
name: 'build-{name}'
defaults: global
disabled: '{obj:prevent_run}'
- project:
name: myproject
jobs:
- 'build-{name}'
- project:
name: mydeadproject
jobs:
- 'build-{name}':
prevent_run: true
In the global default, the variable prevent_run is set to false and is passed to projects and job templates. For the last job, a more specific value is passed to the mydeadproject. The snippet above generates two jobs:
build-mydeadproject disabled
build-myproject enabled
You can reuse the same trick in your example template:
--- reference.yaml 2015-09-08 09:27:56.000000000 +0200
+++ foo.yaml 2015-09-08 10:02:38.000000000 +0200
## -1,11 +1,13 ##
- defaults:
name: global
project-type: 'freestyle'
- disabled: false
+ prevent_run: false
buildTargets: 'build'
+
- job-template:
name: '{jobNameFirstSegment}-{jobNameSecondSegment}'
defaults: global
+ disabled: '{obj:prevent_run}'
builders:
- ant:
You can then pass prevent_run: true on a per job basis:
- project:
name: lorem
jobNameFirstSegment:
- foo:
displayNameFirstSegment: 'Foo'
urlGitRepository: 'git#bitbucket.org:mycompany/foo.git'
prevent_run: true
That disable all jobs having foo for the value of jobNameFirstSegment. I. e. it disables the jobs:
foo-amet
foo-dolor
foo-ipsum
foo-lorem
foo-sit
When a project is realized, a cartesian products of the variables is created to generate the jobs. There is no way here to pass a parameter to a specific couple of values.