I am looking for a way to build continous delivery with cloudbees,so each time a commit is made my code is compiled,unit tested and deployed to staging (running on run#cloud).
For simplity let's call it "staging-build". Then after some manual testing on staging env i would like to have an option to promote a particular (not the last) "staging-build" so the artifact the promoted build produced should be deployed to production env via "prod-build" job.
The basic idea was to configure "promote build" for "staging-build" job with building "prod-build" as promote action. Nevertheless clicking on promote-build does not trigger starting "prod-build". I also tried from the other side: removed building "prod-build" as promote action and enabling "Build when another project is promoted" within "prod-build" configuration. Still no success. After googling for a while i found
http://blog.cloudbees.com/2012/10/continuous-integration-for-mobile-apps.html. Nevertheless blog entry along with jenkins configuration screenshots do not reflect configurations on https://partnerdemo.ci.cloudbees.com
I finally found the root of problem: jobs to be run on promotion must be prefixed with folder name (obviously in case they are in folder). In other case their run is silently skipped - even promotion log has no "scheduling build for ..." entry. IMHO Jenkins web console should show some kind of warning in that case, just like it does when invalid job name is typed-in.
configuring a promotion to trigger your prod-build job is the way to go. There may be some configuration issue that isn't visible in your question. Please give me your account ID so I can double check your configuration
Related
I have installed the copyArtifacts plugin and created two freestyle jobs: experiment-main and experiment-1
experiment-1 just creates a file called artifact.txt with the build # in it, and archives it.
experiment-main triggers experiment-1 and then tries to copy the artifact like this:
but this is the result:
Running as SYSTEM
Building on master in workspace /var/lib/jenkins/workspace/experiment-main
Waiting for the completion of experiment-1
experiment-1 #4 started.
experiment-1 #4 completed. Result was SUCCESS
Build step 'Trigger/call builds on other projects' changed build result to SUCCESS
ERROR: Unable to find a build for artifact copy from: experiment-1
Finished: FAILURE
which isn't what I expected (or at least what I was hoping for)
I hoped it would find the experiment-1 build that was downstream from the current build.
Any ideas?
I figured out that there are variables with the numbers of triggered builds that I can use. To figure out the variable, I just printed all the environment variables with env and then found the right variable in the list.
Then I configured the copy artifacts plugin to use that build number.
I couldn't do it how #alex-o suggested, just getting the last build of the subjob, because I might have more than one job using the subjob at once, but if you don't have that problem, that might work for you.
Yes, this is unexpected behavior indeed.
The reason why this won't work is hidden in the help text of the "Upstream Project Name" input field:
Downstream builds are found using fingerprints of files. That is, a build that is triggered from a build isn't always considered downstream, but you need to fingerprint files used in builds to let Jenkins track them.
So, the Copy-Artifact plugin relies on fingerprint data to determine job ancestry. For that reason, you can not use the "Downstream build of..." feature using the current job as a parent: fingerprints are recorded in a post-build step, so an ongoing build of example-master does not have any fingerprints associated to it by the time it is looking for a matching build of experiment-1.
It is possible to modify fingerprint information at build run-time (e.g., via Groovy), but then, it's probably best to avoid the Copy-Artifact plugin entirely and to implement the whole procedure in Groovy right away.
Your best bet is probably to refer to example-1 via "Last successful build" and to ensure that this is the build that you triggered before (usually this will be correct, but depending on your setup there can be race conditions).
I am running Jenkins version 2.85 on Kubernetes as pod(Affinity set to one workernode). I am creating Jobs using Salt Jenkins module by passing XML to this module.
I am using Jenkins Global Library for preforming job execution.
My Job config looks like this
I am calling GobalLibrary with my parameters like repoURL, componet etc..,
Things goes well for weeks and now I landed to a weird situation where my job configurations(config.xml) gets updated/revert automatically.
Intermittently my "Build with parameter" options disappears and I can see only "Build now" in Jenkins GUI. Initially I thought someone is doing this, so to track the config changes I installed Job config history plugin in Jenkins and what I find is strange. Someone with "SYSTEM" username is making/reverting changes.
This is how it looks
and what I find is SYSTEM user revert only JOB config changes, not the PIPELINE.
I am not sure what's going wrong behind the scenes and how to stop or fix this. This is my Production instance so I am more worried.
I can see a SYSTEM user in my Jenkins
but I can not delete that user
Few relevant Question I find for this but with no answers
Configuration of Jobs getting updated by System user on Jenkins
Jenkins SYSTEM user removes custom workspace configuration
I am not sure if this Jenkins Bug or some plugin is playing with my soul.
Need help! :(
Okay I find the answer to this problem.
I have used properties in my Jekins Global Library something like this
// Disable concurrent builds
//properties([disableConcurrentBuilds()])
which overrides my external job configuration(done via salt).
Hint I get from this blog:
https://st-g.de/2016/12/parametrized-jenkins-pipelines
I also had this problem. For me it was solved when I changed the Build triggers -> Build Periodically settings from 'H 23 * * *' to '00 23 * * *'. (As I want my build to execute every night at 23:00.) Where H lets Jenkins decide when to run the job somewhere between 23:00 and 23:59 to spread load evenly. It seems Jenkins sometimes decided that it would be best to run my job on a different server and changed the parameters automatically.
In my case the issue was that the Jenkinsfile was removing the parameters I added to the pipeline from Jenkins console. Adding the same parameters in the JenkinSfile (stage -> script -> properties -> parameters) solved the issue.
In a nutshell, make sure that your Pipeline script is using the same configuration that your pipeline uses.
Jenkins documentation on parameters: https://www.jenkins.io/doc/book/pipeline/syntax/#parameters
Here is something I want to achieve:
I have a jenkins project which has 4 upstream projects. But I don't want to trigger this project when the upstream jobs are done building, but I want the trigger the project via remote API, which then waits on upstream projects until they are done building, if these projects are building.
Lets say all the 4 upstream projects can build the source code from any branch passed via API, but I want the downstream project to start only when a specific branch is passed to these upstream projects.
Scenario:
Lets say I have two clusters A and B, for the sake of this question, I want to deploy my code to cluster A, i.e front end and backend code. Now I have a project to build front end and 1 project to build backend (these two projects can build code for cluster A and B, based on the branch passed). Now, I have two deploy projects for cluster A which will deploy front end and backend. So, when I pass a branch to build code for cluster A, it will trigger the build projects. But now I only want these two deploy projects to start when this specific branch was passed.
If you want to control the builds remotely then use the Jerkins cli - I have found it very useful http://jenkinshost:8080/cli
You need to get the ssh key config right, add the public key of the user running the cli to the user you want to run the job in Jenkins using the Jenkins user configuration (not on the command line
Test key setup with
java -jar jenkins=cli.jar -s http://jenkinshost:8080 who-am-i
This should then report which user will be used to run the build in Jenkins
But I think you can use the Conditional Build Step plugin for your problem
https://wiki.jenkins-ci.org/display/JENKINS/Conditional+BuildStep+Plugin
This will allow you to put a conditional wrapper around a build step i.e.
if branch==branchA then
trigger step - deploy to clusterA
if branch==branchB then
trigger step - deploy to clusterB
Personally I find this plugin a bit clunky and it makes the job config page a little messy
Another solution I came up with was to always call the child job and then let it decide if it runs.
So I have a script step at the start of the child job to see if it should run
if [${branch}="Not the right branch name" ] ; then
echo "EXIT_GREEN"
exit 1
fi
You have now failed this job which would cause the parent job to go red but by using the Groovy Postbuild plugin https://wiki.jenkins-ci.org/display/JENKINS/Groovy+Postbuild+Plugin you can add a post build step like this
if (manager.logContains(".*EXIT_GREEN.*")) {
manager.addBadge("info.gif","This job had nothing to do")
manager.build.#result = hudson.model.Result.SUCCESS
}
Child job has run green (with an info icon against the build) but has actually not done anything. Obviously if the branch is one you want deploy then the first script step does not run the exit 1 and the job continues as normal
In order to get the fastest feedback possible, we occasionally want Jenkins jobs to run in Parallel. Jenkins has the ability to start multiple downstream jobs (or 'fork' the pipeline) when a job finishes. However, Jenkins doesn't seem to have any way of making a downstream job only start of all branches of that fork succeed (or 'joining' the fork back together).
Jenkins has a "Build after other projects are built" button, but I interpret that as "start this job when any upstream job finishes" (not "start this job when all upstream jobs succeed").
Here is a visualization of what I'm talking about. Does anyone know if a plugin exists to do what I'm after?
Edit:
When I originally posted this question in 2012, Jason's answer (the Join and Promoted Build plugins) was the best, and the solution I went with.
However, dnozay's answer (The Build Flow plugin) was made popular a year or so after this question, which is a much better answer. For what it's worth, if people ask me this question today, I now recommend that instead.
Pipeline plugin
You can use the Pipeline Plugin (formerly workflow-plugin).
It comes with many examples, and you can follow this tutorial.
e.g.
// build
stage 'build'
...
// deploy
stage 'deploy'
...
// run tests in parallel
stage 'test'
parallel 'functional': {
...
}, 'performance': {
...
}
// promote artifacts
stage 'promote'
...
Build flow plugin
You can also use the Build Flow Plugin. It is simply awesome - but it is deprecated (development frozen).
Setting up the jobs
Create jobs for:
build
deploy
performance tests
functional tests
promotion
Setting up the upstream
in the upstream (here build) create a unique artifact, e.g.:
echo ${BUILD_TAG} > build.tag
archive the build.tag artifact.
record fingerprints to track file usage; if any job copies the same build.tag file and records fingerprints, you will be able to track the parent.
Configure to get promoted when promotion job is successful.
Setting up the downstream jobs
I use 2 parameters PARENT_JOB_NAME and PARENT_BUILD_NUMBER
Copy the artifacts from upstream build job using the Copy Artifact Plugin
Project name = ${PARENT_JOB_NAME}
Which build = ${PARENT_BUILD_NUMBER}
Artifacts to copy = build.tag
Record fingerprints; that's crucial.
Setting up the downstream promotion job
Do the same as the above, to establish upstream-downstream relationship.
It does not need any build step. You can perform additional post-build actions like "hey QA, it's your turn".
Create a build flow job
// start with the build
parent = build("build")
parent_job_name = parent.environment["JOB_NAME"]
parent_build_number = parent.environment["BUILD_NUMBER"]
// then deploy
build("deploy")
// then your qualifying tests
parallel (
{ build("functional tests",
PARENT_BUILD_NUMBER: parent_build_number,
PARENT_JOB_NAME: parent_job_name) },
{ build("performance tests",
PARENT_BUILD_NUMBER: parent_build_number,
PARENT_JOB_NAME: parent_job_name) }
)
// if nothing failed till now...
build("promotion",
PARENT_BUILD_NUMBER: parent_build_number,
PARENT_JOB_NAME: parent_job_name)
// knock yourself out...
build("more expensive QA tests",
PARENT_BUILD_NUMBER: parent_build_number,
PARENT_JOB_NAME: parent_job_name)
good luck.
There are two solutions that I have used for this scenario in the past:
Use the Join Plugin on your "deploy" job and specify "promote" as the targeted job. You would have to specify "Functional Tests" and "Performance Tests" as the joined jobs and start them via in some fashion, post build. The Parameterized Trigger Plugin is good for this.
Use the Promoted Builds Plugin on your "deploy" job, specify a promotion that works when downstream jobs are completed and specify Functional and Performance test jobs. As part of the promotion action, trigger the "promote" job. You still have to start the two test jobs from "deploy"
There is a CRITICAL aspect to both of these solutions: fingerprints must be correctly used. Here is what I found:
The "build" job must ORIGINATE a new fingerprinted file. In other words, it has to fingerprint some file that Jenkins thinks was originated by the initial job. Double check the "See Fingerprints" link of the job to verify this.
All downstream linked jobs (in this case, "deploy", "Functional Tests" and "Performance tests") need to obtain and fingerprint this same file. The Copy Artifacts plugin is great for this sort of thing.
Keep in mind that some plugins allow you change the order of fingerprinting and downstream job starting; in this case, the fingerprinting MUST occur before a downstream job fingerprints the same file to ensure the ORIGIN of the fingerprint is properly set.
The Multijob plugin works beautifully for that scenario. It also comes in handy if you want a single "parent" job to kick off multiple "child" jobs but still be able to execute each of the children manually, by themselves. This works by creating "phases", to which you add 1 to n jobs. The build only continues when the entire phase is done, so if a phase as multiple jobs they all must complete before the rest are executed. Naturally, it is configurable whether the build continues if there is a failure within the phase.
Jenkins recently announced first class support for workflow.
I believe the Workflow Plugin is now called the Pipeline Plugin and is the (current) preferred solution to the original question, inspired by the Build Flow Plugin. There is also a Getting Started Tutorial in GitHub.
Answers by jason & dnozay are good enough. But in case someone is looking for easy way just use JobFanIn plugin.
This diamond dependency build pipeline could be configured with
the DepBuilder plugin. DepBuilder is using its own domain
specific language, that would in this case look like:
_BUILD {
// define the maximum duration of the build (4 hours)
maxDuration: 04:00
}
// define the build order of the existing Jenkins jobs
Build -> Deploy
Deploy -> "Functional Tests" -> Promote
Deploy -> "Performance Tests" -> Promote
After building the project, the build visualization will be shown on the project dashboard page:
If any of the upstream jobs didn't succeed, the build will be automatically aborted. Abort behavior could be tweaked on a per job basis, for more info see the DepBuilder documentation.
I have problem in configure jenkins to auto build and deploy java project. I want to build and deploy once a day. However this build only there are changes during the day. IF there is no changes, I don't want jenkins to auto build and deploy.
Note: that I used gitlab as source code management.
Can you help me with this configuration.?
Firstly, you should configure the Git SCM section at the top of the job config page to point to your GitLab repository.
Then you can use the built-in "Poll SCM" build trigger — this will periodically check whether your source code repository has changed — and if it has, a build of this job will be started.
If the repository has not changed since the last build, then no build will be started.
You can configure this trigger — whether using a cron-like syntax, or a shortcut like #daily or #midnight — so that Jenkins only checks the source repository once a day.
Additionally, you should make sure that the "ignore post-commit hooks" option is enabled. If you're using webhooks from your Git repository to start Jenkins jobs whenever a commit occurs, this option prevents your once-per-day job from being triggered for every Git commit.
Here's the detail document: "Jenkins CI integration"
http://doc.gitlab.com/ee/integration/jenkins.html
Update to match your comment.
You don't want to trigger the Jenkins build via webhook. It's ok.
You want to check the code change 1 time a day.
Example on Linux, build at 6:00 AM if there's any code change.
Install
https://wiki.jenkins-ci.org/display/JENKINS/PostBuildScript+Plugin
https://wiki.jenkins-ci.org/display/JENKINS/Text-finder+Plugin
Build Triggers
Build periodically: 0 6 * * *
Execute shell
Like this
SINCE=`curl http://192.168.0.1:8080/job/MyJava/lastStableBuild/buildTimestamp?format=dd-MMM-yyyy`
cd /opt/code/myjava/
git log --pretty="%h - %s" --author=gitster --since=$SINCE --before=$SINCE --no-merges -- t/
Post Build actions
Post build task
Log text: commit
Operation: AND
Script: Your script to build your Java
Jenkins text finder
Also search the console output
Regular expression: Could not match
Unstable if found