I have a (declarative) Jenkins Pipeline that is doing builds and tests continuously. When successful, the application should be deployed on particular test environments once a day, based on some schedule.
For instance, if the build was successful, and current time is
between 11:00 and 14:00, deploy to TestA, but just once a day;
between 14:00 and 18:00 deploy to TestB, but also just once a day;
etc.
I would be able to do the time slot handling in some groovy code, but I'm not sure how to "remember" whether there already was a deployment in this time period as of today. Of course, it is useless to store that information in the workspace, since later builds may be executed somewhere else.
So what options do I possibly have?
Store some marker file in a shared network location, and check this file and its timestamp in later builds to decide whether a deploy is required. This would probably work, but introduces dependency to external resources.
Can I somehow "mark" the Jenkins build when doing deployment, so that following builds can iterate through previous build(s) and search for such marker? Like archiving some small text file with the build?
Alternatively, is there any plugin that supports this scenario?
Or any completely different idea?
This seems to be a frequent scenario in CD pipelines, so I wonder how this is done in the wild... Thanks for any hints!
You should have the build and deploy stages on separate pipelines. That way the build can occur independently, and the deployment can be triggered by the timer to run exactly once per day.
In this case you'd want the build pipeline to archive its artifacts, so that the deploy pipeline can always deploy a successful build. The Copy Artifacts plugin can be used to get the build pipeline's artifacts into the deploy pipeline's workspace.
Related
I want to have one Jenkins job control the build number of another job but without the inconvenience of reloading the entire project configuration from disk. I have seen that it's easily possible to directly update the nextBuildNumber file of the target job (I can do this as a build step of Job A) but this does not take effect immediately. Restarting Jenkins or even reloading the Jenkins configs from disk takes way too long and can only be done when there are no builds in progress.
I have tried the groovy script mentioned in the below post by running it from the Manage Jenkins > Script Console. The same post also suggests the script can be saved as a file and run from the CLI. Can it be run from a build step?
I want Job A to determine Job B's next build number and set it so that Job B can run (later in the same day) with the desired build number.
https://stackoverflow.com/a/20077362/4306857
Perhaps I should clarify. I'm not familiar with Groovy so I'm looking at the various build step options like "Execute Windows batch command" which I have a lot of experience with. I can see an "Invoke Gradle script" option so I was wondering if there may be a plugin that can run Groovy scripts perhaps?
The reason this requirement has arisen is that we are compiling a product for two different platforms. We want to compile the codebase almost simultaneously for both platforms with two jobs (A & B) which will both update the JIRA cases included in the builds. We feel it will be less confusing to have both these jobs running with the same build number so that when we talk about a particular issue being addressed in build #75, say, we won't have to qualify that by stating the full job name. If JOB-A build #75 and JOB-B build #75 are both compiled on the same day from the same codebase we can test and compare the results of both builds with far less confusion than if the build numbers go out of sync.
Obviously, in the short term we will use the Set Next Build Number plugin to manually keep the build numbers in step but we want to automate this if possible.
Depends on whether or not you are using Version Number plugin:
[ X ] Create a formatted version number
Build Display Name [ X ] Use the formatted version number for build display name.
Assuming you are NOT, this groovy script will do:
def NextNumber=42
job=Jenkins.instance.getItemByFullName('path/to/jobName')
job.nextBuildNumber = NextNumber
job.save();
You will need groovy plugin for that. Place that in an "Execute system Groovy script" step. Make sure to choose system groovy. That will execute on the master, where the job config and metadata is stored so you have access to the Jenkins internals and data.
I'd suggest you should really be using the above options rather than relying on "keeping both jobs in sync" via a script or manually. You can then pass the label to be used from the first job as a parameter to the second job. That would also require Parameterized Trigger as well as Version Number plugins.
You can even use ${BUILD_DATE_FORMATTED} or ${BUILD_TIMESTAMP}, etc.
Postdate: thinking about the problemspace from a different perspective, that of running 2+ builds on different platforms (simultaneously), there's a plugin for that: Matrix project. You can run it as a freeatyle job on multiple nodes or is excellently described as Matrix building in scripted pipeline. Not sure how that would tie in to JIRA.
I've been asked to create a CI pipeline for a project at my work, I'm creating a load test with JMeter and Taurus so I plan to integrate it with Jenkins to build all the pipeline. I'm just starting on this field and a question that came to my mind is:
What happens to all the data created by the Load Test? does it goes to the deploy phase or it gets deleted once the test is done, should I clean after the tests end?
The data is being kept in the Jenkins workspace and by default it will be kept in the file system forever.
If you decide to publish the artifacts they will be available at Jenkins build dashboard via the web interface.
You might also be interested in Jenkins Performance Plugin which allows plotting performance trend charts and conditionally marking builds as unstable or failed depending on pass/fail thresholds
Example configuration can be found in the How to Run a Taurus Test through Jenkins Pipelines article
I am not completely familiar with your setup but as far as I can see from a quick research, JMeter does the same as every other testing framework and generates HTML reports. Jenkins wont delete them, unless you explicitly delete them (rm file.html) or call cleanWs (clean workspace). If the job is deleted so are the files.
So the test result file should still be present in the deploy phase. You can use a plugin to collect the result. Or just archive it. Or do whatever fits your workflow.
There is generally no need to clean it up (you usually configure Jenkins to delete old builds which takes care of that)
All my Jenkins jobs are triggered both by a Github webhook, but also via a scheduled build one per week. The build process is heavily cached to make the webhook CI builds finish quickly.
I would like to add a line to my build script which wipes the cache during the weekly scheduled build, and make it build from scratch. Is there a variable in the build script to identify if a build was triggered by a webhook or schedule?
Maybe the envInject plugin will give you what you need?
This plugin also exposes the cause of the current build as an
environment variable. A build can be triggered by multiple causes at
the same time e.g. an SCM Change could have occurred at the same time
as a user triggers the build manually.
The build cause is exposed as a comma separated list:
BUILD_CAUSE=USERIDCAUSE, SCMTRIGGER, UPSTREAMTRIGGER, MANUALTRIGGER
In addition, each cause is exposed as a single envvariable too:
BUILD_CAUSE_USERIDCAUSE=true
BUILD_CAUSE_SCMTRIGGER=true
BUILD_CAUSE_UPSTREAMTRIGGER=true
BUILD_CAUSE_MANUALTRIGGER=true
I was hoping someone could help with ideas in Jenkins for sharing a build increment across multibranch pipelines on multiple build machines. Looking through similar questions, I didn't see anything better than timestamp instead of build number but it's not quite what we were looking for.
I am using Jenkinsfiles to define multiple pipelines, then 'multibranch' to instantiate them across all branches. Currently these just call shared freestyle jobs to implement the stages. Every pipeline invokes the same job to do a build, across a pool of build machines, so we can just use that build number to increment the version. For example I have v1.2 being worked on by several branches, each having a CI, Nightly, and Release pipelines. They all invoke the same build sub-job so you might have CI/BranchA run sub-job #503 on buildVM1 so will have version string 1.2.503, then Nightly/BranchB will run sub-job #504 on buildVM2 so will have version string 1.2.504. This works great as long as I am invoking the same build sub-job.
The next step is to implement the builds as part of the Jenkinsfile pipeline, but then I lose my convenient build increment. How can I define a shared build increment for this component across all branches defining the same major.minor version?
Timestamp is a bit unwieldy since the multiple branches/teams/build machines means I need to go to seconds. Do I really need versions like 1.2.20180118165007? There's got to be a better way.
How else can I manage this?
I know this Forum is not to provide strategy's.
Using Jenkins I have set up CI and CD to my Dev,QA and Staging environments. I am stuck up with Rollback strategy for all my environments.
1- What happens if my build fails in Dev
2- What happens if my build fails in QA and passed in Dev.
3- What happens if my build fails in Staging and passed in Dev and QA.
How should I roll back and get things done considering DB in not in place. I have created sample workflow but not sure its an right process.
Generally you can achieve this in 2 ways:
Setting up some sort of release management tool that tracks every execution of your pipeline and snapshots the variables, artifacts, etc... that was used on that exact execution, then you can just run an earlier release of it (check tools like octopus deploy)
If you are using a branch strategy with tags you can parameterize your jobs, passing the tag you wanna build, and build the "earlier tag" if something fails. Check the rebuild option for older job executions.