I want to build a new pipeline for my project which includes production deployment as a stage. A sample can be seen in the below picture.
Once the pipeline has passed all the quality gates, it will be deployed to dev environment automatically, and functional test will run in the next stage. There is a manual approval required at this stage, once approved the next step will be deploy to prod/preprod. I am able to achieve the goal till this point. I am stuck at following step:
Can we keep this build forever, or for a specified duration, say, 1 or 2 months, and continue from the same stage when required? For example, the git master branch is getting updated very frequently, and the builds are getting triggered automatically, all builds should reach this stage and wait. Only the build having approval should be able to proceed forward.
There can be one other way; I can have a separate job for deploy to preprod/prod. But, I want to know if this is possible.
Regarding - Can we keep this build forever, or for a specified duration, say, 1 or 2 months...
You could achive it partially - build could stop at manual step for a period of time. But remember - while job is running it is using executor from agent. Once you use all executors you can't start another build.
... and continue from the same stage when required? - It's not possible.
Related
I am relatively new to Jenkins.
I created a declarative pipeline in Jenkins where users are asked to enter their branch name and then Jenkins builds that specific branch (for example, origin/mybranch).
This allows me to run a quick set of tests for specific branches.
The developers can run the pipeline multiple times and today I block multiple such pipelines from running simultaneously because if they do, one overwrites the other.
This happens because the first pipeline writes to c:\Jenkins\workspace\QuickBuild and when another such job run is writes to that exact same folder, killing the original run.
Blocking was the solution I found to prevent this but I would like it so that when one run is finishing up (using less than 8 cores) the next run in queue will already start running with whatever cores are freed up.
I would have though this would be a basic concept of Jenkins.
Am I missing something? Am I doing it wrong?
Following MaratC and Zett42's suggestions, I ended up adding this to my script:
agent
{
node {
customWorkspace "${params.Branch}"
}
}
This causes Jenkins to create each build in a different folder and they don't step on each others' toes.
The only downside is that you can't build the same branch simultaneously but that's a corner case.
Also, I could add a random number to the workspace to enable this as well.
I have a (declarative) Jenkins Pipeline that is doing builds and tests continuously. When successful, the application should be deployed on particular test environments once a day, based on some schedule.
For instance, if the build was successful, and current time is
between 11:00 and 14:00, deploy to TestA, but just once a day;
between 14:00 and 18:00 deploy to TestB, but also just once a day;
etc.
I would be able to do the time slot handling in some groovy code, but I'm not sure how to "remember" whether there already was a deployment in this time period as of today. Of course, it is useless to store that information in the workspace, since later builds may be executed somewhere else.
So what options do I possibly have?
Store some marker file in a shared network location, and check this file and its timestamp in later builds to decide whether a deploy is required. This would probably work, but introduces dependency to external resources.
Can I somehow "mark" the Jenkins build when doing deployment, so that following builds can iterate through previous build(s) and search for such marker? Like archiving some small text file with the build?
Alternatively, is there any plugin that supports this scenario?
Or any completely different idea?
This seems to be a frequent scenario in CD pipelines, so I wonder how this is done in the wild... Thanks for any hints!
You should have the build and deploy stages on separate pipelines. That way the build can occur independently, and the deployment can be triggered by the timer to run exactly once per day.
In this case you'd want the build pipeline to archive its artifacts, so that the deploy pipeline can always deploy a successful build. The Copy Artifacts plugin can be used to get the build pipeline's artifacts into the deploy pipeline's workspace.
I know this Forum is not to provide strategy's.
Using Jenkins I have set up CI and CD to my Dev,QA and Staging environments. I am stuck up with Rollback strategy for all my environments.
1- What happens if my build fails in Dev
2- What happens if my build fails in QA and passed in Dev.
3- What happens if my build fails in Staging and passed in Dev and QA.
How should I roll back and get things done considering DB in not in place. I have created sample workflow but not sure its an right process.
Generally you can achieve this in 2 ways:
Setting up some sort of release management tool that tracks every execution of your pipeline and snapshots the variables, artifacts, etc... that was used on that exact execution, then you can just run an earlier release of it (check tools like octopus deploy)
If you are using a branch strategy with tags you can parameterize your jobs, passing the tag you wanna build, and build the "earlier tag" if something fails. Check the rebuild option for older job executions.
We run an environment were multiple features are divided into branches. Those branches are configured to a channel in Octopus Deploy.
I would like to see if there is a way to check in a build definition if another build is queued for a branch.
For context - I have a Octopus Deploy step as the last step in our build definition. To deploy the app - it takes approximately 10 minutes. Our build process is also about 10 minutes (it is a big application).
When a team working on one feature checks in - we end up with 2 and 3 builds queued and waiting for each check-in and build to complete.
What I would like to do is have the deploy task run only if there is NOT another build queued for the same branch. This would ensure we don't waste 30 minutes of unnecessary deployments and only deploy the latest code.
What type of build do you use? XAML build or Vnext build. It seems that you use the CI trigger in your build definition, so it will trigger a build each time you do a check in.
VNext build:
You could select the Batch changes checkbox. According to this document,
If you select this option, when a build is running, the system waits until the build is completed and then queues another build of all changes that have not yet been built.
This will combin the changes into one build when you have a lot build queued.
XAML build:
In the build definition, you could use the Rolling builds trigger. This has the same function like the Batch changes mentioned above.
Note:
You also could use this REST API to get if there're builds of a build definition that are queued.
Http method: Get
http://servername:8080/tfs/DefaultCollection/TeamprojectName/_apis/build/builds?definitions=10&statusFilter=notStarted&api-version=2.0
It ended up being easier than I thought - but not exactly what I tried to do in the question.
The Octopus Create Release Task for TFS and VSTS has an option to "Show Deployment Progress". Checking this causes the build definition to wait - periodically receiving feedback from Octopus through the Octo.exe tool.
The deployment for this is actually a wrapper for several deployments (15 separate projects) - so it would take a long time.
Unchecking this option causes TFS to not wait - but send the create release / deploy command to Octopus.
Now this didn't solve my initial question - but Tingting0929-MFST did help me out in that I explored using the TFS Rest API. I wrote a PS script the effectively did what I asked - not deploy if there was a another build queued up for the same branch. I got it working - but it introduced a different problem in that the release notes from TFS and associated changesets / work items were getting swallowed in between releases.
In other words if three checkins came in fairly close separation, the last one would get deployed. Octopus would show only the release notes for last deployment and not the first two.
For those using Octopus Deploy in a feature branch set-up (one project deploying to multiple channels) - this is an option to get you the Continuous Deployment for each channel (branch).
We've setup a Jenkins build pipeline that uses Maven to build a large project, including stages for update, compile, unit test and deploy (to Nexus). The "deploy to Nexus" stage has concurrency 1 to ensure that no more than one build is in this stage at any point in time. However, this setting does not prevent older builds from entering that stage after a later build has finished it.
For instance, if build #2 is startet after build #1 and hits a fast node, it may outrun build #1 and enter the "deploy to Nexus" stage first. Build #1 cannot enter this stage at the same time, so it waits until build #2 is finished; but then build #1 enters this stage and thus overrides the Maven artifacts deployed by the later build, which is not what you want.
There must be a way to avoid this, i.e. to prevent older builds from entering a stage that was already executed successfully by later builds. I just could not find a solution for this problem... Any thoughts?
The upcoming milestone step is supposed to solve this.
Send the current build number of the job and add a simple condition to test it with the latest successful build number of the job using the following Jenkins URL:
http://JenkinsMaster:Port/job/MyJob/lastSuccessfulBuild/buildNumber
If the current build number is smaller than the last successful one then skip the upload.
Good Luck!