I'm working with Jenkins that runs on a server.
I have a pipeline which is triggered by a user that pushes something on a GitHub repository.
It performs a script which makes sure the GitHub repository is deployed to the SAP Cloud Platform.
It uses the MTA Archive Builder for building the MTA application which creates a .mtar file.
The MTA application has a HTML5 module.
After building the .mtar file with the MTA Archive Builder, I deploy it using the NEO Java Web SDK (the library you need to perform neo deploy-mta).
"neo deploy-mta" is a command that performs the actual request for deploying your html5 application.
This works fine and the project is successfully deployed on the SAP Cloud Platform.
The problem is: if a user rapidly pushes 2 times on GitHub, my Jenkins pipeline is triggered twice and performs "neo deploy-mta" 2 times.
In a normal case the SAP Cloud platform should deploy 2 versions, but when I look it only deployed the first deployment request. So it skipped the second request for deployment.
My question is how can I make sure there are 2 versions deployed on the SAP Cloud Platform when 2 pushes happened?
The Jenkins instance is already waiting until there is no build running.
The problem was that the SAP Cloud Platform didn't deploy 2 versions when there were 2 requests for deployment.
The solution for this problem is to add the "--synchronous" parameter to the "neo deploy-mta" command. Now this script will wait until there is no deployment (for this application) running on the SAP Cloud Platform.
Most likely it happens because the SAP MTA deployer detects that you have another deploy in progress and thus stops the second deployment.
One version to go about it is to ensure from Jenkins that you don't run the second build until the first one has finished. You can do this with the help of a lock / semaphore like mechanism. There are several ways to do this via Jenkins plugins:
Lockable Resources
Exclusion Plugin
Build Blocker
Also look at How can I prevent two Jenkins projects/builds from running concurrently?.
Related
I'm using Jenkins for Continuous Integration tool with DevOps tools like JIRA, Confluence, Crowd, SonarQube, Hygieia, etc.
But the environments are changed to deploy microservices to PaaS.
So I got the issues to resolve below.
Deployment Monitoring
to view which application is deployed to what instance with which version.
Canary Deployment
deploy to 1 instance and then deploy to all instances(after manual approval or auto).
Deploy to Cloud Foundry
more specifically IBM Bluemix
So I examined Spinnaker but I found that the cloud driver for CF is no longer maintained.
https://github.com/spinnaker/clouddriver/pull/1749
Do you know another open-sourced CD tool?
take a look at concourse : https://concourse-ci.org/
Its open source, you can us it to deploy either application or cloud foundry. It's a central tool for DevOps. Basically you have pipelines that can trigger tasks (manually or automatically). You have some already created ressources (github connector, etc ...) but you can also create your own tasks. Its running docker containers as workers to execute tasks/jobs.
Best,
I find it relatively easy to integrate a CD server to any PaaS provider. You will have to either use a plugin or create your own integration.
My top two recommendations would be gitlab or Bamboo in that order.
Given your preference for Jira, you might prefer Bamboo as it has very good integration with the rest of that Atlassian tools but it is not open source.
I am in the process of migrating a Jenkins server from an internal resource to AWS EC2. I have completed the copying of all files in /var/lib/jenkins. However, when I start Jenkins it immediately wants to run builds, and they all fail because I need to make some changes. Devs don't like the tons of emails.
How do I start Jenkins with all jobs/builds disabled by default, so I can test and configure things before cutting over to the new server installation?
Here is a useful link! This groovy script needs to be placed in $JENKINS_HOME/init.groovy
For one of our application we are trying to automate the deployment process. We have end to end Continuous Integration implemented (Build/Test/Package/Report) for this application. Now we are trying to implement the automated deployment. This application needs to be deployed in 2000 servers and 50 clients under each server. Some of the components will be installed on the server and some of them will be installed on the client.
Tools used for CI: Jenkins, Github, msbuild, nunit, specflow, wix.
I have read the difference between continuous delivery and continuous deployment and understood that continuous delivery means the code/change is proven to go to live at any point of time and continuous deployment means the proven code/change will be automatically deployed to production server.
Most of the articles on the net explain how we can automate the deployments part of continues delivery / deployment to one of the servers (DEV/Staging/Preproduction/production). None of the article talks about deploying the application to large number of servers and clients.
Now my questions are
1) Deploying application to 2000+ servers and clients is part of continuous deployment or it should be handled out of CI/CD?
2) If it can be handled within CI/CD then how do I model this in the Jenkins delivery pipeline and trigger the deployment for all the servers from the CI tool?
3) Is there any tool which I can integrate with the CI tool to automate the deployment?
Thanks
I'd keep separate these 2 aspects:
deployment on a large number of servers (it doesn't matter if the artifacts to deploy come from CI or not)
hooking up such deployment into CI to perform CD
For #1 - I'm sure there are profesional IT tools out there that could be used. Check with your IT department. If deployment does't require superuser permissions or if you have such privileges (and knowledge) you could also write your own custom deployment/management tools.
For #2 - CD doesn't really specify if you should deploy to a single server, a certain percentage of your production servers or all of them). Your choice should be based on what makes sense or is more appropriate for your particular context. How exactly it's done if you decide to go that way? It really depends on #1 - you just need to trigger the process from your CI. Should be a breeze with a custom deployment tool :)
The key requirement (IMHO) in Continuous Deployment is the process orchestration, jenkins isn't ideal tool for this, but you can write an own groovy script wrapper or to invoke the jobs remotely form an another orchestration tool. Another issue in jenkins, at least for me, is the difficulty to track the progress.
I would model it as the following:
Divide the deployment process to logical levels, e.g Data centers->Application->Pools and create a wrapper for each level. It will allow you to see the progress at highest level in the main wrapper and drill down in case of need as you wish
Every wrapper should finish as SUCCESS only if ALL downstream jobs were SUCCESSFUL, otherwise it should be UNSTABLE or FAILURE . In this case there is no chance that you will miss something at the low level jobs.
1 job per 1 per product/application/package
1 job to control the single sequence run. For example I would use mcollective to run the installation job sequentially/parallel on the selected servers
1 wrapper job for every logical level
I would use:
mcollective - as mentinoed above
foreman to query the puppet to select the servers list for every sequence run
The package/application/artifact installation on the server I would prefer to do with a native OS software, e.g yum on the linuxserves. It will allow you to rely on their mechanism of installation verification
I'm sure that I missed something but I hope it will give you an acceptable start point
I would like to raise from scratch a build server for .NET applications using Jenkins, please note that i'm new to Jenkins CI.
Several Questions:
1) How should I decide on the build server specs? except for the OS which would be windows server 2012, how should I decide on the RAM and the CPU and HD space?
2) Should the Jenkins sit in the build machine or not, what is the recommended approach? I understood that the build server should be isolated from the Jenkins master
3) How do I decide on the Master/Slave approach, when should I use only Master and when should I use master and slave or slaves?
4) How would you recommend me to run the build and deployment tasks in the Jenkins CI, using NAnt/Python or any other scripting language ?
10x, and sorry for the igonrance :)
Responding to each in turn:
You can run Jenkins as a windows service (instructions here) and the machine can be a VM, so it doesn't have to be huge.
a) RAM and CPU: I'll put these together and will depend on how many jobs you plan to have running at the same time. The default number of build executors is 3 but can be increased as a global config change.
b) HDD: This depends on how many jobs you plan to have. Jenkins will checkout the source code (as well as the compiled output) to its home directory on a per job basis. This can get big. I would also recommend using the ThinBackup plugin to backup the Jenkins configuration.
Jenkins is the build machine. A vanilla installation of Jenkins is the master. In my experience you will not need a separate slave machine unless you're needing to do native builds on other platforms or have LOTS and LOTS of jobs. I've seen single masters running happily with hundreds of jobs.
Further to 2. above, suggest you start with a master and set up a slave later if you really need one.
As you have stated you are building .NET applications, you can simply install the MSBuild plugin which should serve you well. Builds for .NET applications in Jenkins are Freestyle builds so you will be using Windows Batch build steps often as well. This also is a great blog on Jenkins in a .NET environment.
Using a single server that is only contains one Jenkins building for dev, test, etc.
Using separate Jenkins on each dev, test servers to build and run tests.
Edit ;
this is an explanation of step by step our deployment and release model
Our server side developers develop and commit/push their code to github.
CI server that Jenkins is located in poll SCM and fetch changes than build. (within CI server), run unit tests.
After building process and deploying artifacts to repository server (artifactory server)
Then CI server starts to deploy latest successful build into Development Server.
then client mobile developers can develop on latest successful snapshot build of server side.
These are our standard deployment process.
By the way,
We are also doing test deployment to test server via CI server with another different job on Jenkins (same CI server) but, this is handling/triggering by manual.
Preproduction and production transitions are done by manual also. (preproduction and production are different servers of course)
Questions;
Integration tests should be run on test server. How can i figure it out by building system on remote CI server instead of building system on the same machine (test server) ?
As a further step, what would the best option be to construct a Continuous Delivery system. ?
Thanks
A good approach is to have a single CI system that builds the system continuously as development makes changes. This build will on each build run all the unit tests as well and result in some kind of package that can be deployed. That can be further connected with automation that deploys and runs other tests or it can be used by e.g. testers to further test the system.
Depending on your release model and branching strategy as well as type of system/product this basic setup can be adjusted to fit your needs.
If you want more details please explain what you build and how you release/deploy.