My development setup is such that for every svn checkin code is built,unit tested, packaged and published in Artifactory. Now I want to automate my deployment process & run integration(Selenium) test as part of this process. I am thinking of using Puppet to managed the deployment
Is puppet the correct tool for this
What is the process I should use to trigger puppet master to initiate a fresh installation on agents, I couldn't find any Jenkins plugin that would actually trigger puppet. One option is to call
puppet apply ...
as a Jenkins post build task
Any suggestions welcome, thank you.
Have a look at this Selenium Jenkins article from Saucelabs, a service that automates cross-browser testing. Though they are a vendor with a service to sell, the article covers how to do Selenium testing yourself with Jenkins. It also exposes common pain points you are likely to run into with this approach.
A Puppet master doesn't serve the function of orchestrating client convergences. Take a look at Mcollective. This is a tool that will allow you to trigger puppet runs on target systems from a Jenkins agent via script commands.
Some Mcollective getting started material:
http://www.slideshare.net/PuppetLabs/presentation-16281121
http://puppetlabs.com/mcollective
Related
We are starting to develop CI workflow for our systems in my company.
Currently we just making few basic tasks like build, tests, and upload to Nexus.
The tech stack is a Java project which build in Gradle and Jenkins makes our build.
Currently i'm working with some basic Groovy script to make what we need, but each time i'm copy and paste my updated code to Jenkins and running the job from Jenkins UI to see the results, and to me it seems like not a very good approach for developing such automation code.
My question is, what is the best practice to build and run Jenkins jobs?
Is it possible to run it straight from Intellij ?
Do we need to create a Jenkins project which should be saved as a repository and then deploy it to Jenkins machine?
Do we need to use some Intellij plugins in order to work with Jenkins?
More best practices are welcome :)
Jenkins has an API - so you can do whatever you want!
But in general, for small to medium teams it's better to use Jenkinsfile and let Jenkins pull code changes (or pull-requests) from SCM and trigger builds. You can also configure hooks to trigger builds if your SCM supports this (Github & bitbucket supports this).
If you are eventually pushing your artifacts to a docker image, I would highly recommend docker multi-stage builds.
If you are completely new to CI/CD stuff - Atlassian has a lot of good resources https://www.atlassian.com/continuous-delivery/principles/continuous-integration-vs-delivery-vs-deployment
As part of QA pipeline(in Jenkins), goal is to automate provisioning and configuration of a VM to run the QA tests.
Jenkins pipeline can trigger Terraform code to automate provisioning of VM and ansible code for configuration of a VM, but, issues like rollback, error handling is not easy unless we use some vendor specific template like AzureResourceManager template.
So, with Jenkins pipeline,What should be the best approach to provision and configure a VM in Azure cloud? we write pipeline scripts for jenkins pipeline...
As the goal is to know the best approach to automate provisioning and configuration of a VM to run the QA tests so I would go with simple jenkins pipeline script by leveraging Azure CLI commands in it.
To be precise, I would just add an Azure service principal to Jenkins credential. And then write simple Jenkins pipeline script by having 'withCredentials([azureServicePrincipal('SERVICEPRINCIPALCREDENTIALID')])' and then by using 'sh' part to have Azure CLI command to provision and configure VM. For illustration related to this you may refer https://learn.microsoft.com/en-us/azure/jenkins/execute-cli-jenkins-pipeline#add-azure-service-principal-to-jenkins-credential.
Regarding the issues like rollback and error handling when going with the approach of having Jenkins pipeline that triggers Ansible code (with or without using ARM templates) that can automate provisioning and configuration of a VM to run the QA tests, (you might already be aware of this but wanted to let you know that) for certain types of tasks you may write custom modules that can leverage the error handling functionality and in few scenarios you may leverage 'failed_when' option. Also you may leverage 'blocks' functionality by which you can define a set of tasks to be executed in the rescue: section. This 'blocks' functionality specially should help in enabling us to get the things rolled back.
Hope this helps!! :)
I am looking for implementation of CI/CD in to my current project here is what i think will work.
Environment consists of
- Jenkins
- git
- docker
- gradle
- Linux servers
- Sonar
- Ansible.
Each tool will be used as following.
Git:- Developers will push there code to this CVS.
Jenkins:- On detecting Check-in Jenkins will trigger a build and will deploy to one of the server.
Sonar:- will be used for code coverage and will check the code before building the same through Jenkins.
ansible:- ansible will be used to quickly prepare added nodes so that code can be deployed to them.
Docker in case if we need fresh test environments every time we can use docker+ ansible combo for doing the things.
Flow of work will be
User run unit test cases on his machine and commits the code to the server.
Jenkins will pull the code from git and will run sonar on the same and will generate reports.
jenkins will create build and will deploy the same on dev server.
A jenkins job will run and will perform the integration testing on the dev server
Any other automated tests can be run.
Finally builds pushed to next server using Jenkins.
I will use shell commands inside Jenkins to push compiled code from one server to another.
In my this scenario can some one answer me following.
Where will sonar get fit and how to use the same?
I see there are CD tools, cant i push compiled code to the servers using shell scripts written inside the Jenkins jobs to automatically deploy the things? What extra benefits a CD tool provides
Is is wise to create fresh test environment or we can keep using the old one again and again?
Will this complete CI/CD?
can someone share there implementation
You say you plan to use Git. I'll outline a scenario using Git on GitHub
Developers push code changes here as pull requests
The SonarQube GitHub Plugin kicks off an initial analysis of only the code changed in the PR looking for the introduction of new issues (note that coverage and duplications are not included in this check)
Once the PR is merged, Jenkins (in one job or several, depending on your needs)
builds
fires integration tests & any other automated tests
runs SonarQube Scan. Note that this comes last to include integration test results.
pushes build to next server
Note that the ability to break the build when the project doesn't pass the SonarQube Quality Gate you've set up may be desirable in your situation. Unfortunately, it's not available in the current server version: 5.2. It is available in 5.1, and is should return soon.
Recently, in our company, we decided to use Ansible for deployment and continuous integration. But when I started using Ansible I didn't find modules for building Java projects with Maven, or modules for running JUnit tests, or JMeter tests.
So, I'm in a doubtful state: it may be I'm using Ansible in a wrong way.
When I looked at Jenkins, it can do things like build, run tests, deploy. The missing thing in Hudson is creating/deleting an instance in cloud environments like AWS.
So, in general, for what purposes do we need to use Ansible/Jenkins? For CI do I need to use a combination of Ansible and Jenkins?
Please throw some light on correct usage of Ansible.
First, Jenkins and Hudson are basically the same project. I'll refer to it as Jenkins below. See How to choose between Hudson and Jenkins?, Hudson vs Jenkins in 2012, and What is the most notable difference between Jenkins and Hudson from a user perpective? for more.
Second, Ansible isn't meant to be a continuous integration engine. It (generally) doesn't poll git repos and run builds that fail in a sane way.
When can I simply use Jenkins?
If your machine environment and deployment process is very straightforward (such as Heroku or iron that is configured outside of your team), Jenkins may be enough. You can write a custom script that does a deploy as the final build step (or a chained step).
When can I simply use Ansible?
If you only need to "deploy" without needing to build/test, Ansible might be enough. For instance, you can run a deploy from the commandline or using Ansible Tower. This is great for small projects, static sites, etc.
How do they work together?
A good combination is to use Jenkins to build, test, and save artifacts. Add a step to call Ansible or Ansible Tower to handle the actual deployment process. That allows Ansible to handle machine configuration and lets Jenkins handle the CI process.
What are the alternatives to Jenkins?
I strongly recommend Thoughtworks Go (not to be confused with Go the language) instead of Jenkins. Others include CruiseControl, TravisCI, and Integrity.
Ansible is just a "glorified SSH loop".
CI is not only the software running, but the whole process of how success and failure is handled, who gets notification, and how the change is merged into the target version control.
If we only focus on the software, CI is a reactive scheduler triggered by code changes, and triggering typical build-validate-release-deploy sequence of "steps".
So in respect of software, Ansible without additional "sugaring" is just a toolkit to run things, which can be those very steps, but it is not CI.
The Ansible (without tower) totally lacks this reactive nature.
If you want to marry Ansible with CI, you can.
Ansible tower is a very Ansible oriented scheduler, but if you need CI software, I think you not necessarily need it. Any CI app capable of running shell script would be capable to launch Ansible playbooks.
Yet unlike Ansible tower - CI tools know to display test reports of all test frameworks, trigger notifications, etc.
Ansible tower can make sense in a complex environment with lots of groups touching Ansible code... The truth is I haven't seen a single real reason to pay for it. But if a manager liked the web interface nothing can stand "but others use it" logic.
I suspect the concept of Ansible tower was in response to puppet enterprise.
:)
I'm running Jenkins on one server and want to use chef and automatically install a snapshot (including runtime artifacts etc) on a separate server.
Currently Jenkins will use ssh to invoke chef on the seperate machine. Is there a better way?
Maven is also involved in this.
I've found that majority of "Deploy" type plugins are lacking in customization. We use "Execute" (bash or batch) build steps to trigger deployment scripts on remote machines (written in house, be they Puppet, Chef, or plain bash/batch).
The correlation between builds and deployments is achieved through "Promotions" and explained in detail here:
How to promote a specific build number from another job in Jenkins?