I feel it's a little crazy I couldn't find anything along these lines, especially as it's an incredibly simple requirement: Is there a way you can deploy from Jenkins using SSH/SCP, yet write only one instance of a transfer-set/exec script?
As it stands, deploying to servers is kind of INSANE in that I need to create a new "Deploy to SSH" task, choose a different server from the drop down and then copy/past all transfer-sets and execs from the previous entry. Then do it again. And again. And again.
There must be a better way?
This may not be a short-term immediate solution for your question---
(On long run this can be used)
Your requirement seems to me like you need a configuration management equipment. You could use Chef, Puppet or Ansible. And automation of this deployment can be done using Jenkins CI.
One example of how to deploy an application on jboss using Ansible -
Deploy a hello world application
jboss: src=/tmp/hello-1.0-SNAPSHOT.war deployment=hello.war state=present
Of course, this will require installation of Ansible and little bit of initial configuration. Ansible is simplest of all deployment mechanisms.
Check this for more details - http://docs.ansible.com/ansible/intro.html
Related
The Jenkins landscape is vast and new progress is difficult to keep track especially if you are not a regular DevOps.
I am currently in process of setting up a Jenkins CI system from scratch. I am looking for the best possible ways to get the Jenkins instance up and running. I have looked at options such as running from the JAR, setting it up a service, docker, blue ocean, etc.
I was wondering if you can please share your experience if there is a pre-configured setup or a scalable Jenkins solution already available in the market which is ready to be configured/deployed.
One of the key tenant on this Jenkins instance would be test automation guys running their Selenium tests (or I am ideally looking at Windows server installation although CentOS is an option) and would like to make it working for them as easy as possible.
I'm a Jenkins admin. In my company I've set up Jenkins on our Kubernetes cluster using the Helm chart with a custom docker image preloaded with plugins (you don't want to rely on the plugin update site during startup). All configuration is done with the Configuration as Code Plugin. We're using the Kubernetes plugin to do horizontally scaling. No builds are allowed on the build controller, everything is done within agents, which is custom docker images inspired by these images. and we don't allow no builds to run on the build controller. This works very well, and I'm very happy with the setup. There is also a Jenkins Kubernetes Operator which looks promising, but I havent tried it myself.
If you're not on Kubernetes, you can take a look at the Jenkins Evergreen project.
PS: The Blue Ocean project is dead, but the folks over at Cloudbees are currently in the process of overhauling the UX. They just released a weekly version where they got rid of all tables so the design is slowly becoming more and more responsive, and also a new set of icons is also coming up.
Maybe the nearest you can get to a pre-configurated Jenkins Instance is using the Docker Image (https://hub.docker.com/r/jenkins/jenkins). But also with the docker image, you have to selected plugins and so on. Maybe you want to raise an issue as purposal in the Jenkins Docker repository to make it possible to pre-configure Jenkins (Github Repo: https://github.com/jenkinsci/docker/issues)?
I have recently started to mess about with Jenkins and am unsure how to deploy my web app to a basic server. I've gotten into the Pipeline (https://jenkins.io/doc/book/pipeline/) and it seems like a fantastic way to work.
Where I'm a bit stuck is in two spots:
Once my repo is in my workspace within Jenkins, how do I prep it so I am only deploying the files necessary for the application? For example, I don't need my src/ directory or my Vagrantfile when I'm deploying things.
How do I deploy my app to the server? I see examples all over the place, but I am getting a bit lost since there seems to be so many ways to do this. I'm assuming scp or something like that...?
To build off of #2, is there a way to deploy web apps as transactions (in one shot) rather than file-by-file?
Please let me know if I can provide any information for potential answers!
I can't speak to your specific use case but a common way to do this is the build-and-deploy model, where you will have 2 Jenkins jobs. The "build" job will check out from source, run build commands such as maven or make, and lastly will "archive" the build artifacts. The latter is an option under the 'post-build actions' tab at the bottom.
In the "deploy" job, you will grab the artifacts of your choice. You can fetch a single file, all of them, and everything in between. This requires use of the 'Copy Artifact' plug-in and it allows you to copy files generated by other jobs. Now you can run your usual deploy script in the 'Execute Command' box. Most command line paradigms are supported out of the box such as setting environment variables.
The instructions above assume that you want to run your application off of a host that you've provisioned as a Jenkins slave.
Use artifacts as mentioned by Paul Back, or a 3rd party artifactory server as in video
This is always tricky and error-prone. Why not spin up a fresh server with new release (humanly verified once)
Jenkins & Ansible is the answer here. This is how I deploy to production, since I am in no need to use anything like Docker (too many issues with particular app) so have to run the app natively. Quick example would be
You monitor a specific branch in gitlab / github or whatever else and then call a webhook on push / merge etc on that branch, at this point you deal with anything you need to do by running a playbook on the jenkins job that monitors that branch (jenkins).
in my case jenkins and ansible run on the same server. Jenkins runs the ansible playbook that does whatever I need to do.
for example with ansible, I copy certain files that need to be there, run configs / change filenames etc. setup nginx, run composer,
you get the point.
I wanted to build a Jenkins server which would run test of my puppet code on Vagrant. The issue I found is that the we run our server as VMs already, either in vmWare or AWS and Vagrant will not work as another virtualisation.
Does anyone have an idea how can I create a test platform for my puppet code. What I want to test the deployment of manifest on the nodes them self i.e. If I deploy a class web server or make changes to it I would like to check if it affects/breaks deployment of other classes.
The idea would be to iterate over all the classes/roles and see if the deployments are passing. I would like to make it automatic and independent of our engineers. At the moment we are running manual test with vagrant up however there are too many roles to do that by hand.
Any ideas how can I tackle this?
You can use either Docker or AWS provider for Vagrant.
In case of AWS provider you need to set-up RSync to get your environment into newly launched instance.
If your Vagrant scripts are robust, you can use the same script for both local deployment on your workstation and AWS/Docker deployment on CI server.
There are drawbacks to doing these techniques, in case of Docker you are limited to the same kernel that Jenkins server is running, in case of AWS you will incur additional costs. However, for AWS your don't need to allocate as much resources for your Jenkins server, so you might even save money this way because you will be using paying for extra VMs only when you are running you tests. Just make sure you will shut them down after you done.
Is there any special reason why you want to use vagrant? I'm not sure if you are setting up your production environment with vagrant or not.
In case you are not bound to vagrant, I would recommend you to think about using a docker image to prepare a lightweight environment to run your setups and verifications in.
When doing your tests, spin up a container from your image that contains your puppet distribution and run your setups/tests inside. If you have special kernel requirements, use a separate jenkins slave/agent machine rather than executing jobs on the jenkins master.
If you are not sure how to get started using jenkins with docker, have a look into the examples section of the Jenkins Documentation. The provided examples are showing the declarative pipeline syntax thats still a bit new. Also consider the collapsed Toggle Scripted Pipeline Sections which show the groovy pipeline scripts that are alot more forgiving for jenkins pipeline beginners.
Those should be quite good pointers to get started with running+testing your puppet scripts inside docker. For building and using a docker image there should be more than enough tutorials out there.
Let me know if this was a hint in the right direction or if I mistinterpreted your question.
I want to use Amazon EC2 plugin for setting up autoscaled Slaves.
We aim to script everything using Chef and so far I haven't found anything for this Jenkins plugin. I want to write a cookbook of my own but am wondering what is the best way to do it?
Generally management of the build machine would be done through the EC2 plugin itself, it already installs the Jenkins remote jar for you, so all you need to do beyond that is make sure Java is installed.
There are two methods to use Amazon EC2 plugin and Chef together:
Run Chef to do provisioning on each slave launch or build start
Build pre-baked slave images using Chef and something like Packer and provide them to Jenkins Amazon EC2 plugin
Cons of the first approach:
May take a lot of time depending on what software you are installing with Chef. So it would give a latency for build start and extra bill for machine time.
You can't always get the same build environment you have last time. May lead to heisenbugs and hard troubleshooting.
The second approach is known as Immutable Server. It has its cons too:
Gives you an extra bill for AMI storage.
Less flexible — you can't just fix some version numbers or add requirements for some new software and start a new Jenkins build. You have to rebuild your slave images first. And if you need even slightly different environments you have to build and keep several pre-baked images.
I myself use the second approach right now. You could check source code here. Specifically, configuration of Amazon EC2 plugin with Chef is done here.
Recently, in our company, we decided to use Ansible for deployment and continuous integration. But when I started using Ansible I didn't find modules for building Java projects with Maven, or modules for running JUnit tests, or JMeter tests.
So, I'm in a doubtful state: it may be I'm using Ansible in a wrong way.
When I looked at Jenkins, it can do things like build, run tests, deploy. The missing thing in Hudson is creating/deleting an instance in cloud environments like AWS.
So, in general, for what purposes do we need to use Ansible/Jenkins? For CI do I need to use a combination of Ansible and Jenkins?
Please throw some light on correct usage of Ansible.
First, Jenkins and Hudson are basically the same project. I'll refer to it as Jenkins below. See How to choose between Hudson and Jenkins?, Hudson vs Jenkins in 2012, and What is the most notable difference between Jenkins and Hudson from a user perpective? for more.
Second, Ansible isn't meant to be a continuous integration engine. It (generally) doesn't poll git repos and run builds that fail in a sane way.
When can I simply use Jenkins?
If your machine environment and deployment process is very straightforward (such as Heroku or iron that is configured outside of your team), Jenkins may be enough. You can write a custom script that does a deploy as the final build step (or a chained step).
When can I simply use Ansible?
If you only need to "deploy" without needing to build/test, Ansible might be enough. For instance, you can run a deploy from the commandline or using Ansible Tower. This is great for small projects, static sites, etc.
How do they work together?
A good combination is to use Jenkins to build, test, and save artifacts. Add a step to call Ansible or Ansible Tower to handle the actual deployment process. That allows Ansible to handle machine configuration and lets Jenkins handle the CI process.
What are the alternatives to Jenkins?
I strongly recommend Thoughtworks Go (not to be confused with Go the language) instead of Jenkins. Others include CruiseControl, TravisCI, and Integrity.
Ansible is just a "glorified SSH loop".
CI is not only the software running, but the whole process of how success and failure is handled, who gets notification, and how the change is merged into the target version control.
If we only focus on the software, CI is a reactive scheduler triggered by code changes, and triggering typical build-validate-release-deploy sequence of "steps".
So in respect of software, Ansible without additional "sugaring" is just a toolkit to run things, which can be those very steps, but it is not CI.
The Ansible (without tower) totally lacks this reactive nature.
If you want to marry Ansible with CI, you can.
Ansible tower is a very Ansible oriented scheduler, but if you need CI software, I think you not necessarily need it. Any CI app capable of running shell script would be capable to launch Ansible playbooks.
Yet unlike Ansible tower - CI tools know to display test reports of all test frameworks, trigger notifications, etc.
Ansible tower can make sense in a complex environment with lots of groups touching Ansible code... The truth is I haven't seen a single real reason to pay for it. But if a manager liked the web interface nothing can stand "but others use it" logic.
I suspect the concept of Ansible tower was in response to puppet enterprise.
:)