I have a jar and a docker image that I wish to deploy to my Compute Engine instance and run docker compose down/up after it being there. I can use git on the instance if that helps.
I want to do this using CI/CD tools, something like Google cloud build, gitlab, bitbucket pipelines. Ideally something that has a free tier.
I am aware this might be a bit vague, so am willing to add more details if necessary
In your case you can try Jenkins and use an ssh plugin to execute commands on your remote instance and send the files. There are some considerations that you might want to take before doing that.
1.- Add your ssh keys in the metadata for that instance .
2.- Make sure your firewall rules allow incoming traffic on port 22.
Once your instance allows incoming traffic on port 22 and you’d installed the ssh plugin, you just have to type the commands (docker-compose up/down) in the ssh section added by the plugin.
Related
I need some guidance and advice please on if / how I can implement a CI / CD pipeline for a corporate PHP Laravel application that is hosted on an internal server with limited access and no public IP. Unfortunately my DevOps knowledge is limited and despite a lot of Google searches, I have no idea if I am on the right track or if I am a million miles away. Everything that I have read so far looks at using Web Hooks which as far as I can tell are reliant on a public IP / domain.
At present I can work on the application on my laptop and push changes to BitBucket Cloud. While I have managed to create a bitbucket-pipelines.yml file that will automatically build and test any branches that are pushed, I then have to access the server to pull the code manually and run the various scripts as required, which I would much prefer to automate.
I need to understand please what options there are for implementing continuous deployment given the circumstances and limitations?
If anyone could offer some pointers then I would be very grateful.
Thanks
You can't push to a machine that is unreachable but there are some alternatives.
Configure a bastion host, so an external CI/CD runner can ssh through it into your server. Allow inbound port 22 connections from your CI/CD provider CIDR blocks to your bastion host https://ip-ranges.atlassian.com/
Setup a CI/CD self-hosted runner in the same network than the server. https://support.atlassian.com/bitbucket-cloud/docs/runners/ Use that runner for the deployment step to ssh into your server private IP address.
Setup a pull-based deployment strategy. Your deployment step only registers the new wanted version, e.g. moving a release/production branch on the commit ref where the deploy was triggered. In your server, setup some kind of subscription to the latest release/deployment, e.g. a cron task to frequently fetch the production git branch. Upon changes, restart services and run any task you need. ansible-pull can be handy for this purpose.
We're setting up multiple more or less static servers in AWS. These are primarily configured via Ansible and that's also the ultimate source of truth when it comes to their existence, grouping, host names and IPs. But then there's Jenkins deploying configuration files to these servers based on new commits added to a git repository.
I'm having an issue with listing the target servers directly in a Jenkinsfile. How shall I proceed? Which are the most common ways of dealing with this?
I understand this is mostly an opinion based topic. But maybe there's a particular Jenkins feature which I don't know about...?
Thank you.
This is very subjective. Following are a few ways to do this.
Store the details somewhere accessible after the Ansible step. Possibly commit to a Github repo and retrieve these details within the Jenkins Job.
Using AWS APIs/CLI to retrieve server details. You can either set up AWS CLI in Jenkins Agent or use something like AWS Step Plugin.
Do an API call to Jenkins after the Ansible script is executed and update the server details in the Job itself.
Hey I am currently learn Jenkins pipeline for CI and CD
I was successfully deploy my express js by Jenkins
On locally machine my server
It was for server and my ENV was show off on my public repository
I am here trying to understand more how to hide that ENV on my Jenkins? That use variable
And is that possible to use variable on Dockerfile also to hide my ENV ?
On my Jenkins Pipeline
I run my ENV on docker run -p -e myEnV=key
I do love to hide my ENV so people didn't know my keys inside on my Jenkinsfile and Dockerfile
I am using multi branches in jenkins because I follow the article on hackernoon for deploy react and node js app with Jenkins
And anyway, what advantages to push our container or image to Docker Hub?
If we push it to there and if we want to move our server to another server
We just need to pull our repo Docker Hub to use that to new server because what we have been build everytime it push to our repo Docker Hub , right ?
For your first question, you should use EnvInject Plugin. or If you are running Docker from the pipeline, then set Environment variable in Jenkins, then access these environment variables in Docker run command.
in the pipeline, you can access environment variable like this
${env.DEVOPS_KEY}
So your Docker run command will be
docker run -p -e myEnV=${env.DEVOPS_KEY}
But make sure you have set DEVOPS_KEY in the Jenkins server.
Using EnvInject it pretty much simple.
You can also inject from the file.
For your Second Question, Yes just the pull the image from docker-hub and use it.
Anyone from your Team can pull and run so only the Jenkins server will build and push the image. So it will save time for others and the image will be up to date and will also available remotely.
Never push or keep sensitive data in your Docker image.
Using Docker Hub or any kind of registry like Sonatype Nexus, Registry, JFrog Artifactory helps you to keep your images with their tags and share it with anyone. It also means that the images are safe there. If your local environment goes down, the images will stay there. It also helps for version control. If you are using multibranch pipelines, that means that you probably will generate different images with different tags.
Running Jenkins, working the jobs, doing the deployment is not a good practice. In my experience from previous work, the best exaples are: The server starts being bloated after some time, Jenkins doesn't work the most important times that you need it, The application you have deployed does not work because Jenkins has too many jobs that takes all the resources.
Currently, I am running different servers for Jenkins Master and Slave. Master instance does not run any jobs, only the master instances do. This keeps Jenkins alive all the time. If slaves goes down, you can simply set another slave.
For deployment, I am using Ansible which can simultaneously deploy the same docker image to multiple servers. It is easy to use and in my opinion quite safe as well.
For the sensitive data such as keys, password, api keys, you are right about using -e flag. You can also use --env-file. This way, you can keep it outside of docker image and keep the file. For passwords, I prefer to have a shell script that generates the passwords in environment files.
If you are planning to use the environment as it is, you can keep the value that you are going to set as environment variable inside Jenkins safely. then you can get that value as a variable. You can see it in Jenkins website
I'm little confused about Jenkins and was hoping someone could clarify some matter for me.
After reading up on Jenkins, both from official docs and various tutorials I get this:
If I wanna set up auto deplyoment or anything Jenkins related, I could just install docker jenkins image, launch it and access it via localhost. That is clear to me.
Then, I just put Jenkinsfile into my repository, so that it knows what and how to build my repo and stuff.
The questions that I have are:
It seems to me that Jenkins needs to be running all the time, so that it can watch for all the repo changes and trigger code building, testing and deploying. If that is the case, I'd have to install Jenkins on my droplet server. But how do I then access my dashboard, if all I have is ssh access?
If Jenkins doesn't need to be up and running 24/7, then how does it watch for any changes?
I'll try to deploy my backend and front apps on docker-compose file on my server. I'm not sure where does Jenkins integrates in all that.
How Jenkins can watch for all the repository changes and trigger code building, testing and deploying?
If Jenkins doesn't need to be up and running 24/7, then how does it watch for any changes?
Jenkins and other automation servers offer two options to watch source code changes:
Poll SCM: Download and compare source code at predefined intervals.This is simple but, hardware consumption is elevated and is a little outdated
Webhooks: Optimal functionality offered by github, bitbucket, gitlab, etc. in which Github, for example, at any git event, makes an http request to your automation server, sending all the information like branch name, commit author, etc). Here more info about webhooks and jenkins.
If you don't want a 24/7 dedicated server, you can use:
Some serverless platform or just a simple application able to receive http posts + webhook strategy. For instance, Github will perform a post requet to your app/servlerless and at this point, just execute your build, test or any other commands to deploy your application.
https://buddy.works/. It is like a mini-jenkins.
If I'd have to install Jenkins on my droplet server. But how do I then access my dashboard, if all I have is ssh access?
Yes. Jenkins is an automation server, so it needs its own dedicated server.
You can install jenkins manually or use docker in your droplet. Configure 8080 port for your jenkins. If everyting is ok, just access to your droplet public ip offered by digitalocean, like: http://197.154.458.456:8080. This url must load the Jenkins dashboard.
We are currently running several Projects on several Servers. For our workflow we installed Telnet to communicate with theses servers. More than this, we run the Build/Make mechanism (own development php) on theses servers.
Now we like to introduce a CI mechanism to this. Therefore we installed a central Jenkins application, from which we like to trigger all these builds. Our standard way is to open a telnet connection an trigger the build command on shell. Now we like to adopt this mechanism and bring it to Jenkins.
Is there a possibility to open a Telnet connection via Jenkins?
Installing a slave Jenkins on the remote Servers is a not as much preferred option, because we don't like to install more Software on the server as necessary.
Yes, It is posible, you can use pipeline to include groovy program that use telnet commands, it could be one posibility another you can use ssh commands in a free style job.