Although it could be a silly question! but I can't find better solution so posting here.
There is existing old rails project with following detials:
Repository of on Gitlab
Using drone CI & Capistrano (already functional and doing deployments with CI)
Target server is our own server
We want to dockerize our rails app! So, Added Dockerfile and docker-compose files in project & using db from host machine. Project is working fine at local
Want to run this dockerize rails app in CI and deploy to target ENV (Staging, UAT, Production etc)?
Note: We have our own server to save docker images. Plus we don't wanna use sh scripts for the deployment.
I think using docker & drone CI, Capistrano will be removed!
Related
Absolute beginner in DevOps here. I have a Gitlab repo that I would like to build and run its tests in the Gitlab pipeline CI.
So far, I'm only testing locally on my machine with a specific runner. There's a lot information out there and I'm starting to get lost with what to use and how to use it.
How would I go about creating a container with the tools that I need ? (VS compiler, cmake, git, etc...)
My application contains an SDK that only works on windows, so I'm not sure building on another platform would work at all, so how do I select a windows based container?
How would I use that container in the yml file in gitlab so that I can build my solution and run my tests?
Any specific documentation links or suggestions are welcomed and appreciated.
How would I go about creating a container with the tools that I need ? (VS compiler, cmake, git, etc...)
you can install those tools before the pipeline script runs. I usually do this in before_script.
If there's large-ish packages that need to be installed on every pipeline run, I'd recommend that you make yourown image, with all the required build dependencies, push it to GitLab and then just use it as your job image.
My application contains an SDK that only works on windows, so I'm not sure building on another platform would work at all, so how do I select a windows based container?
If you're using gitlab.com - Windows runners are currently in beta, but available for use.
SaaS runners on Windows are in beta and shouldn’t be used for production workloads.
During this beta period, the shared runner quota for CI/CD minutes applies for groups and projects in the same manner as Linux runners. This may change when the beta period ends, as discussed in this related issue.
If you're self-hosting - setup your own runner on Windows.
How would I use that container in the yml file in gitlab so that I can build my solution and run my tests?
This really depends on:
previous parts (you're using GL.com / self hosted)
how your application is built
what infrastructure you have access to
What I'm trying to say is that I feel like I can't give you a good answer without quite some more information
I have an existing Symfony 5 project with a mysql database and a nginx Webserver. I wanna dockerize this project, but on the web I have found different opinions how to do that.
My plan is to write a multi-stage Docker file with at least a dev and a prod stage and let this build with docker-swarm. In my opinion it is useful to install the complete code during the build and having multiple composer.json files (one for every stage). In the web I have found opinions to not install the app new on every build but to copy the vendor and var folder to the container. Another opinion was to start the installation after the build process of the container is ready. But I think with that the service is not ready, when the app is successfully deployed.
What are you thinking is the best practice here?
Build exactly the same image for all environments
Do not build 2 different images for prod and dev. One of the main docker benefits is, that you can provide exactly the same environment for production and dev.
You should control your environment with ENV vars. for example, you can enable Xdebug for dev and disable it for prod.
Composer has option to install dev and production packages. You should use this feature.
If you decide to install some packages to dev. Try to use the same Dockerfile for both environment. Do not use Dockerfile.prod and Dockerfile.dev It will introduce some mess in the future.
Multistage build
You can do multistage build described in the official Docker documentation if your build environment requires much more dependencies than your runtime.
Example of it is compilation of the program. During compilation you need a lot of libraries, and you produce single binary. So your runtime does not need all dev libraries.
First stage can be build in second stage you just copy binary and it is.
Build all packages into the docker image
You should build your application when Docker image is building. All libraries and packages should be copied into image, you should not install them when the application is starting. Reasons:
Application starts faster when everything is installed
Some of the libraries can be changed in future or removed. You will be in troubles and probably you will spend a lot of time to do debugging.
Implement health checks
You should implement Health-Check. Applications require external dependencies, like Passwords, API KEY, some non sensitive data. Usually, We inject data with environment variables.
You should check if all required variables are passed, and have good format before your application is started. You can implement Health-Check or you can check it in the entrypoint.
Test your solution before it is released
You should implement mechanism of testing your images. For example in the CI:
Run your unit tests before image is built
Build the Docker image
Start new application image with dummy data. If you require PostgreSQL DB you can start another container,
Run integration tests.
Publish new version of the image only if all tests pass.
I am trying to setup a fairly simple CI/CD toolchain in TravisCI for a PHP project using composer libraries, resulting in deployment on a baremetal server via rsync.
Steps are:
Getting the code from the Github Repo upon git push.
Run composer install to get the dependencies.
(Perform Unit tests - Integration tests) - Not setup yet
Lint, codequality steps
Deploy the code to a remote apache server via rsync, using ssh keys.
Toolchain works OK so far, but I can't seem to get my head around on how the SQL migrations (in Doctrine or Phinx) can be executed automatically on the remote server.
Is the strategy of executing doctrine:migrations:migrate via ssh as the last step on the deploy section of TravisCI the best choice, or is there another better option? How do you deploy your migrations?
Thanks a lot
I once deployed to Heroku using Travis.
It was for a project using Laravel.
Because Heroku is sofisticated I have been able to tell it (from its configuration) to migrate your database after you have deployed.
However, with a classic rsync server you would need to connect to it from travis using SSH in order to migrate. (If your are as lazy as me and want to automate everything).
According to this doc you can add a after_deploy or after_success step. From this step you would run your ssh commands and migrate your database.
Apparently you can even run commands or a script via ssh so it might not be that hard. Look at the following: https://www.shellhacks.com/ssh-execute-remote-command-script-linux/
You have to pay EXTRA attention at what you put in your github repo in order to avoid security troubles with your rsync server.
Whether use this way to provide credentials to your Travis Job or that way
I have created a development environment with docker. This dev environment includes ubuntu 14.04 container (installed apache and node dependencies) , mysql container (official image) and a phpmyadmin container. How can I ensure that all containers are working fine and all dependency software installed in relevant containers using Travis-CI ?.
I could not find any test tool that can use to test docker containers. But in order to test whether docker containers are up and running, I wrote some CLI command test cases (functional tests) using a testing framework. I suggest you to try codeception PHP testing framework which is built by extending PHPUnit. And it is very easy to write shell command test cases using codeception framework.
we can simply automate the codeception test cases in travis-ci
I don't like when it comes to release my projects on production server.. May be i just don't have enough experience, nobody taught me how to do this in a right way.
For now i have several repos with scala (on top of spray). I have everything to build and run this projects on my local machine (of course, i develop them). So installed jenkins on my production server in order to sync from git, build and run. It works for now but i don't like it, because i need to install jenkins on every machine i want to have run my projects. What if i want to show my project to my friend in cafe?
So i've come with idea: what if i run tests before building app, make portable build (e.q. with sbt native packager) and save it on remote server "release server". That server just keeps these ready to be launched apps.
Then i go to production server, run bash script that downloads executables from release server and runs my project on a machine
In future i want to:
download and run projects inside docker containers.
keep ready to be served static files for frontend. Run docker
container with nginx and linked volume with static files
I heard about nexus (http://www.sonatype.org/nexus/), that artist use to save their songs, images, so on. I believe there should be open source projects that expose idea like mine
Any help is appreciated!
A common anti-pattern, in my opinion, is to build the software every time you perform a deployment.You are best advised to separate the process of build from the act of deployment by introducing a binary repository manager (you've mentioned on such example, nexus).
Best Practice - Using a Repository Manager
Binary repository manager
How can I automatically deploy a war from Nexus to Tomcat?
Only successfully tests builds get pushed to the repository, so you can treat each successful build as a mini-release. A by-product of this is that your production server does not have to have all the build software pre-installed (like, Jenkins, ANT , Maven, etc).
It should be noted that modern repository managers like Nexus and Artifactory now support Docker registries too, so that you use these for deploying docker images too.
Update
A related chef question, a technology where there is no intermediate binary file (like a jar). In this case the software is still "released" by creating a tar distribution stored in the repo.
chef cookbook delivery - chef server vs. artifactory + berkshelf