I've been running a Gitlab CI runner in a linux server for a while configured with docker executor. This has been working for a while now until I have been asked to add a docker-windows executor to the pool in order to compile windows projects under docker.
It seems that the windows-docker executor is now trying to also execute linux based jobs and failing with the following error:
Running with gitlab-runner 14.0.1 (c1edb478)
on XXXXXXXXXXXXX_
Preparing the "docker-windows" executor 00:14
Using Docker executor with image SOME_LINUX_BASED_IMAGE_HERE ...
Authenticating with credentials from job payload (GitLab Registry)
Pulling docker image SOME_LINUX_BASED_IMAGE_HERE ...
WARNING: Failed to pull image with policy "always": image operating system "linux" cannot be used on this platform (manager.go:205:2s)
ERROR: Preparation failed: failed to pull image "SOME_LINUX_BASED_IMAGE_HERE" with specified policies [always]: image operating system "linux" cannot be used on this platform (manager.go:205:2s)
I have searched Gitlab documentation on how to avoid this but have been unable to do so. How can I configure my runners to avoid this?
Thanks in advance
You can assign tags to runners and then a specific runner will only accept jobs that have a specific tag.
cf. https://docs.gitlab.com/runner/#tags and https://docs.gitlab.com/ee/ci/yaml/#tags
I introduced OS specific tags for all my runners and configured my default runners (majority is linux) to also accept jobs without tags in order to avoid tagging of all jobs that should run on the default runners.
My final configuration to avoid the problem is the following:
Tag all the runners with the corresponding OS tag when registering a runner.
Because my organization runs by default on linux, linux runners are configured to allow to pick up jobs without tags.
Related
How to automate Jenkins for Docker Swarm deployment.
I am wondering if there are any plugins available in Jenkins which will help in Docker Swarm deployment or any other alternative way through which we can achieve the automation of Swarm deployment using Jenkins existing plugins?
Fixed this problem by using a plugin called Publish over SSH
Need to install a Jenkins plugins “Publish over SSH”, this plugin will allows us to
Sends files over SSH(SFTP)
Execute commands on a remote server
First step will be to add remote hosts and second will be to add an execution/build step where the commands will be executed
Follow this link for step by step instruction
I am developing a small website (Ruby/Sinatra) to be used internally where I work. (Simply, it crunches some source data and generates reports.)
I'm want to deploy it using Docker and have a set up that works on my dev environment, but I'm trying to understand the workflow for "production" deployment (we're using Jenkins).
I've read lots of articles about deployment workflows using Docker, but they all seem to stop at "and then push your image to the Docker registry". What seems to be missing is how to then take that image and actually update the application.
I appreciate that every application is likely to be different, but what is the next step? I'm aware of lots of different frameworks like Chef, Puppet, Ansible that could be used, but my question really is - how do I integrate that into my CI/CD pipeline? E.g. does a job "push" the changes to the production server, or should a Jenkins slave be running on the production server to execute a job directly on the server?
There are several orchestration tools like docker-swarm, kubernetes and rancher. In docker swarm for example you create services and can update the versions in blue-green deployment manner also for just one instance (then there is no blue-green :) ) and if you just use docker run you should check your running container, stop and remove it if its running an start your docker container with the newer image version.
It depends on how your application is configured to run. In my case, I have a call to "docker run" in a systemd script. It's configured to just restart if it ever stops.
So, in my Jenkinsfile, after I push the image to the registry, I do a "docker pull" (my Jenkins agent is running on the same box that the application is running on), and then a "docker stop". That causes the application to exit, then restarts, which causes it to get the new version that was just pulled, and now it's running the new version.
I want to setup Jenkins master on server A and slave on server B with use of Docker.
Both servers are virtual machines dedicated for Jenkins.
Currently I have started Docker container on server A for master, based on the official Jenkins docker image. But what docker image should I use for Jenkins slave?
That actually depends on the environment and tools you need in your build environment. For example, if you build a C project, you would need an image containing a C compiler and possibly make if you use Makefiles. If you build a Java project, you would need a JDK with a Java compiler and possibly Ant / Maven / Gradle if you use them as part of your build.
You can use the evarga/jenkins-slave as a good starting point for your build slave.
This image already contains JDK. If you simply need JDK and Maven on your build slave, you can build your Docker image with the following Dockerfile:
FROM evarga/jenkins-slave
run apt-get install maven
Using Docker images for build slaves is actually a good idea. Some of the reasons appear at Templating Jenkins Build Environments with Docker Containers:
Docker has established itself as a popular and convenient way to
bootstrap isolated and reproducible environments, which enables Docker
containers to be the most maintainable slave environments. Docker
containers’ tooling and other configurations can be version controlled
in an environment definition called a Dockerfile, and Dockerfiles
allows multiple identical containers can be created quickly using this
definition or for more customized off-shoots to be created by using
that Dockerfile’s image as a base.
I suggest you take trying to use dynamic|ephemeral docker nodes, instead of manually creating nodes and connecting to them via ssh. Take a look at https://engineering.riotgames.com/news/putting-jenkins-docker-container, it's very powerful and I think it's one of killer usecases for Docker.
I'm trying to run build Docker task to create a docker image. I set up a docker host, I'm using defautl Docker Hub as registry and my whole environment is on Azure.
When I queue a build task it fails at Task Docker.
Log output:
check path : null
task result: Failed
Not found docker: null
Finishing task: Docker
[error]Task Docker failed. This caused the job to fail. Look at the logs for the task for more details.
Does someone have any thought on what may be happening?
After looking into this, it would seem this happens if Docker is not properly installed on the build agent for the service principal the agent is running under.
Keep in mind that:
The Build must be run in a private agent, as the hosted ones do not yet have Docker installed, as per a very small footnote in the bottom of the documentation.
The VSTS agent must be running with a principal that has the environment variables set for docker to run; the default is LocalService account, which won't have that installed. This turns out to be a problem with other stuff as well and I've found it best to have a special user principal to run the agent under, that can also log into the system.
Fixing these two issues made it work for me.
I was able to switch the agent to Hosted VS2017 which has Docker support.
If Linux is an option, try Hosted Linux Preview
I've been trying gitlab and its CI workflow these days, but found myself confused when I saw these messages during a build:
gitlab-ci-multi-runner 0.6.2 (3227f0a)
Using Docker executor with image mydocker:latest ...
Running on runner-5498280b-project-20053-concurrent-0 via jls-MacBook-Pro...
I register a project specified runner instead of using the shared ones.
Is gitlab actually running all the CI build process via my own machine? What if other co-workers push to this project while my computer was off? I just thought that gitlab would provide every project with a cloud CI server... So , I don't want to turn my own computer into such a server. Am I missing something on its docs?
See this Gitlab Docs
Is gitlab actually running all the CI build process via my own machine?
--Yes, Because you specified it. If you don't want to run into your machine, Then deploy the docker to other server.
What if other co-workers push to this project while my computer was off?
--Then, no build will going to happen,