I have a swarm of three nodes (one manager, two workers). In my swarm, I am running a jenkins service with docker-swarm-plugin (https://github.com/jenkinsci/docker-swarm-plugin) installed. I want to use the plugin to create a build agent container in my swarm for every jenkins job. For the agents I want to user the jenkins/jnlp-slave docker image (https://hub.docker.com/r/jenkinsci/jnlp-slave/). The image expects two arguments for the start:
secret (can be set via JENKINS_SECRET environment variable)
agent name (can be set via JENKINS_AGENT_NAME environment variable)
The docker-swarm-plugin creates three environment variables:
$DOCKER_SWARM_PLUGIN_JENKINS_AGENT_SECRET (I use this to set the secret)
$DOCKER_SWARM_PLUGIN_JENKINS_AGENT_JAR_URL
$DOCKER_SWARM_PLUGIN_JENKINS_AGENT_JNLP_URL (this contains the agent name)
I pass the secret to the agent via JENKINS_SECRET environment variable (in ENV section of Jenkins plugin configuration):
JENKINS_SECRET=$DOCKER_SWARM_PLUGIN_JENKINS_AGENT_SECRET
I tried to pass the agent name by using a regular expression (also in ENV section of Jenkins plugin configuration):
JENKINS_AGENT_NAME=`echo $DOCKER_SWARM_PLUGIN_JENKINS_AGENT_JNLP_URL | sed ...`
But the command is not executed (I understand that this is for security reasons to avoid code injection).
What do I want to reach:
I want to run jenkins on my docker swarm and I want jenkins to run every job in an own build agent container that is dropped after the job finished. And I want the build agent containers to spread across the swarm (jenkins docker-plugin launches them on the node where the jenkins master is running). I understood that the docker-swarm-plugin should do exactly what I want to do. And I think the jenkins/jnlp-slave image is there to build agent containers as I want to use. But I can't find a solution how to get them work together.
Can anyone give me some advice?
Should I maybe use another image that is working better with the plugin?
I opened issue https://github.com/jenkinsci/docker-swarm-plugin/issues/37 on docker-swarm-plugin for this and now with PR https://github.com/jenkinsci/docker-swarm-plugin/pull/39 a new environment variable is added with the created agents name. This can be passed to the docker image and everything works fine!
Related
i am using kubernetes plugin in jenkins.i want to run an init script before slave container provision on kubernetes.the kubernetes plugin pod template won't allow me to or i cannot find a way to run it.can anybody please help me with this.i need to run a certain set of commands on the kubernetes slave container before it provisions.this is how my config looks like
1 - You don't need to use the slave image, jenkins will have an agent container regardless, you should only specify containers you want it to run additionally.
2 - On the containers you can specify the entrypoints or pre-provision whatever you want beforehand and just worry about the execution. That means you can get a container ready to go and assume the code will be there, if you need to run any extra commands on the code, you can just add an extra script step
3 - In order for your step to be executed in a container, you have to be explicit in your pipeline, otherwise it will run on the master.
I can't really guide you to using the UI because I use the Jenkinsfile inside projects I want to build.
Hey I am currently learn Jenkins pipeline for CI and CD
I was successfully deploy my express js by Jenkins
On locally machine my server
It was for server and my ENV was show off on my public repository
I am here trying to understand more how to hide that ENV on my Jenkins? That use variable
And is that possible to use variable on Dockerfile also to hide my ENV ?
On my Jenkins Pipeline
I run my ENV on docker run -p -e myEnV=key
I do love to hide my ENV so people didn't know my keys inside on my Jenkinsfile and Dockerfile
I am using multi branches in jenkins because I follow the article on hackernoon for deploy react and node js app with Jenkins
And anyway, what advantages to push our container or image to Docker Hub?
If we push it to there and if we want to move our server to another server
We just need to pull our repo Docker Hub to use that to new server because what we have been build everytime it push to our repo Docker Hub , right ?
For your first question, you should use EnvInject Plugin. or If you are running Docker from the pipeline, then set Environment variable in Jenkins, then access these environment variables in Docker run command.
in the pipeline, you can access environment variable like this
${env.DEVOPS_KEY}
So your Docker run command will be
docker run -p -e myEnV=${env.DEVOPS_KEY}
But make sure you have set DEVOPS_KEY in the Jenkins server.
Using EnvInject it pretty much simple.
You can also inject from the file.
For your Second Question, Yes just the pull the image from docker-hub and use it.
Anyone from your Team can pull and run so only the Jenkins server will build and push the image. So it will save time for others and the image will be up to date and will also available remotely.
Never push or keep sensitive data in your Docker image.
Using Docker Hub or any kind of registry like Sonatype Nexus, Registry, JFrog Artifactory helps you to keep your images with their tags and share it with anyone. It also means that the images are safe there. If your local environment goes down, the images will stay there. It also helps for version control. If you are using multibranch pipelines, that means that you probably will generate different images with different tags.
Running Jenkins, working the jobs, doing the deployment is not a good practice. In my experience from previous work, the best exaples are: The server starts being bloated after some time, Jenkins doesn't work the most important times that you need it, The application you have deployed does not work because Jenkins has too many jobs that takes all the resources.
Currently, I am running different servers for Jenkins Master and Slave. Master instance does not run any jobs, only the master instances do. This keeps Jenkins alive all the time. If slaves goes down, you can simply set another slave.
For deployment, I am using Ansible which can simultaneously deploy the same docker image to multiple servers. It is easy to use and in my opinion quite safe as well.
For the sensitive data such as keys, password, api keys, you are right about using -e flag. You can also use --env-file. This way, you can keep it outside of docker image and keep the file. For passwords, I prefer to have a shell script that generates the passwords in environment files.
If you are planning to use the environment as it is, you can keep the value that you are going to set as environment variable inside Jenkins safely. then you can get that value as a variable. You can see it in Jenkins website
I'd like to be able to dynamically provision docker child nodes for builds and have the configuration / setup of those nodes be part of the Jenkinsfile groovy script it uses.
Limitations of the current setup of jobs means Jenkins has one node/executor (master) and I'd like to support using Docker for nodes to alleviate this bottleneck.
I've noticed there's two ways of using a docker container as a node:
You can use the agent section in your pipeline file which allows you to specify an image to use. As part of this, you can target a specific node which supports running docker images, but I haven't gotten that far as to see what happens.
You can use the Jenkins Docker Plugin which allows you to add a Docker Cloud in Jenkins' configuration. It allows you to specify a label which, when used as part of a build, will spawn a container in that "cloud" from the image chosen in the cloud configuration. In this case, the "cloud" is the docker instance running on the Jenkins server.
Unfortunately, it doesn't seem like you can use both together - using the label but specifying a docker image in the configuration (1) where the label matches a docker cloud template configuration (2) does not seem to work and instead produces a label not found error during the build.
Ideally I'd prefer the control to be in the pipeline groovy file so the configuration is stored with the application (1), not with the Jenkins server (2). However, it suggests that if I use the agent section and provide a docker image, it still must target an existing executor first (i.e. master) which will cause other builds to queue until the current build is complete.
I'm at a point of migrating builds, so not all builds can support using a docker container as the node yet, and builds will have issues when ran in parallel on the master node.
Is there a way for a docker pipeline file to determine the image of the child node it runs on?
There are a few options I have considered but not attempted yet:
Migrate jobs to run on the "docker cloud" until all jobs support running on child container nodes, then move the configuration from Jenkins to the pipeline build file for each job and turn on parallel builds on the master node.
Attempt to add a new node configuration which is effectively a copy of master (uses the same server, just different location). Configure it to support parallel builds, and have all migrated jobs target the node explicitly during builds.
I'm looking for a way to run Jenkins jobs/build inside Jenkins slaves, dynamically (on-demand) started docker. Attaching schema for visual understanding.
What I'm actually looking for and my flow looks like:
1) Triggering Jenkins job (manually/git/gerrit)
2) Jenkins master (running in docker) starts slave machine docker (and pass script/instructions of the build)
3) Build is running on Jenkins slave (or slaves if parallel/pipeline)
4) Result returned to Jenkins master
5) Jenkins slave docker stops
Is it possible to do it this way?
Docker slave image creation steps like installing openssh, user creation, mentioned in the below link. Install docker plugin from the below link.
Click here!
Go to jenkins global configuration, Under cloud headings, docker configuration will be there, enter docker host url with port number, credential not required. give some values for connection timeout & read timeout.
Under docker template - Enter the docker image name which we created in the point number 1.
Set Label number (Give this label name during Jenkins job creation and restrict to this slave name),
Select the usage option - > only build job with label restriction.
No of executor -> minimum 1. Select launch method as ssh, enter the user credential to login, which we created in docker image in the step number 1.
create a job restrict to docker slave label, run, ondemand it wil spun up container.
Use this plugin: https://wiki.jenkins-ci.org/display/JENKINS/Yet+Another+Docker+Plugin
After instalation (it requires Java 1.8) naviaget to configuration. There are two steps:
configure docker "cloud"
add "instances" (docker images) you want to run the build on
Every image should have label assigned - use this label in you job configuration to tell Jenkins explicitlyon which node the job should be run
I have latest Jenkins and using it's latest Swarm Plugin.
I have written Ansible modules/roles/playbooks to setup install various tools/configuration on a given target node (which I would like to use as a Swarm slave node).
After Ansible playbook run is complete, I now see a new Slave is created and attached to my Jenkins master but Swarm Plugin's docs (Available Options) doesn't mention how to create ENVIRONMENT variables in the slave. https://wiki.jenkins-ci.org/display/JENKINS/Swarm+Plugin
My question is:
How can I have multiple slaves created on a same target machine and they all have their own individual settings for setting various tools like JAVA_HOME, M2_HOME, GRADLE_HOME, PATH etc.
How can I set ENVIRONMENT variables for a slave using Swarm plugin?
This is required as if I created a slave whose default JAVA is jdk1.7.0_67, then I would like to create another slave whose default JAVA_HOME is jdk1.8.0_45. Similarly, the end goal is to have various flavors of such slaves with various tools if possible, where each slave's tools are slightly different. I'll assign the LABEL(s) accordingly and use it in a Jenkins job's configuration so that a job runs only using / on these slave if the associated label is assigned/tied to the job.
I tried using https://github.com/MovingBlocks/GroovyJenkins/blob/master/src/main/groovy/AddNodeToJenkins.groovy but not sure how I can automatically define/set ENVIRONMENT variables in the slave's configuration.
I'm assuming you're running on Linux here.
You can have a shell script to export the new environment before calling the swarm-client. These variables will be inherited by the new swarm slave
https://unix.stackexchange.com/questions/130985/if-processes-inherit-the-parents-environment-why-do-we-need-export
Alternatively you could run docker and have a separate swarm slave containers https://hub.docker.com/r/csanchez/jenkins-swarm-slave/ and put your specific install into the Dockerfile and add a new ENTRYPOINT in the bottom of the Dockerfile
ENTRYPOINT ["/usr/local/bin/jenkins-slave.sh" \
"-labels", "label1", "-labels", "label2"]