I built a scripted pipeline that requires docker to spin up to run the new code being deployed.
Since each docker instance runs on port 8081, only one build can run per agent.
Each build agent has 4 available executors, so I need to manually set the executors to 1 each time an agent spins up.
Is there any way I can still continue using a scripted pipeline and make sure that only one job is executed per agent?
If you have only one job you could use the following property to get running only the latest build
disableConcurrentBuilds()
You could also use the lock/milestone steps to block a stage
https://jenkins.io/blog/2016/10/16/stage-lock-milestone/
Finally you could set the port dynamically to be something else than 8081
with a bash command and python:
PORT=$(python -c 'import socket; s=socket.socket(); s.bind(("", 0)); print(s.getsockname()[1]); s.close()')
then you whould pass the PORT variable to your docker command in order to map the internal port to the new external one we got from the variable
Related
I am new to jenkins. I am trying to moniter performance test for my project.
I have my scripts in Jmeter.
I have created paarmeterize job in jenkins as shown.
Threads: 1
RampUp: 1
Loop: 40.
I am using backend listner to check data in Grafana and Appdynamics.
Now when i start the build the scripts run only once, but i am expecting script must run for 40 times (With build success).
But when i run it through jmeter, scripts run for 40 times succesfully. (Some issue with jenkins i suppose)
Please suggest how can i resolve the issue in jenkins as my project requeiremnt needs it to run script from jenkins.
Thank you in advance!
It's hard to say what's wrong without seeing your Jenkins job and JMeter Thread Group configuration.
In order to apply external settings in JMeter you need to define threads, ramp-up and the number of iterations using JMeter Properties via __P() function like:
Once done you will be able to override the values of these properties using -J command line argument, for example in Jenkins:
jmeter -Jthreads=1 -Jrampup=1 -Jloops=40 -n -t test.jmx -l result.jtl
This way you will be able to pass whatever number of virtual users/iterations without having to change your script.
More information: Apache JMeter Properties Customization Guide
I'm experiencing some odd behavior with a Jenkins build (Jenkins project is a multi-branch pipeline with the Jenkinsfile provided by the source repository). The last step is to deploy the application which involves replacing an artifact on a remote host and then restarting the process that runs it.
Everything works perfectly except for one problem - the service is no longer running after the build completes. I even added some debugging messages after the restart script to prove with the build output that it really was working. But for some reason, after the build exits the service is no longer running. I've done extensive testing to ensure Jenkins connects to the remote host as the correct user, has the right env vars set, etc. Plus, the restart script output is very detailed in the first place - there would be no way to get the successful output if it didn't actually work. So I am assuming the process that runs the deploy steps on the remote host is doing something else after the build completes execution.
Here is where it gets weird: if I run the same exact deploy commands using the Script Console for the same exact remote host, it works. And the service isn't stopped after successfully starting up.
By "same exact" I mean the script is the same, but the DSL is different between the Script Console and the pipeline. For example, in the Script Console, I use
println "deployscript.sh <args>".execute().text
Whereas in the pipeline I use
pipeline {
agent {
node 'mynode'
}
stages {
/*other stages commented out for testing*/
stage('Deploy') {
steps {
script {
sh 'deployscript.sh <args>'
}
}
}
}
}
I also don't have any issues running the commands manually via SSH.
Does anyone know what is going on here? Is there a difference in how the Script Console vs the Build Agent connects to the remote host? Do either of these processes run other commands? I understand that the SSH session is controlled by a Java process, but I don't know much else about the Jenkins implementation.
If anyone is curious about the application itself, it is a Progress Application Server for OpenEdge (PASOE) instance. The deploy process involves un-deploying the old WAR file, deploying the new one, and then stopping/starting the instance.
UPDATE:
I added 60-second sleep to the end of the deploy script to give me time to test the service before the Jenkins process ended. This was successful, so I am certain that when the Jenkins build process exits is when it causes the service to go down. I am not sure if this is an issue with Jenkins owning a process, but again the Script Console handles this fine...
Found the issue. It's buried away in some low-level Jenkins documentation, but Jenkins builds have a default behavior of killing any processes spawned by the build. This confirms that Jenkins was the culprit and the build indeed was running correctly. It was just being killed after the build completed.
The fix is to set the value of the BUILD_ID environment variable (JENKINS_NODE_COOKIE for pipeline, like in my situation) to "dontKillMe".
For example:
pipeline {
agent { /*set agent*/ }
environment {
JENKINS_NODE_COOKIE="dontKillMe"
}
stages { /*set build stages*/ }
}
See here for more details: https://wiki.jenkins.io/display/JENKINS/ProcessTreeKiller
I want to jenkins to execute a list of commands on different linux node in a network.
What steps should I take to run a command on another linux node by adrressing its ip address
if I understood you correctly you should add this node as a slave machine to the Jenkins.
go to Manage Jenkins section and then to Manage Nodes and just add a new Node
once you added the nodes.
in pilpeline groovy script
use :
node('node1'){
//command execution
}
node('node2'){
//command execution
}
I'm new to Jenkins and I'm trying to setup a project which will use few build executors.
The flow shall be as follows:
two build executors with webservice label return their IP addresses and wait for the third build executor to finish its job
third build executor with tester label collects those IP addresses and performs some long running job (e.x. sends HTTP requests to the webservices deployed on those two agents)
How to achieve such behavior in Jenkins?
I've found that when an build executor finishes its job it is immediately released and I don't know how to make it wait for other build executors to finish their jobs.
Edit:
I forgot to mention that I want the build executors with the webservice label to be reserved (not available for other jobs) till the build executor with the tester label will finish its long-running job.
Also all these build executors should be on separate slaves each. That means each slave has only one build executor.
I've finally managed to do this using Pipeline and below script:
node('webservice') {
def firstHostname = getHostname()
node('webservice') {
def secondHostname = getHostname()
node('tester') {
println 'Running tests against ' + firstHostname + ' and ' + secondHostname
// ...
}
}
}
def getHostname() {
sh 'hostname > output'
readFile('output').trim()
}
It acquires two build executors with webservice label. I'm getting their hostnames (I'm using them instead of the IP addresses) and pass them to the build executor with a tester label. Finally the tester runs some long-running tests.
Those two webservice build executors are blocked till the tester finishes its job, and no other project may use them during that time.
As Alex O mentioned, you can configure the master and slave relationship between the projects /executors inside the Jenkins projects /executors. There is option for that, "Build Triggers" -> Build after other projects are built
or use plugin to achieve it
https://wiki.jenkins-ci.org/display/JENKINS/Parameterized+Trigger+Plugin
or
https://wiki.jenkins-ci.org/display/JENKINS/Join+Plugin
What you actually want is probably that your job uses three slaves at the same time.
Re-thinking the setup in that way, it won't be necessary to consider the collection of IPs and the subsequent usage of the slaves as three different steps that must be aligned in some way.
Unfortunately, Jenkins does not support using multiple slaves for one build out-of-the box, but it will be possible to achieve what you want e.g. using the Multijob plugin and the Join plugin that Aaron mentioned already.
See also this question for information on how to use two slaves at the same time.
I'm trying to configure a jenkins job so that based on the choice of the user the node/slave should be selected.
example : if choice = windows ->slave1
if choice = Linux ->slave2
I've tried with configuration matrix i'm getting error that the nodes are offline.Is there any plugin to do such selections in jenkins
The simplest is to create two build jobs that are then tagged to a specific slave.
When you configure the job you can add labels to it that specify the requirements for this job.
Example:
linux-build, add the label linux
windows-build: add the label windows
Then, when you create the slaves you need to assign them labels as well specifying the capabilities.
Example:
Windows system, add the windows label
Linux system, add the linux label
After this your builds will automatically go to the correct system and you never have to specify anything again. This is better then having to manually specify and trigger a job.
One more advantage, if the job fails you know why.. maybe you linux job always succeeds but you windows job always fails, if you combine this in one job you can't really see this pattern but if you have two separate builds you will immediately see a dark cloud forming over you windows build.
Checkout the following plugin:
https://wiki.jenkins-ci.org/display/JENKINS/NodeLabel+Parameter+Plugin
I used the Jenkins CLI to implement the dynamical slave selection.
1) create two jobs: job A triggers job B
2) at job A, input the followings at Build/Execute Shell
if choice = windows, SERVER=slave1
if choice = Linux, SERVER=slave2
java -jar jenkins-cli.jar -s http_to_jenkins-server:port build buildname -p SERVER_LABEL=$SERVER -v -w --username yourusername --password yourpassword
3) at job B, select "This build is parameterized" and add a String Parameter SERVER.
Hope it helps.