I wanted to understand how Jenkins picks up node dynamically in a given label. Suppose I have a label in which the following nodes are present.
user1-1.1.1.1
user2-1.1.1.1
user3-1.1.1.1
user1-2.2.2.2
user2-2.2.2.2
user3-2.2.2.2
How does it pick up the node from the above list ?
Is it like user1-1.1.1.1, user1-2.2.2.2, user2-1.1.1.1 , user2-2.2.2.2?
or
Is it like user1-1.1.1.1, user2-1.1.1.1, user3-1.1.1.1 ?
or
Does it pick up randomly?
Defines a logical expression which determines which agents may execute builds of this project. This expression, when tested against the name and labels of each available agent, will be either true or false. If the expression evaluates to true, then that agent will be allowed to execute builds of this project.
A label expression can be as simple as entering a single label or agent name, for example android-builder, or linux-machine-42.
You can also make use of various operators to create more complex expressions.
For example, you can use "windows && jdk9" expression in your jobs to run on windows slaves with jdk9. You have to tag(label) your nodes(slaves) appropriate way.(windows, jdk9)
Related
When i have two build clients, where one has a label of "Windows" (1st char is capitalized) and the other has a label of "windows" (all lower case), will i either need to write a job label formula of "(Windows || windows)" (assumed the case of the label is respected) or is either "Windows" or "windows" (assumed the comparison is case-insensitive) sufficient to freely run the job on any of both machines, whichever is first or free?
I have to ask, because i felt like i was unable to determine from docs in what fashion this is set up. (Some docs even indicate that some other check-operations are configurable in respect to case'ness.)
The Node labels are case sensitive in jenkins. So, When you write (Windows || windows) as a target node, jenkins will first try to run the job on the agent with label "Windows" in case if that agent doesn't respond then it will try to run the same job on the second agent with label "windows". If you want to run a job freely on any of the available agents then there are two way to accomplish that
Define the RegEx for those agents with OR (||) symbol (for example "Windows || windows"), which you already have.
Have the same label name on both agents (for example "windows") and have your job run with label "windows". It will run in a little different manner. In this case when you run that job with target label "windows", jenkins will send the request to both nodes but jenkins will run the job on the agent which will respond first.
I have job on jenkins, to which I pass config as a variable on the axis.
I'd like to add a second variable to the same job.
So now I have:
Campaign1: A
Campaign2: B
etc.
I'd like to introduce a second variable so I'll have
Campaign1: A Alpha
Campaign2: B Beta
If I introduce a second axis - I'll get a matrix build, and I don't want that.
Do you have any ideas?
Well as it turned out the easiest way to do it is to make the job parametrized. Or write a bunch of exceptions if that's your cup of tea and you have time for that.
Update: In the end I just put a delimiter between 2 variables and passed them as one and then I cutted them in a script.
Jenkins newbie, but have other build-server experience.
I'm in progress of setting up a test job, where software on two nodes need to ping-pong with each other.
I have a pool of labeled nodes (lets call them A, running windows 7) to run the testing software and another pool of labeled nodes (lets call these B, running lubuntu 14.10).
The testing is performed through TCP/IP and needs various command line stimuli on both A and B nodes throughout the test. In the end I need to gather artifacts from both the A and B nodes.
I imagine the requirement and need to control multiple nodes simultaneous isn't so rare, but I'm really having a hard time locating info on this on the web.
Might there be a plugin for this that I've missed?
Below are my thoughts of what needs to be performer, should a single plugin not exist to help me out.
My preferred solution would be a single job, but then I need to find out how to perform following:
Check out from SVN to Node A.
Check out from SVN to Node B.
Execute Windows script on Node A.
Execute Linux script on Node B.
Collect artifact from Node A.
Collect artifact from Node B.
Alternative to all the even bullets above, might be to perform those actions using SSH from either the master or the A node to control the B Node, but that leaves the following questions:
How to select one B node out of the B node pool - and mark it in use?
How to use the Jenkins SSH/slave credentials?
A totally different alternative could be to set up two jobs, one for Node A's and one for Node B's. But then I need to find out how to perform the following:
Associate one Node A job with a Node B job, so they are both aware of the association.
Perform two-ways inter-communication, allowing the Node A job to wait for a signal from a Node B job and visa verse.
Eagerly looking forward to your answers!
In a similar scenario we use, we have two jobs (I'm not aware of a way to run a single job on two nodes).
Job A runs on node A and sets up a server and then triggers job B, which is set to run on node B (as a build step).
Job B will start its client app, which is configured to work with the server installed by A (an IP configuration in my case).
The job A (server) goes into a wait loop (using bash while) that checks if client running on B has connected (I use a flag file in a shared location).
Once connected, both jobs do a few more steps and complete. Each job will end with its own reporting.
I hope this helps.
In my case I've used Multi-configuration project with Slave axis. Then you can synchronize execution between nodes using your own code, or mix it with (system) Groovy script to communicate between different configurations.
ie.
def siblings = []
build.getParentBuild().getExactRuns().each { run ->
if (!build.is(run)) {
if (run.hasntStartedYet()) {
println "Build " + name + " hasn't started yet!"
} else {
if (!run.isBuilding()) {
println "Build " + name + " has already completed!"
break
}
executor = run.getExecutor()
if (executor) {
hostname = executor.getOwner().getHostName()
// In case hostname could not be found explicitly,
// then set it based on node's name.
if (!hostname) hostname = executor.getOwner().getName()
if (hostname) {
siblings.add(hostname)
}
}
}
}
}
Suppose that I have a (fictional) set of projects: FOO and BAR. Both of these projects have some sort of multi-configuration option.
FOO has a matrix on axis X which takes values in { x1, ..., xn } (so there are n builds of FOO). BAR has a matrix on axis Y which takes values in { y1, ..., ym } (so there are m builds of BAR).
However, BAR needs to copy some artifacts from FOO. It turns out that Y is a strictly finer partition than n. For example, X might take the values { WINDOWS, LINUX } and Y might be { WINDOWS_XP, WINDOWS_7, DEBIAN_TESTING, FEDORA } or whatever.
Is it possible to get BAR to do some sort of table lookup to work out what configuration of FOO it needs when it copies artifacts across? I can easily write a shell script to spit out the mapping, but I can't work out how to invoke it when Jenkins is working out what it needs to copy.
At the moment, a hacky solution is to have two axes on FOO, on for X and one for Y, and then filter out combinations that don't make sense. But the resulting combination filter is ridiculous and the matrix is very sparse. Yuck.
A solution that I don't like is to parametrise FOO on Y instead: this would be a huge waste of compile time. And, worse, the generated artefacts are pretty big, so even if you did some sort of caching, you'd still have to keep unnecessary copies floating around.
Can't say I fully understand the intricacies if your matrices, but I think I can help you with your actual question
"I can easily write a shell script to spit out the mapping, but I can't work out how to invoke it when Jenkins is working out what it needs to copy"
The Archive the artifacts and Copy artifacts from another project post-build actions can take java style wildcards, like module/dist/**/*.zip as well as environment variables/parameters, like ${PARAM} for the list or artifacts. You can use commas , to add more artifacts.
The on-page help for Copy artifacts from another project states how to copy artifacts of a specific matrix configuration: To copy from a particular configuration, enter JOBNAME/AXIS=VALUE, this is for the Project Name attribute. That project name attribute can also contain params as ${PARAM}
So, in your BAR job, have a Copy Artifacts build step, with Project Name being FOO/X=${mymapping}. What this will do is: every time a configuration of BAR is run, it will copy artifacts only from FOO with configuration of X=${mymapping}.
Now you need to set the value of ${mymapping} dynamically every time BAR is run. A simple script like this may do the trick:
[[ ${Y:0:7} == "WINDOWS" ]] && mymapping=WINDOWS || mymapping=LINUX
Finally, you need to use EnvInject plugin to make this variable available to the rest of the build steps, including the Copy Artifacts step.
So, every time BAR configuration runs, it will look at its own configuration axis Y, and if that axis starts with WINDOWS, it will set the ${mymapping} to WINDOWS, else set it to LINUX. This ${mymapping} is then made available to the rest of the build steps. When Copy Artifacts build step is executed, it will only copy artifacts from FOO where the X axis matches ${mymapping} (i.e. either WINDOWS or LINUX).
Full Setup
Install EnvInject plugin.
In BAR job configuration, tick Prepare an environment for the run (part of EnvInject plugin).
Make sure both checkboxes for keeping existing variables are checked.
In Script Content copy your script:
[[ ${Y:0:7} == "WINDOWS" ]] && mymapping=WINDOWS || mymapping=LINUX
Under Build steps, configure Copy Artifacts build step.
Set Project name parameter to FOO/X=${mymapping}
Configure the rest as usual.
I want to have an EC2 based cluster that can grow and shrink at will. No node will be special in any way nor do I want them to have to coordinate their names with any other nodes. I don't want to hard code the names since I want to use one image and spin them up as needed. I understand nodes have to have names to communicate, though. What's a good strategy for automatically and dynamically coming up with a name at start script time?
Use uuidgen, or the equivalent API if your shell scripting language isn't (ba)sh. If you don't want to see single quotes around node names, then replace - with _ and add a prefix, just in case the first character is a decimal digit.
(Sorry, if I'm stating the obvious, but...) If you want the node to be discoverable, get the initialisation code to publish the node to some kind of directory service node.
Call it node#my.ip.add.ress. I did this on EC2 and had an erlang cluster I could spin up at an arbitrary size directly from the AWS console.