When i have two build clients, where one has a label of "Windows" (1st char is capitalized) and the other has a label of "windows" (all lower case), will i either need to write a job label formula of "(Windows || windows)" (assumed the case of the label is respected) or is either "Windows" or "windows" (assumed the comparison is case-insensitive) sufficient to freely run the job on any of both machines, whichever is first or free?
I have to ask, because i felt like i was unable to determine from docs in what fashion this is set up. (Some docs even indicate that some other check-operations are configurable in respect to case'ness.)
The Node labels are case sensitive in jenkins. So, When you write (Windows || windows) as a target node, jenkins will first try to run the job on the agent with label "Windows" in case if that agent doesn't respond then it will try to run the same job on the second agent with label "windows". If you want to run a job freely on any of the available agents then there are two way to accomplish that
Define the RegEx for those agents with OR (||) symbol (for example "Windows || windows"), which you already have.
Have the same label name on both agents (for example "windows") and have your job run with label "windows". It will run in a little different manner. In this case when you run that job with target label "windows", jenkins will send the request to both nodes but jenkins will run the job on the agent which will respond first.
Related
I got a WS variable which is saving the Jenkins WORKSPACE variable.
During stages I always check if the WS == WORKSPACE Jenkins generic variable.
What I saw that during parallel runs ( when 2 different workspaces are created inside C:/jenkins/workspace#1 and C:/jenkins/workspace#2 the $WORKSPACE is swapping between those 2 parallels builds.
The problem is reproduced rarely , less than 10 percent of cases but I find it quite strange , for the picture above -> The first workspace its AVB_Aplicattions_BOSCH-3 its going trough 3 stages and in the 4th stages the $WORKSPACE variable its swapping with AVB_APLICATTIONS_BOSCH-4( the other parallel build).If I will look on the other build (AVB_APLICATTIONS_BOSCH-4 I will see the same problem-> the workspace is becoming AVB_APLICATTIONS_BOSCH-3).
During this builds I compile using cmake files, I'm afraid that the results will be not correct.
I was thinking to use the in build dir() function during each stage to be sure im on the same workspace .
Some one know a good approach for this case and also why its happening?
I don't want to deactivate parallel's builds.
The problem is that Jenkins variable scopes aren't really safe with parallel stages (see further sources / link). There are actually several questions and answers (and workaround) how to work with variables when using parallel stages. Long story short: don't use global variables but pay extra attention to define local variables with def to limit their scope:
Jenkins - matrix jobs - variables on different slaves overwrite each other?
Variables get overwritten in parallel stages
Parallel matrix and global variables without race condition?
Jenkins community links:
https://issues.jenkins.io/browse/JENKINS-55040
https://issues.jenkins.io/browse/JENKINS-54732
I wanted to understand how Jenkins picks up node dynamically in a given label. Suppose I have a label in which the following nodes are present.
user1-1.1.1.1
user2-1.1.1.1
user3-1.1.1.1
user1-2.2.2.2
user2-2.2.2.2
user3-2.2.2.2
How does it pick up the node from the above list ?
Is it like user1-1.1.1.1, user1-2.2.2.2, user2-1.1.1.1 , user2-2.2.2.2?
or
Is it like user1-1.1.1.1, user2-1.1.1.1, user3-1.1.1.1 ?
or
Does it pick up randomly?
Defines a logical expression which determines which agents may execute builds of this project. This expression, when tested against the name and labels of each available agent, will be either true or false. If the expression evaluates to true, then that agent will be allowed to execute builds of this project.
A label expression can be as simple as entering a single label or agent name, for example android-builder, or linux-machine-42.
You can also make use of various operators to create more complex expressions.
For example, you can use "windows && jdk9" expression in your jobs to run on windows slaves with jdk9. You have to tag(label) your nodes(slaves) appropriate way.(windows, jdk9)
I want to run a job on different slaves, but in a predefined order.
i.e. I would like the job to run on the second machine only when all the executors of the first slave are busy.
I am trying to use "Scoring Load Balancer" plugin to allocate scores to different slaves,i.e. if I have 3 nodes, NodeA,NodeB and NodeC, having preferences of 9,5 and 1 respectively and no. of executors 10 on each node.
The nodes have a common label "WindowSlave".
Also, I have defined a job,"ProjectX", with project preference score of 10 and the label "WindowSlave" as the preferred label.
I had expected that if I run 100 concurrent builds for the "ProjectX" then the execution would happen in the order :
NodeA(10 builds) -> NodeB(10 builds) -> NodeC(10 builds) -> NodeA(10 builds) -> NodeB(10 builds)->... and so on.
From my observations it is still not clear if above scenario would always be achieved.
Also it happens that any random slave starts behaving as the main slave and co-ordinates with the other slave,such that all the build workspace are created on that particular slave.
What am i missing here??
Jenkins newbie, but have other build-server experience.
I'm in progress of setting up a test job, where software on two nodes need to ping-pong with each other.
I have a pool of labeled nodes (lets call them A, running windows 7) to run the testing software and another pool of labeled nodes (lets call these B, running lubuntu 14.10).
The testing is performed through TCP/IP and needs various command line stimuli on both A and B nodes throughout the test. In the end I need to gather artifacts from both the A and B nodes.
I imagine the requirement and need to control multiple nodes simultaneous isn't so rare, but I'm really having a hard time locating info on this on the web.
Might there be a plugin for this that I've missed?
Below are my thoughts of what needs to be performer, should a single plugin not exist to help me out.
My preferred solution would be a single job, but then I need to find out how to perform following:
Check out from SVN to Node A.
Check out from SVN to Node B.
Execute Windows script on Node A.
Execute Linux script on Node B.
Collect artifact from Node A.
Collect artifact from Node B.
Alternative to all the even bullets above, might be to perform those actions using SSH from either the master or the A node to control the B Node, but that leaves the following questions:
How to select one B node out of the B node pool - and mark it in use?
How to use the Jenkins SSH/slave credentials?
A totally different alternative could be to set up two jobs, one for Node A's and one for Node B's. But then I need to find out how to perform the following:
Associate one Node A job with a Node B job, so they are both aware of the association.
Perform two-ways inter-communication, allowing the Node A job to wait for a signal from a Node B job and visa verse.
Eagerly looking forward to your answers!
In a similar scenario we use, we have two jobs (I'm not aware of a way to run a single job on two nodes).
Job A runs on node A and sets up a server and then triggers job B, which is set to run on node B (as a build step).
Job B will start its client app, which is configured to work with the server installed by A (an IP configuration in my case).
The job A (server) goes into a wait loop (using bash while) that checks if client running on B has connected (I use a flag file in a shared location).
Once connected, both jobs do a few more steps and complete. Each job will end with its own reporting.
I hope this helps.
In my case I've used Multi-configuration project with Slave axis. Then you can synchronize execution between nodes using your own code, or mix it with (system) Groovy script to communicate between different configurations.
ie.
def siblings = []
build.getParentBuild().getExactRuns().each { run ->
if (!build.is(run)) {
if (run.hasntStartedYet()) {
println "Build " + name + " hasn't started yet!"
} else {
if (!run.isBuilding()) {
println "Build " + name + " has already completed!"
break
}
executor = run.getExecutor()
if (executor) {
hostname = executor.getOwner().getHostName()
// In case hostname could not be found explicitly,
// then set it based on node's name.
if (!hostname) hostname = executor.getOwner().getName()
if (hostname) {
siblings.add(hostname)
}
}
}
}
}
Suppose that I have a (fictional) set of projects: FOO and BAR. Both of these projects have some sort of multi-configuration option.
FOO has a matrix on axis X which takes values in { x1, ..., xn } (so there are n builds of FOO). BAR has a matrix on axis Y which takes values in { y1, ..., ym } (so there are m builds of BAR).
However, BAR needs to copy some artifacts from FOO. It turns out that Y is a strictly finer partition than n. For example, X might take the values { WINDOWS, LINUX } and Y might be { WINDOWS_XP, WINDOWS_7, DEBIAN_TESTING, FEDORA } or whatever.
Is it possible to get BAR to do some sort of table lookup to work out what configuration of FOO it needs when it copies artifacts across? I can easily write a shell script to spit out the mapping, but I can't work out how to invoke it when Jenkins is working out what it needs to copy.
At the moment, a hacky solution is to have two axes on FOO, on for X and one for Y, and then filter out combinations that don't make sense. But the resulting combination filter is ridiculous and the matrix is very sparse. Yuck.
A solution that I don't like is to parametrise FOO on Y instead: this would be a huge waste of compile time. And, worse, the generated artefacts are pretty big, so even if you did some sort of caching, you'd still have to keep unnecessary copies floating around.
Can't say I fully understand the intricacies if your matrices, but I think I can help you with your actual question
"I can easily write a shell script to spit out the mapping, but I can't work out how to invoke it when Jenkins is working out what it needs to copy"
The Archive the artifacts and Copy artifacts from another project post-build actions can take java style wildcards, like module/dist/**/*.zip as well as environment variables/parameters, like ${PARAM} for the list or artifacts. You can use commas , to add more artifacts.
The on-page help for Copy artifacts from another project states how to copy artifacts of a specific matrix configuration: To copy from a particular configuration, enter JOBNAME/AXIS=VALUE, this is for the Project Name attribute. That project name attribute can also contain params as ${PARAM}
So, in your BAR job, have a Copy Artifacts build step, with Project Name being FOO/X=${mymapping}. What this will do is: every time a configuration of BAR is run, it will copy artifacts only from FOO with configuration of X=${mymapping}.
Now you need to set the value of ${mymapping} dynamically every time BAR is run. A simple script like this may do the trick:
[[ ${Y:0:7} == "WINDOWS" ]] && mymapping=WINDOWS || mymapping=LINUX
Finally, you need to use EnvInject plugin to make this variable available to the rest of the build steps, including the Copy Artifacts step.
So, every time BAR configuration runs, it will look at its own configuration axis Y, and if that axis starts with WINDOWS, it will set the ${mymapping} to WINDOWS, else set it to LINUX. This ${mymapping} is then made available to the rest of the build steps. When Copy Artifacts build step is executed, it will only copy artifacts from FOO where the X axis matches ${mymapping} (i.e. either WINDOWS or LINUX).
Full Setup
Install EnvInject plugin.
In BAR job configuration, tick Prepare an environment for the run (part of EnvInject plugin).
Make sure both checkboxes for keeping existing variables are checked.
In Script Content copy your script:
[[ ${Y:0:7} == "WINDOWS" ]] && mymapping=WINDOWS || mymapping=LINUX
Under Build steps, configure Copy Artifacts build step.
Set Project name parameter to FOO/X=${mymapping}
Configure the rest as usual.