Distribute test script to slaves with same build - jenkins

Our current setup consist of a single master controlling several slave machine. Usually, the master would issue the test scripts to slaves based on list. Which ever machine is finish a script or idle, it would continue down the list. All of these are tested against a same single build.
We are moving towards CI way and would wish to do something similar to our current plans. I would think it could work the same whereby the master would build, then distribute to the slave for testing. How would this be done?
edit: Just to clarify my position, I have not use Jenkins before, therefore the lack of knowledge for it. Before flaming me into oblivion, I just need someone to point me to the right direction, not asking for a spoon feed solution.

Since you didn't accept hyde's response, I'll assume you're looking for a more detailed answer. First, if you're new to Jenkins I highly recommend Jenkins: The Definitive Guide. Jenkins isn't difficult to use, but it can be hard to get started. That book provides a nice guided tour from the beginning into some of the more intricate parts of running a Jenkins installation. Now on to your question. Jenkins allows you to try things out very quickly without having to really "install" it, so follow along here. You'll also want access to a second computer to make the temporary slave (don't worry, there's no real installation, so just use a coworker's computer).
These instructions look long, but only because I'm walking through every single step. This won't take much time. Also, feel free to click on the little question marks next to all of the fields in Jenkins as we go. By the way, when I quote something, type what's inside the quotes but not the quotes themselves.
Go to Meet Jenkins and scroll down to the Launch button to start Jenkins without installing anything.
Go to http://localhost:8080/. Look! Jenkins is running!
On the left, click Manage Jenkins, then click the Manage Nodes item.
On the left, click New Node. Name it "Foo", select "Dumb Slave", and click OK.
Set "# of executors" to 1 (it's the number of builds you want this slave to run concurrently).
Set "Remote FS root" to "c:\jenkins" or "/tmp" (this will create some files on the slave machine, so use a directory that has write permissions).
Set "Labels" to "tester"
Set "Launch method" to "Launch slave agents via Java Web Start" (you probably don't want to do this in production, but it will allow us to bring up a slave quickly).
Click Save
Click on Jenkins in the upper left corner to go back to the main page.
Click New Job on the left.
Name the job "My build" and select "free-style software project" and click OK.
Check Restrect where this project can be run and type "master" for the label (it should even auto-complete). This makes the build only run on the master.
Towards the bottom click Add build step.
If your master (the current machine) is a Windows machine, select "Execute Windows batch command", otherwise, select "Execute shell".
Either way, for the command type "echo Building". This isn't necessary, but it allows you to see that the build really did something that you told it to do.
At the bottom of the screen, click Save.
You should now be at the "Project My Build" page. Click the Build Now link on the left. Since the build barely does anything, it finishes almost instantly. An entry appears in the history with a blue circle, indicating that the build was successful.
Go back to the main Jenkins screen and add a New Job.
Name this one "My test", make it another "free-style software project", and click OK.
Again, check "Restrict where this project can be run", but make the label "tester" (again, it should auto-complete).
Scroll down and click Add build step again. Select "Windows batch" or "shell" depending on what the slave computer (e.g., your coworker's computer) is.
Type "echo Testing"
Click Save down at the bottom.
Now for the fun part. Go to your other computer (or tell a coworker that their headlights are on and use their computer). Open the web browser and go to http://your-masters-ip-address:8080/
Click Manage Jenkins, and then Manage Nodes.
Click Foo, and click the Launch button. Click through any security/"are you sure" warnings, and you should see a little window saying Jenkins is connected. Now your slave is running.
Go back to your master computer, go to your Jenkins home screen, and click My test in the list of builds.
On the left, click Build Now. This will take a little longer, because it has to contact the slave and farm out the job, but it should finish and add the build to the history.
Click the entry in the build history. Then, on the left, click Console Output.
The log should say that the build was remotely executed on Foo, and have the echo statement. Woohoo! We now have remote builds working.
Lastly, we'll link the builds together. Navigate your way back to "My build" and click Configure on the left.
In the Post-build Actions section, click Add post-build action, and select "Build other projects". Type "My test" in the field (it should autocomplete). Click Save.
Go to the Jenkins main screen, and click the clock/play icon to the right of "My Builds". If you watch carefully, you may see the jobs appear in the "Build Queue" or the "Build Executor Status" on the left.
Wait a minute or so, and then refresh the page, and you should see the "Last Success" time for both of your builds update. Clouds should part, and you should hear angels singing, but don't celebrate just yet.
You now have a master doing the "build" and a slave doing the "test", and a build triggers the test. You could add more slaves and so long as they have the label "tester", the build server will auto-distribute. You could also have many tests following the build, and have the build trigger all of them. That's all the easy stuff, the thing that requires some elbow grease is getting Jenkins to run your build and tests. Start small and work your way up. Oh, and seriously, go buy that book I linked to (it's available in dead-tree and DRM-free ebook, and no, I don't have any connection to O'Reilly or the author, it's just a very useful book) so you have an idea of what Jenkins can do and how best to fit it with your system and what you want.

Is this what you mean:
Go to slave node configuration, and give slave appropriate label, like "unittest". Repeat this for all slaves you want for unit tests (note that slave can have many labels if needed).
Go to job configuration, tick "restrict where this job can run", type "unittest" there.
Also read help texts of these fields!
For more advanced/complex CI setup, also check out node label parameter plugin, if you want to parametrize where a job can run.

Related

Jenkins Pipeline Builds: Viewing Workspace in the UI

We are now experimenting with multi-branch Pipeline builds. The main advantage of the feature is that it allows us to automatically create new Jenkins jobs whenever a new branch is created.
However, it also is a bit more difficult to implement than the old way of selecting how to do the build using the UI. Plus, certain features seem to be missing.
For example, in Jenkins Freestyle jobs, we are able to use the Jenkins UI to browse through the workspace, download individual files, and even wipe out the workspace. We found this helpful when builds went awry or if the developer needed a particular built asset that wasn't archived.
I've noticed in the Jenkins Pipeline jobs the UI no longer offers access to the workspace. I know I can archive the workspace, but I really don't want to save it -- especially for each and every build. I simply want to be able to browse the workspace or clean it out if something is causing problems with the build process.
Is there a way to get back this feature via the pipeline? I don't want to archive the workspace for each build (space issues), but I do want to be able to see what the workspace looks like if there are problems.
Freestyle Job with Workspace UI
Pipeline Job. No ability to browse Workspace
You can see your work space in three simple steps:
First, go to the build run you are interested in and click “pipeline steps”.
Then click “allocate node: start”. If you have multiple nodes, you’ll need to do this more than once.
Then you click the workspace link. (it appears on the left pane).
You can also click on Build Artifacts on build page:

How to make a specific job wait for a slave indefinitely?

Problem :
A certain task often crashes the slave, which then should reboot (sometimes it doesn't). If the reboot fails, it may take a week until someone reboots the computer. Then the build which failed couldn't archive the artifacts.
Idea to solve it (if there is a better one please tell me) :
To have a task always launch after this one which checks if the artifacts failed to be made (I have a way to check it), and then wait forever until the slave becomes available again, to archive the artifacts.
I know it's possible to do such for every job, but I cannot find how to do it for a specific job.
The best solution is to actually fix what crashes your server.
But for now you can add another job:
In Jenkins create a "New Item".
Make this a freestyle project.
In the "Build Trigger" section, select "Build after other projects are built".
Here you select the other job that causes the slave to crash.
Afterwards add you monitoring-and-restart-script as a build step.

Difference between Jenkins Job and Project

When I am on the main Jenkins screen, I get a link to Create a New Job.
When I create a new job and call it 'New Job' and run it I get results. When I review the results, I get a link to Back to Project. When I click this I see Project New Job on the top of the page.
So is a job and a project the same thing in Jenkins? Or are the names used interchangeably?
This puzzled me quite a bit, too, and although this is an old question I thought I'd expand #Jan's comment with some content from the discussion he linked, as it might help other users.
From the Jenkins CI forum:
A project is a special case of a job, i.e. all projects are jobs, but
not all jobs are projects.
Perhaps it helps to extend mentally the names to "general purpose
jobs" and "software building projects" (although the latter don't need
to build software, they are tailored to this use case).
In the case of "new job", Jenkins can't know yet if the new job
will be of the project subtype. On the other hand, once you have chosen
to create a project, this is reflected as specific as possible in the UI
("Project Foo"). If you look at the precise wording on the "New Job"
page, you'll realize the subtle differences (e.g. "Build a free-style
software project" as opposed to "Monitor an external job").
An example of a job that is not a project is the lesser known
"external job" that receives notifications from events that happen
outside Jenkins, but should be tracked inside Jenkins (e.g. system
restarts, etc.). See
https://wiki.jenkins-ci.org/display/JENKINS/Monitoring+external+jobs for
more about this job type.
For the records, this is the actual class model used by Jenkins internally
(extending sub-classes are indented below their super-class):
Job
AbstractProject
MatrixProject (a.k.a. "multi-configuration project")
Project
FreeStyleProject
MatrixConfiguration (a single configuration of a matrix project)
StubJob
ViewJob
ExternalJob
Only FreeStyleProject, MatrixProject, and ExternalJob are visible on the
"New Job" page.
Yes, they mean totally the same thing! Below is from the Jenkins guide:
Setting up the project
Go to Jenkins top page, select "New Job", then choose "Build a
free-style software project". This job type consists of the following
elements:
It seems the project is the job. Actually when you "create a new job", you will see four types of jobs, and three of them are described as "project".
Job is a deprecated term that is synonymous with Project. They both mean:
A user-configured description of work which Jenkins should perform, such as building a piece of software, etc.

How do I force a build to start in TFS?

I checked in some code and my build was added to the build queue. I can't go home until the build passes.
There is nobody in front of me, but it's been well over half an hour since my checkin and my build hasn't started (plus it's a half an hour to build the 49 large projects against the build environment). I've been at work since morning and just want to go home before midnight (it's past 8:30 already). My checkin's just sitting their in queue, not being fired off.
How do I force the build to start in an empty build machine if there is nobody in front of me in queue and nothing else going on?
edit: This is what I get for breaking my own rule of never checking in after 3pm unless it's a mandate.
Open the Team Explorer pane and navigate to your team project. Expand the 'Builds' node and find the build definition that you need to build. You should then be able to right-click it an choose 'Queue new build..'. (This does require a separate permission, and you may not have that permission).
This will then bring up the confirmation dialog where you can specify the shelveset to build (if its a gated build).
If the Builds screen shows that there are queued builds, and none running - it could be that there are no available Build Agents or they are marked offline. See Manage Your Build System for more information about how to check the status of your build agents.

"Delete Build in Jenkins after Keep Forever"

I've used the Simple Promote Plugin in Jenkins and it set my build to "keep this build forever".
Is there any way I can delete it?
I got access to the slave who build it and to the master (tried to find anything related in there but no luck).
I'm not sure whether (or how) the Simple Promote Plugin affects this at all, but can't you just click the button that says "Don't keep this build forever", followed by "Delete"?
When a build is marked as "keep forever" (and the padlock icon shows next to the build), you should be able to "unlock" it by pressing that "Don't keep..." button on the build page.
Just as Christopher said, you can remove the build just by deleting the build directory on the master, inside the job directory.
However if you access the page again, the build data gets dumped to disk again. So you either shut down Hudson first, or you go to the Hudson management console and 'Reload Configuration from Disk' which basically discards whatever's in memory and reloads from your config files.
Just make sure you do it right after deleting the folder.
In order to delete builds marked "keep this forever" you should change that flag to opposite state.
Please follow next steps
Open within a text editor $HUDSON_HOME/jobs/job-name/builds/xx/build.xml
Change
<keepLog>true</keepLog> to <keepLog>false</keepLog>
Go to Manage Jenkins -> Reload Configuration from Disk
After these steps you will be able to delete build marked as "keep this forever"
I think that the final (and may be the only one step) to completely delete build from Hudson's history is to modify "nextBuildNumber" file placed in job directory.
Please remember that "Don't keep this build forever" button will only appear when you enable "Discard Old Builds" inside job configuration. I am not sure if it affects Simple Promote Plugin, but it work in this way with 'Release Plugin'.
MichalT
Not sure about masters and slaves, but there is an answer here that allowed me to delete a build "explicitly marked to be kept".
List all keep-forever builds in Jenkins?

Resources