I'm working on a multiconfiguration job(Regression_L1) in Jenkins whose task is to run 2 kinds of tests(test1 and test2). In the multiconfiguration job, it triggers an executor job(Regression_executor) to run script for the selected test. The Regression_L1 job is restricted to run in matrix_service_jobs node, while the matrix jobs are to run in the slave node custom_matrix_service_jobs node. The Regression_executor job is restricted to run in remote machines with a specific label(RL1_Test_Machine).
My goal is to test custom build from developers. And so I added a File Parameter(config - File Location: CUSTOMBUILD/mybuild.zip) for the job. The question is how can I access the uploaded file?
Some important info:
Regression_executor's workspace: /home/regressionexec/
Regression_L1's workspace: /var/work/matrix_service_jobs/Regression_L1
Regression_l1 matrix workspaces: /var/work/workspace_user_matrix/workspace/Regression_L1/TEST_PHASE/test1/label/custom_matrix_service_jobs/ and /var/work/workspace_user_matrix/workspace/Regression_L1/TEST_PHASE/test2/label/custom_matrix_service_jobs/
$JENKINS_HOME: var/work/jenkins_home
I did not know where to find the uploaded files so I did a search ung linux find. The result is:
/var/work/jenkins_home/Regression_L1/TEST_PHASE/test2/label/custom_matrix_service_jobs/builds/${BUILD_NUMBER}/fileParameters/CUSTOMBUILD/mybuild.zip.
How can I copy it to the slave node that executes the test script?
Whatever you enter under "File location", that would be the location and the variable that holds the original filename of the uploaded file.
However, on *nix, neither / nor . are valid variable name characters, so in your case, if "File Location" is CUSTOMBUILD/mybuild.zip system cannot create a variable ${CUSTOMBUILD/mybuild.zip}
The file though is still placed under ${WORKSPACE}/CUSTOMBUILD/mybuild.zip. You can access it with this path too.
You can then use Copy To Slave plugin, to copy the file from master to your slaves
I couldn't find my uploaded file under the WORKSPACE, so I ended using something like
"%JENKINS_HOME%\jobs\%JOB_NAME%\builds\%BUILD_ID%\fileParameters\myUploadedFile"
Related
I have 3 Jenkins jobs to be run in serial.
Run a Ant File
Run another ANT File
Run a command line
All the above jobs use a file path which is set in a properties file.
Ex Job 1 , Executes ANT file placed in file path location
Job 2 , Executes another file placed in same file path location
Job 3 , Executes command line to do SVN update in same file path location
I need to parameterize the file path in all three builds from properties file.
Can anyone help me with possible approach?
Thanks In Advance
This answer could be a little high level. You can use Jenkins Pipeline as a code for this approach instead of using 3 freestyle jobs.
You can create 3 stages which performs these 3 steps. Pipeline as a code supports reading of properties from different file types (json, yaml etc.)
Look for the "EnvInject" plugin. This lets you inject properties into your build as environment variables; these assignments survive build step boundaries.
If the property file is checked in, you can load it in the Build Environment section before the build steps start executing. If the property file is generated during the build sequence, you can add a build step between where the property file is created and where it is used.
Once set, if the property file contains "FOO=/path/to/folder" then in configuring Jenkins things you would refer to $FOO or ${FOO} (for example, an Ant build step might specify "${FOO}/build.xml"; in Windows batch script execution FOO shows up as an environment variable and is referenced by %FOO% (i.e., "#echo Some_Useful_Piece_Of_Data > %FOO%\data.txt"
More information can be found here: https://wiki.jenkins.io/display/JENKINS/EnvInject+Plugin
I have a jenkins job that pulls source code from GitHub public repo. I need to pass some files such as instance-specific configuration files containing secrets to the job and merge with source code prior to running build because these files are obviously inappropriate to be put in public SCM. The Jenkins instance is a multi-tenanted shared service.
The config files don't change often so I don't want to implement using file parameter which forces user manually input the file on every run. Another reason file parameter doesn't work is some builds are triggered automatically by SCM.
I don't want to use Config File Provider Plugin either, because the plugin requires jenkins admin access but I want users with job-level privileges manage the files themselves.
Ideally the uploaded files are saved alongside with job config.xml instead of in workspace, because I would like to delete workspace after each build. I can write scripts to copy the files from job config folder to workspace.
Are there any solutions available? Thanks.
If the "special" files are being placed in a folder with say some access privileges to it, couldn't you either run a Pre-SCM-Buildstep to move the files with shell commands, or introduce a regular build step (i.e. after the SCM stuff and before the other build steps) that would also use shell commands to move files?
I have a multi-configuration Jenkins job which runs my tests in parallel on multiple slave nodes. I want to collect some artifacts from those slave builds back on the master, and then run a post build step to build a custom HTML page from them.
I have everything set up so those files appear to be copied back onto the master, but then I can't seem to access them. Does anybody have a working example of this configuration that they can share?
Here's the job configuration of the matrix axes:
and the build, which just creates a tmp.txt file:
and fingerprint, archiving artifacts and then post build step to copy that file back to the master:
The slave Console Output says the copy succeeded (and we can see from the plugin source code that any error would be reported - and I've definitely seen these errors getting logged when things go wrong):
17:39:23 [copy-to-slave] Copying '**/tmp.txt', excluding nothing, from 'file:/scratch/jenkins/workspace/Test%20Matrix%20Build%202/label/android/node_index/1/' on 'com.cloudbees.jenkins.plugins.mtslavescloud.MansionSlave#c9964fd8' to 'file:/home/<myaccount>/hudson_home/workspace/Test%20Matrix%20Build%202/label/android/node_index/1/' on the master.
However, I can't seem to access those files from a Post Build script running on the master:
17:39:24 + cat label/android/node_index/1/tmp.txt
17:39:24 cat: label/android/node_index/1/tmp.txt: No such file or directory
The files seem to be listed in the Workspace of the master:
I tried versions of the same command using the $WORKSPACE environment variable and they didn't work either, e.g.
20:40:13 + cat '/scratch/jenkins/workspace/Test Matrix Build 2/label/android/node_index/1/tmp.txt'
20:40:13 cat: /scratch/jenkins/workspace/Test Matrix Build 2/label/android/node_index/1/tmp.txt: No such file or directory
...and since "ws" was in the URL I tried adding that in as well (also didn't work):
20:42:51 + cat '/scratch/jenkins/workspace/Test Matrix Build 2/ws/label/android/node_index/1/tmp.txt'
20:42:51 cat: /scratch/jenkins/workspace/Test Matrix Build 2/ws/label/android/node_index/1/tmp.txt: No such file or directory
Currently I'm using the url with lastFailedBuild to display png (capturing page errors) in the browser (jenkins report) : http ://-jenkins-/job/jobName/lastFailedBuild/artifact/screenshots/Fail1.png/
But the problem is : I would be able to store as many artifacts as maximum build I set in the conf. So I would like to keep the artifacts for each build. We can already do it, but the url contains the build number (and I'd like to avoid manipulating the path to get back the build number). Is there a jenkins alias, a plugin, or can we use wildcards for that?
Have an alias like currentBuild wich returns the build number would be perfect (and simple). By 'current build', I mean to refer to the number build report I'm watching, not the last build report.
Also, where are physically stored the artifacts? I mean, I know where the files are stored, in the workspace, but for artifacts the url displays /job/, where is the folder contening artifacts in my jenkins server? Is it a sort of symbolic link to files in workspace?
I wonder : if I delete the image (in the workspace) before each new build, will it keep the previous artifacts ? I think it's yes because when I overwrite a png image the artifact is kept (it seems to me).
I think this topic : aliasing jenkins artifact URLs doesn't answer my question.
More details :
Here my current report, now I want to refer to http ://-jenkins-/job/JDN/55/artifact/screenshots/Fail1.png/ if I'm on build report #55, or http ://-jenkins-/job/JDN/50/artifact/screenshots/Fail1.png/ if I'm on build report #50.
I could do it in my script looking for the last number build but it's a little heavy. I'd like to know if Jenkins manages that, like lastFailedBuild, lastSuccessfulBuild alias. -> an alias which refers to the artifacts of the observed report. -> it could be something as : http ://-jenkins-/job/JDN/currentReportNumber/artifact/screenshots/Fail1.png/
There is permalink to /lastBuild, which was what I think you mean by "current build"
You can also add /buildNumber to any permalink to get just the value of the build number, for example /lastBuild/buildNumber will return the numeric value of the last executed build, while /lastFailedBuild/buildNumber will return the numeric value of the last failed build.
Physically, the artifacts are stored on the server alongside your WORKSPACE. Under $JENKINS_HOME (or %JENKINS_HOME% for Windows), look for /jobs/<jobname>. There you will see
- config.xml (your job configuration)
- nextBuildNumber (contains the next build number, don't modify this)
- workspace folder (this is the job's WORKSPACE that Jenkins uses during build
- builds folder (this is the history of all your retained builds)
Open the builds folder, and you will see all your saved builds (in time-stamped folders), including symlinks representing the permalinks (such as lastFailedBuild, etc). Under each time-stamped folder, you will see archive folder. This is where the archived artifacts are stored.
To access the WORKSPACE files through the URL use http://<jenkins-url>/job/<job-name>/ws/<path-to-files>
Jenkins has a list of variables you can use , see Jenkins Set Environment Variables. So in the shell script launched by the job; you can see the build number with echo $BUILD_NUMBER and use it. Or directly use BUILD_URL.
I obtained the specific url for each build with that.
I am new to Jenkins and just started configuring it. This is what i have done till now:
Installed and configured Jenkins to display the home page. Added PMD plugin.
Set the HUDSON_HOME to a specific directory > C:\Work\Jenkins
Configured a test build to run a simple do-nothing ant script. It runs successfully
Written an independent pmdbuild.xml to run checks on a set of files in C:\myview (I am using clearcase). This xml also copies the output pmd_results.xml to the workspace directory in $HUDSON_HOME/[job-name]/workspace
Now I added the pmdbuild.xml as a step in my primary build. So my build has 2 steps:
a. Run a simple script, do-nothing.
b. Run pmdbuild.xml which generate pmd_results.xml and place it in $HUDSON_HOME/[job-name]/workspace (HARD-CODED as Jenkins PMD plugin expects the file there)
Jenkins picks up the pmd_results.xml automatically with the plugin and displays warnings and everything.
Now the problem:
If I click on a filename in the PMD results, it gives a filenotfound exception as it is looking for the source file in $HUDSON_HOME/[job-name]/workspace.
My java code files are placed in C:\myview (a clearcase snapshot view)
My question is, do I need all my code files to be present inside $HUDSON_HOME/[job-name]/workspace ?? Meaning can't I tell Jenkins to look for the PMD input files in C:\myview or any other directory instead of $HUDSON_HOME/[job-name]/workspace ??
Sorry for the extremely long description.
Jenkins expects that all the code is in the workspace. Usually Jenkins is used to check out a copy of the code into the workspace, and then runs all build steps on the Sources in the Workspace.
Might seem restraining at first, but it saves you a lot of trouble if you need to move Jenkins to another server, or create a slave instance.
So I would suggest you let Jenkins check out your code (there should be a clearcase plugin) into the workspace, and run the analysis on the checked out code.
If there are compelling reasons why your code has to stay where it is (C:\myview in your case) you can still set the workspace of your build to that directory (find this in the job configuration page, you need to click on the 'extended' button to see the option).