In my release definition, I want to run a single PS script which lives in source control (TFVC in my case). I don't see a way to do this without TFS pulling down the entire source tree containing the one script on the agent machine. I currently have an unversioned copy of the script out on the agent machine, and I reference its absolute path from the release definition. This works, but I'm not guaranteed the latest version of this script to be run at release time.
You have at least two way to do it:
define a mapping that picks only what you need -- you can define a mapping up to a single file, e.g. cloak $/ and map $/path_to_my_file
use a dummy build that collects the file you need and save them as artifacts, I explained this technique in http://blog.casavian.eu/2017/03/04/mixing-tfvc-and-git/
Related
I want my Cloud Build to push an image to a registry with an incremented tag. So, when the trigger arrives from GitHub, build the image, and if the latest tag was 1.10, tag the new one 1.11. Similarly, the 1.11 value will serve in multiple other steps in the build.
Reading the registry and incrementing the tag is easy (in a bash Cloud Build step), but Cloud Build has no way to pass parameters. (Substitutions come from outside the Cloud Build process, for example from the Git tags, and are not generated inside the process.)
This StackOverflow question and this article say that Cloud Build steps can communicate by writing files to the workspace directory.
That is clumsy. But worse, this requires using shell steps exclusively, not the native docker-building steps, nor the native image command.
How can I do this?
Sadly you can't. The Cloud Builder image have each time their own sandbox and only the /workspace directory is mounted. By the way, all the environment variable, binaries installed and so, doesn't persist from one container to the next one.
You have to use the shell script each time :( The easiest way is to have a file in your /workspace directory (for example env.var file)
# load the environment variable
source /workspace/env.var
# Add variable
echo "NEW=Variable" >> /workspace/env.var
For this, Cloud Build is boring...
I wonder if it is possible to remove only one build (including artifacts) from job workspace.
I tried to "Delete Build" in Build History but all it does is remove build reference from Build History table. I know I can ssh to a server and delete files from the command line but I am looking for a way to do it from Jenkins web interface.
After installing Workspace Cleanup Plugin I am able to wipe out current workspace but I want to keep my other builds in the workspace.
In your Jenkins instance, to be able to have folder/per build - set flag "Use custom workspace" in your job's settings. Here is a brief help info from the setting description:
For each job on Jenkins, Jenkins allocates a unique "workspace directory."
This is the directory where the code is checked out and builds happen.
Normally you should let Jenkins allocate and clean up workspace directories,
but in several situations this is problematic, and in such case, this option
lets you specify the workspace location manually.
One such situation is where paths are hard-coded and the code needs to be
built on a specific location. While there's no doubt that such a build is
not ideal, this option allows you to get going in such a situation.
...
And your custom directory path would look like this:
workspace\$JOB_NAME\$BUILD_NUMBER ~> workspace\my-job-name\123
where $JOB_NAME will be "my-job-name" and $BUILD_NUMBER is the build number, eq. "123".
There is one nasty problem with this approach and this is why I wouldn't recommend to use it - Jenkins will not be able to reclaim disk space for outdated builds. You would have to handle cleanup of outdated builds manually and it is a lot of hassle.
Alternative approach, that gives you more control, tools and is able to keep disk space usage under control (without your supervision) is to use default workspace settings and archive your build output (files, original source code, libraries and etc.) as a post-build action. Very-very handy and gives you access to a whole bunch of great tools like, Copy Artifact Plugin or ArtifactDeployer Plugin in other jobs.
Hope that info helps you make a decision that fits your needs best.
I also use "General/Advanced/Use custom workspace" (as in #pabloduo's answer) on a Windows machine with something like:
C:\${JOB_NAME}\${BUILD_NUMBER}
Just wanted to add a solution for getting rid of the build job's workspaces.
I use Groovy Events Listener Plugin for this.
Using the plug-in's standard configuration I just use the following Groovy script:
if (event == Event.JOB_DELETED){
new File(env.WORKSPACE).deleteDir()
}
And now the custom workspace is deleted when the build job is deleted.
Just be aware that this would also delete non-custom workspaces (because the event is triggered for all jobs on your Jenkins server).
I'm working on a multiconfiguration job(Regression_L1) in Jenkins whose task is to run 2 kinds of tests(test1 and test2). In the multiconfiguration job, it triggers an executor job(Regression_executor) to run script for the selected test. The Regression_L1 job is restricted to run in matrix_service_jobs node, while the matrix jobs are to run in the slave node custom_matrix_service_jobs node. The Regression_executor job is restricted to run in remote machines with a specific label(RL1_Test_Machine).
My goal is to test custom build from developers. And so I added a File Parameter(config - File Location: CUSTOMBUILD/mybuild.zip) for the job. The question is how can I access the uploaded file?
Some important info:
Regression_executor's workspace: /home/regressionexec/
Regression_L1's workspace: /var/work/matrix_service_jobs/Regression_L1
Regression_l1 matrix workspaces: /var/work/workspace_user_matrix/workspace/Regression_L1/TEST_PHASE/test1/label/custom_matrix_service_jobs/ and /var/work/workspace_user_matrix/workspace/Regression_L1/TEST_PHASE/test2/label/custom_matrix_service_jobs/
$JENKINS_HOME: var/work/jenkins_home
I did not know where to find the uploaded files so I did a search ung linux find. The result is:
/var/work/jenkins_home/Regression_L1/TEST_PHASE/test2/label/custom_matrix_service_jobs/builds/${BUILD_NUMBER}/fileParameters/CUSTOMBUILD/mybuild.zip.
How can I copy it to the slave node that executes the test script?
Whatever you enter under "File location", that would be the location and the variable that holds the original filename of the uploaded file.
However, on *nix, neither / nor . are valid variable name characters, so in your case, if "File Location" is CUSTOMBUILD/mybuild.zip system cannot create a variable ${CUSTOMBUILD/mybuild.zip}
The file though is still placed under ${WORKSPACE}/CUSTOMBUILD/mybuild.zip. You can access it with this path too.
You can then use Copy To Slave plugin, to copy the file from master to your slaves
I couldn't find my uploaded file under the WORKSPACE, so I ended using something like
"%JENKINS_HOME%\jobs\%JOB_NAME%\builds\%BUILD_ID%\fileParameters\myUploadedFile"
Currently I'm using the url with lastFailedBuild to display png (capturing page errors) in the browser (jenkins report) : http ://-jenkins-/job/jobName/lastFailedBuild/artifact/screenshots/Fail1.png/
But the problem is : I would be able to store as many artifacts as maximum build I set in the conf. So I would like to keep the artifacts for each build. We can already do it, but the url contains the build number (and I'd like to avoid manipulating the path to get back the build number). Is there a jenkins alias, a plugin, or can we use wildcards for that?
Have an alias like currentBuild wich returns the build number would be perfect (and simple). By 'current build', I mean to refer to the number build report I'm watching, not the last build report.
Also, where are physically stored the artifacts? I mean, I know where the files are stored, in the workspace, but for artifacts the url displays /job/, where is the folder contening artifacts in my jenkins server? Is it a sort of symbolic link to files in workspace?
I wonder : if I delete the image (in the workspace) before each new build, will it keep the previous artifacts ? I think it's yes because when I overwrite a png image the artifact is kept (it seems to me).
I think this topic : aliasing jenkins artifact URLs doesn't answer my question.
More details :
Here my current report, now I want to refer to http ://-jenkins-/job/JDN/55/artifact/screenshots/Fail1.png/ if I'm on build report #55, or http ://-jenkins-/job/JDN/50/artifact/screenshots/Fail1.png/ if I'm on build report #50.
I could do it in my script looking for the last number build but it's a little heavy. I'd like to know if Jenkins manages that, like lastFailedBuild, lastSuccessfulBuild alias. -> an alias which refers to the artifacts of the observed report. -> it could be something as : http ://-jenkins-/job/JDN/currentReportNumber/artifact/screenshots/Fail1.png/
There is permalink to /lastBuild, which was what I think you mean by "current build"
You can also add /buildNumber to any permalink to get just the value of the build number, for example /lastBuild/buildNumber will return the numeric value of the last executed build, while /lastFailedBuild/buildNumber will return the numeric value of the last failed build.
Physically, the artifacts are stored on the server alongside your WORKSPACE. Under $JENKINS_HOME (or %JENKINS_HOME% for Windows), look for /jobs/<jobname>. There you will see
- config.xml (your job configuration)
- nextBuildNumber (contains the next build number, don't modify this)
- workspace folder (this is the job's WORKSPACE that Jenkins uses during build
- builds folder (this is the history of all your retained builds)
Open the builds folder, and you will see all your saved builds (in time-stamped folders), including symlinks representing the permalinks (such as lastFailedBuild, etc). Under each time-stamped folder, you will see archive folder. This is where the archived artifacts are stored.
To access the WORKSPACE files through the URL use http://<jenkins-url>/job/<job-name>/ws/<path-to-files>
Jenkins has a list of variables you can use , see Jenkins Set Environment Variables. So in the shell script launched by the job; you can see the build number with echo $BUILD_NUMBER and use it. Or directly use BUILD_URL.
I obtained the specific url for each build with that.
I have set my workspace directory to C:\jenkins_builds\workspace and I want to build ProjA and ProjB, each having a local workfolder (same as project name).
When fetching the source code from my repository, the first two things the TFS plugin does are:
tf workspace -new %workspace-name-A%;%user-name% -server:%my-server%
tf workfold -map $%branch% ProjA -workspace:%workspace-name-A% -server:%my-server%
Which goes fine when building ProjA. The problem is, the first command maps the root directory from the repository directly to my C:\jenkins_builds\workspace directory. The second command does what I actually want, i.e. mapping %branch% to the ProjA subfolder. Later on, when building ProjB, the first command fails (and consequently the build) with the following error message:
The path C:\jenkins_builds\workspace is already mapped in workspace %workspace-name-A%;%user-name%.
OK, it seems like a bad idea to map the root directory to the work directory. But why does this automatically happen when the TFS plugin runs the workspace new line? Currently I have to clean things up between building ProjA and ProjB by running the -unmap command.
My team is using Team Foundation 3.0.
We have the same situation and there are 2 ways to solve this:
use different workspace-root-directories for the two builds
This results in the need for two checkouts => double the space and slower, but better isolation between the two builds
"hardcode" the workspace name to the same for both builds
By default jenkins creates a workspace containing the build name, which can be changed in the "advanced" section of the TFS config, and then you can use the same workspace-/workfolder-mapping for several builds - in our case we called them ProjectName_${NODE_NAME} so it even works on several nodes