How to make a Java console output a environment variable in Jenkins? - jenkins

I am new to Jenkins and even though I found a few similar questions, none of the solutions seemed to work for me the way I need it to. It might look like a basic problem to some but for me it's a very big deal that I'm struggling with.
Basically, I built a project that executes Java Selenium code, which displays session ID in Jenkins' Console Output and that's what I need to add to environment variables to be used in the projects triggered after completion of this one.
I tried some Groovy scripts but I don't think I understand enough how to work with it and so whatever I was given, wasn't what I hoped to get.
Has anyone done something similar to provide some tips on how to achieve that?
Many thanks

There are two options (in theory, one of them doesn't work, see 2. below) depending on whether the printing is under your control or not.
Printing is under your control:
Write the session ID to a properties file, e.g. from-build-log.properties:
sessionId=...
Add post-build action → Trigger parameterized build on other projects →
This plugin triggers builds on other projects, with parameters that are predefined, or supplied by the finished build.
Every parameter will be passed to the target project(s), even if the target is not parameterized, or if no property of that name is defined.
Add parameters → Parameters from properties file
Use properties from file: from-build-log.properties
Printing is not under your control:
Add post-build action → Post build task → :
This feature allows you to associate shell or a batch scripts that perform some tasks on Hudson depending on the build log output. If the log text matches somewhere in the build log file, the script will execute. [...]
Java Regex are allowed, and groups can be used as script parameters. If the text is "Last Build : #(\d+)" and the script is "script.sh", then if the log contains a line "Last Build : #4", the script "script.sh 4" will be called.
Tasks → Script → :
[...] References %1, .. %n are allowed, and will be replaced by the groups matched by the Regex. %0 is the whole match.
Unfortunately this doesn't to work since there is an issue known since 2013: [JENKINS-17268] Post build task plugin: Passing arguments does not work as documented.
Build → Execute Windows batch command → Command:
#echo( & echo CMD: sessionId=123456789
Post build task → Tasks:
Log text: sessionId=(\d+)
Script:
#echo( & echo sessionId='%1'(!) of '%0'
Console Output:
...
[Freestyle-project] $ cmd /c call C:\Windows\TEMP\hudson4684581005071706054.bat
CMD: sessionId=123456789
C:\Program Files (x86)\Jenkins\workspace\Freestyle-project>exit 0
Performing Post build task...
Match found for :sessionId=(\d+) : True
Logical operation result is TRUE
Running script : #echo( & echo sessionId='%1'(!) of '%0'
[Freestyle-project] $ cmd /c call C:\Windows\TEMP\hudson1525182929053902824.bat
sessionId=''(!) of 'C:\Windows\TEMP\hudson1525182929053902824.bat'
C:\Program Files (x86)\Jenkins\workspace\Freestyle-project>exit 0
POST BUILD TASK : SUCCESS
END OF POST BUILD TASK : 0
Finished: SUCCESS
%0 is not the "the whole match" but the script's name, as usual with Windows command line. %1 is empty.
A workaround is:
Add build step → Execute shell → Command:
sed -En 's/.*(sessionId=[0-9]+)/\1/p' \
../../jobs/${JOB_NAME}/builds/${BUILD_NUMBER}/log > from-build-log.properties
Add post-build action → Trigger parameterized build on other projects
Add parameters → Parameters from properties file
Use properties from file: from-build-log.properties

Related

jenkins parameter From Properties file

I have 3 Jenkins jobs to be run in serial.
Run a Ant File
Run another ANT File
Run a command line
All the above jobs use a file path which is set in a properties file.
Ex Job 1 , Executes ANT file placed in file path location
Job 2 , Executes another file placed in same file path location
Job 3 , Executes command line to do SVN update in same file path location
I need to parameterize the file path in all three builds from properties file.
Can anyone help me with possible approach?
Thanks In Advance
This answer could be a little high level. You can use Jenkins Pipeline as a code for this approach instead of using 3 freestyle jobs.
You can create 3 stages which performs these 3 steps. Pipeline as a code supports reading of properties from different file types (json, yaml etc.)
Look for the "EnvInject" plugin. This lets you inject properties into your build as environment variables; these assignments survive build step boundaries.
If the property file is checked in, you can load it in the Build Environment section before the build steps start executing. If the property file is generated during the build sequence, you can add a build step between where the property file is created and where it is used.
Once set, if the property file contains "FOO=/path/to/folder" then in configuring Jenkins things you would refer to $FOO or ${FOO} (for example, an Ant build step might specify "${FOO}/build.xml"; in Windows batch script execution FOO shows up as an environment variable and is referenced by %FOO% (i.e., "#echo Some_Useful_Piece_Of_Data > %FOO%\data.txt"
More information can be found here: https://wiki.jenkins.io/display/JENKINS/EnvInject+Plugin

How do I pass information from one step in a TFS 2017 build to a later Copy Files step

OK - I've read all about environment variables and how they can't be set and read by the same process (even this can't be read by a later step in the build):
Environment.SetEnvironmentVariable("Major_Build_Number", BaseReleaseNumber, EnvironmentVariableTarget.Machine)
So has anyone come up with a simple way to pass along info from one build step to another? In my first step I determine the build number (this is a fairly complex process believe it or not) and I need to pass that build number to the last build step (which is a Copy Files step) so that it can copy the build into a folder that's named with the build number. I've tried setting an environment variable, but unfortunately that can't be set and read from the same session. There's gotta be a simple way to do this. Yes I could write a PS or batch script to do it and store the value in a file or the registry, but I would prefer to use the Copy Files task and I can't figure out how to pass that value along.
I tried defining a variable in the build definition and storing the value there but I can't seem to change that value after it's set in the build definition.
BTW, this is an on-premises installation - not VSTS.
Anyone have any ideas?
Thanks Andy for your response. So I tried this in SetBuildNumberENVVar.ps1
param([Int32]$MajorBuildNumber=0,[Int32]$MinorBuildNumber=0)
Write-Host "##vso[task.setvariable variable=MajorBuildNumber]$MajorBuildNumber"
Write-Host "##vso[task.setvariable variable=MinorBuildNumber]$MinorBuildNumber"
I then run it from the command line:
c:\powershell .\SetBuildNumberENVVar.ps1 23 45
and then try to echo the variable:
echo %MajorBuildNumber%
%MajorBuildNumber%
and as you can see it doesn't appear to work. I tried this from a C# script:
int.TryParse("$(MajorBuildNumber)", out mbn);
and mbn = 0 after this runs.
Any idea what I'm doing wrong?
Generally you can use the predefined variale $(Build.BuildNumber) to name the folder, it can be used directly in entire build process. See Predefined variables for details.
If you customed the build number variable as you said : "I determine the build number (this is a fairly complex process believe it or not)". Then you can pass the value with the Logging Command: ##vso[task.setvariable]value:
Add a PowserShell task in you build definition
Copy and paste below script and save it as *.ps1 file
$value= "The value of the build number here"
Write-Host "##vso[task.setvariable variable=BuildNumber]$value"
Check in the PS file, then run the PS file in PowerShell task
After that, you can use the variable $BuildNumber directly in any tasks behind the PowserShell task.
You can reference my answer in another similar thread : Custom TFS Enviroment Variable doesn't read $(Date)
UPDATE:
You need to run the PowerShell script in TFS build process.
See below example:
I created two PS scripts, one to set the variables to pass the value, another to use the variables to create a folder named with the passed values:
PS1: PassBuildNumber
param([Int32]$MajorBuildNumber=0,[Int32]$MinorBuildNumber=0)
Write-Host "##vso[task.setvariable variable=MajorBuildNumber]$MajorBuildNumber"
Write-Host "##vso[task.setvariable variable=MinorBuildNumber]$MinorBuildNumber"
PS2 : Use the variables
Write-Host "The Major build number is:" $env:MajorBuildNumber
Write-Host "The Minor build number is:" $env:MinorBuildNumber
$foldername = $env:MajorBuildNumber + "." + $env:MinorBuildNumber
Write-Host "foldername:" $foldername
$path = "\\myshare\DirA\$foldername"
Write-Host "path:" $path
New-Item -Path $path -ItemType directory # Create a folder
Write-Host "##vso[task.setvariable variable=path]$path" # Set path as a variable to be used in Copy Task
Then you can use Copy Files task to copy files to that folder.

Custom TFS Enviroment Variable doesn't read $(Date)

I want to use a custom tfs variable like this:
MergedVersion: $(BuildVersion.Major).$(BuildVersion.Minor).$(Date:yy)$(DayOfYear)$(Rev:.r)
My problem is that $(Date), $(Rev:r) and $(DateOfYear) don't work outside the BuildNumberFormat-Settings.
My result is:
invalid version string: '1.0.$(Date:yy)$(DayOfYear)$(Rev:.r)'.
While with the buildnumberformat like shown here - works correctly:
Result $(Build.BuildNumber) is MyBuildName_1.0.18004.15
Some tokens are only available in the Build number format section, such as $(Date), $(Rev:r) and $(DateOfYear) you mentioned here. See Build definition options
As a workaround, to use $(Rev:r)you can set the build number format as $(Rev:r), then use the $(Build.BuildNumber) variable in your tasks.
To use $(Date:yy)$(DayOfYear), you can set the variables via PowerShell task as ChamindaC mentioned above.
Add a PowserShell task in you build definition
Copy and paste below script and save it as *.ps1 file
Check in the PS file, then run the PS file in PowerShell task
$time=$(Get-Date -Format 'yy') # you can set the date format based on your requirement
$doy = (Get-Date).DayofYear
Write-Host "##vso[task.setvariable variable=Date]$time"
Write-Host "##vso[task.setvariable variable=DayOfYear]$doy"
Then you can use the variables $(Date) and $(DayOfYear) in other build tasks.
Use following script in a PowerShell Task in your build definition
$date=$(Get-Date -Format 'yy');
Write-Host "##vso[task.setvariable variable=Today]$date"
Then you can use $(Today) in your subsequent build tasks. However, usage like $(Today:yy) with format is not possible as it is supported only in build number format.
Building on top of #ChamindaC and #Andy Li-MSFT's answers, and using Peter Groenewegen's Xpirit Run Inline Powershell and Azure Powershell build extension, I was able to retrieve the $(rev:r) from the build number:
In the following MSBuild task I reference that as $(Revision).

Jenkins "Console Output" log location in filesystem

I want to access and grep Jenkins Console Output as a post build step in the same job that creates this output. Redirecting logs with >> log.txt is not a solution since this is not supported by my build steps.
Build:
echo "This is log"
Post build step:
grep "is" path/to/console_output
Where is the specific log file created in filesystem?
#Bruno Lavit has a great answer, but if you want you can just access the log and download it as txt file to your workspace from the job's URL:
${BUILD_URL}/consoleText
Then it's only a matter of downloading this page to your ${Workspace}
You can use "Invoke ANT" and use the GET target
On Linux you can use wget to download it to your workspace
etc.
Good luck!
Edit:
The actual log file on the file system is not on the slave, but kept in the Master machine. You can find it under: $JENKINS_HOME/jobs/$JOB_NAME/builds/lastSuccessfulBuild/log
If you're looking for another build just replace lastSuccessfulBuild with the build you're looking for.
Jenkins stores the console log on master. If you want programmatic access to the log, and you are running on master, you can access the log that Jenkins already has, without copying it to the artifacts or having to GET the http job URL.
From http://javadoc.jenkins.io/archive/jenkins-1.651/hudson/model/Run.html#getLogFile(), this returns the File object for the console output (in the jenkins file system, this is the "log" file in the build output directory).
In my case, we use a chained (child) job to do parsing and analysis on a parent job's build.
When using a groovy script run in Jenkins, you get an object named "build" for the run. We use this to get the http://javadoc.jenkins.io/archive/jenkins-1.651/hudson/model/Build.html for the upstream job, then call this job's .getLogFile().
Added bonus; since it's just a File object, we call .getParent() to get the folder where Jenkins stores build collateral (like test xmls, environment variables, and other things that may not be explicitly exposed through the artifacts) which we can also parse.
Double added bonus; we also use matrix jobs. This sometimes makes inferring the file path on the system a pain. .getLogFile().getParent() takes away all the pain.
You can install this Jenkins Console log plugin to write the log in your workspace as a post build step.
You have to build the plugin yourself and install the plugin manually.
Next, you can add a post build step like that:
With an additional post build step (shell script), you will be able to grep your log.
I hope it helped :)
Log location:
${JENKINS_HOME}/jobs/${JOB_NAME}/builds/${BUILD_NUMBER}/log
Get log as a text and save to workspace:
cat ${JENKINS_HOME}/jobs/${JOB_NAME}/builds/${BUILD_NUMBER}/log >> log.txt
For very large output logs it could be difficult to open (network delay, scrolling). This is the solution I'm using to check big log files:
https://${URL}/jenkins/job/${jobName}/${buildNumber}/
in the left column you see: View as plain text. Do a right mouse click on it and choose save links as. Now you can save your big log as .txt file. Open it with notepad++ and you can go through your logs easily without network delays during scrolling.
I found the console output of my job in the browser at the following location:
http://[Jenkins URL]/job/[Job Name]/default/[Build Number]/console
This is designed for use when you have a shell script build step. Use only the first two lines to get the file name.
You can get the console log file (using bash magic) for the current build from a shell script this way and check it for some error string, failing the job if found:
logFilename=${JENKINS_HOME}/${JOB_URL:${#JENKINS_URL}}
logFilename=${logFilename//job\//jobs\/}builds/${BUILD_NUMBER}/log
grep "**Failure**" ${logFilename} ; exitCode=$?
[[ $exitCode -ne 1 ]] && exit 1
You have to build the file name by taking the JOB_URL, stripping off the leading host name part, adding in the path to JENKINS_HOME, replacing "/job/" to "/jobs/" to handle all nested folders, adding the current build number and the file name.
The grep returns 0 if the string is found and 2 if there is a file error. So a 1 means it found the error indication string. That makes the build fail.
Easy solution would be:
curl http://jenkinsUrl/job/<Build_Name>/<Build_Number>/consoleText -OutFile <FilePathToLocalDisk>
or for the last successful build...
curl http://jenkinsUrl/job/<Build_Name>/lastSuccessfulBuild/consoleText -OutFile <FilePathToLocalDisk>

Jenkins: Escaping % in parameterized jobs

I'm trying to run this command from shell as a build step on a Windows machine:
for /R . %f in (*.bin) do copy "%f" "..\Release\"
After struggling for a while figuring out why it doesn't work, I noticed this is a parameterized build and that parameters are of the form %var%. I assume is trying to replace % with an empty value since there's nothing defined. It is translated to on the first loop:
for /R . f" "..\Release\Newer\"
I guess I could define a parameter like %f% = %f and it might do the trick, but seems unnecessary.
First, I assume you are using Windows environment (you should really specify), and as such it is not a "shell" build step, but "Execute Windows Batch Command" build step (if it's not, you need to use "Execute Windows Batch Command" build step)
Next, it's a peculiarity of Windows Batch, that when you run it from command line, you can use a single % in the FOR statement, however when you run it from a batch file (and the batch build step is a temporary batch file), you need to use %%. Note, this only applies to the FOR statement as it uses a special syntax for the variables.
So change your line to:
for /R . %%f in (*.bin) do copy "%%f" "..\Release\"

Resources