How would you go about accessing contents of the build (console) log from within a running build script?
I have a deploy script that runs, logs into a series of servers and runs scripts on those servers. I need to obtain certain output from some of those remote scripts and use them later in the build process and also in the completion email.
You can do something like this in the Post Build section, but I don't think you can do it earlier in the job. With the groovy post build plugin you can get information from the console log:
if(manager.logContains("text to find")) {
do something
}
Related
I'm experiencing some odd behavior with a Jenkins build (Jenkins project is a multi-branch pipeline with the Jenkinsfile provided by the source repository). The last step is to deploy the application which involves replacing an artifact on a remote host and then restarting the process that runs it.
Everything works perfectly except for one problem - the service is no longer running after the build completes. I even added some debugging messages after the restart script to prove with the build output that it really was working. But for some reason, after the build exits the service is no longer running. I've done extensive testing to ensure Jenkins connects to the remote host as the correct user, has the right env vars set, etc. Plus, the restart script output is very detailed in the first place - there would be no way to get the successful output if it didn't actually work. So I am assuming the process that runs the deploy steps on the remote host is doing something else after the build completes execution.
Here is where it gets weird: if I run the same exact deploy commands using the Script Console for the same exact remote host, it works. And the service isn't stopped after successfully starting up.
By "same exact" I mean the script is the same, but the DSL is different between the Script Console and the pipeline. For example, in the Script Console, I use
println "deployscript.sh <args>".execute().text
Whereas in the pipeline I use
pipeline {
agent {
node 'mynode'
}
stages {
/*other stages commented out for testing*/
stage('Deploy') {
steps {
script {
sh 'deployscript.sh <args>'
}
}
}
}
}
I also don't have any issues running the commands manually via SSH.
Does anyone know what is going on here? Is there a difference in how the Script Console vs the Build Agent connects to the remote host? Do either of these processes run other commands? I understand that the SSH session is controlled by a Java process, but I don't know much else about the Jenkins implementation.
If anyone is curious about the application itself, it is a Progress Application Server for OpenEdge (PASOE) instance. The deploy process involves un-deploying the old WAR file, deploying the new one, and then stopping/starting the instance.
UPDATE:
I added 60-second sleep to the end of the deploy script to give me time to test the service before the Jenkins process ended. This was successful, so I am certain that when the Jenkins build process exits is when it causes the service to go down. I am not sure if this is an issue with Jenkins owning a process, but again the Script Console handles this fine...
Found the issue. It's buried away in some low-level Jenkins documentation, but Jenkins builds have a default behavior of killing any processes spawned by the build. This confirms that Jenkins was the culprit and the build indeed was running correctly. It was just being killed after the build completed.
The fix is to set the value of the BUILD_ID environment variable (JENKINS_NODE_COOKIE for pipeline, like in my situation) to "dontKillMe".
For example:
pipeline {
agent { /*set agent*/ }
environment {
JENKINS_NODE_COOKIE="dontKillMe"
}
stages { /*set build stages*/ }
}
See here for more details: https://wiki.jenkins.io/display/JENKINS/ProcessTreeKiller
I'm trying to implement polling of BrowserStack from Jenkins in order to keep the build execution open: https://www.browserstack.com/docs/automate/cypress/polling-callback
During the build execution the console logs print the BrowserStack Build ID, and then the tests start:
12:27:38 Visit the Automate dashboard for test reporting: https://automate.browserstack.com/dashboard/v2/builds/abc123
I need to extract this Build ID 'abc123' to use later.
How can I extract this info, set it as a variable and use again in the shell command?
Use it like so: $ browserstack-cypress build-info <buildId>
https://www.browserstack.com/docs/automate/cypress/cli-reference#get-the-build-information
You can use the callback_url key in run_settings option in browserstack.json to get an update after that build is done running. The payload POSTed to this callback URL is the same as the one you get with the build-info BUILD_ID command.
Protip: You can keep polling the build status using the build-info command in a loop to monitor the build status, and then close the Local connection, and fail the build if the tests failed.
The callback URL needs to be the URL on which the status of the build will be posted after the build is completed. You can generate a callback URL from 'https://webhook.site/' for testing. You will also get the build ID in the Payload.
I have built a Jenkins job to run automated ZAP-Proxy scans.
I used curl -X POST -u YOUR_USER:YOUR_USER_PASSWORD http://YOUR_JENKINS_URL/job/YOUR_JOB to build the job from the terminal. Is there a way to display the console output in the terminal while the job is building?
It is possible, but it's a little more complicated for two reasons:
When you trigger a new build via curl, then the build will not start immediately. The build will enter the build queue, and it will only start executing once Jenkins found a suitable executor. Before that time, there is no build URL at all.
Technically, the (continuous) console output is delivered in multiple fragments that must be retrieved individually via HTTP.
So, once you triggered the build, you need to find its URL after if left the build queue. It can be done nicely in Groovy -- as a simpler heuristic, you could just wait for certain time and then use the lastBuild reference.
For fetching the console log fragments, you'll use the <buildUrl>/logText/progressiveText end point. Create a loop fetching that URL and checking the X_More_Data and X_Text_Size HTTP headers for information on whether (and what) console output is available. You can do that in bash with curl in a loop; I found this Groovy example on the Web.
In the end, the most elegant solution is probably to
trigger a new build by submitting a Groovy script that will trigger the build and then waits for the build to leave the build queue, returning the build URL.
Then use another script that will poll/fetch/display that build's console output.
If you use the CLI interface for submitting the script, then you can do all those steps in a single script. If you use the REST API, then second part ("continuous output") probably won't work due to output buffering on REST API side.
We want to have a pipeline, that builds our application than pauses and after the built application was manually tested resumes and delivers the tested application.
So I came up with the idea of using a Input to pause the pipeline like this:
...
stage ("Build"){
// build application here and archive it as artefact
}
timeout(time:5, unit:'DAYS') {
input message:'Approve deployment?'
}
stage ("Deliver"){
// deliver the built application
}
The tester got 5 days to test the application then resumes the pipeline and it gets delivered.
My problem here is, while the build is still running, the tester can't yet access the artifact on the status page.
So is there any way to provide any kind of Download-Link in the log output, that points to the application file I archived in the build stage?
Or is there any other good way to achieve this build->pause->test->resume->deliver workflow in one single pipeline job?
Automation of the test in the pipeline is not an option, as the application needs to be manually flashed on some hardware.
This will get you to the artifacts list (you can add more after artifact if you want the link to point to the specific file):
...
timeout(time:5, unit:'DAYS') {
echo "Archive available for download: ${env.BUILD_URL}artifact"
input message:'Approve deployment?'
}
This requires the value of JENKINS_URL to be set in the system configuration: From your Jenkins home, click on Manage Jenkins --> Configure System and look for the Jenkins URL under Jenkins Location
If you don't have admin access to Jenkins, and JENKINS_URL is not set, you could fudge this with something like
https://known-jenkins-url/job/${JOB_NAME}/${BUILD_NUMBER}/artifact
Is it possible to run a Nant script as a Jenkins post build action?
I get the option to run a script as a Build Step but not as a build action. Is there any particular plugin which enables this option.
The reason I am looking for this functionality is that I need to run a script which depends on the ArtifactDeployer post build action. If i specify the code in the build step it gets executed before the ArtifactDeployer and the build fails
You can use the Post Build Task Plugin
edit
One way of getting the build number if it's not working with this plugin is using the Groovy Post Build Plugin
With it you can execute groovy code as a post build action, get the build number and execute NAnt
the build number is accessible from the following property
manager.build.number
Post-build Actions -> Execute a set of script run after a build when it succeeds or fails. My experience shows that it only sometimes runs when a build is aborted.
As advised above, the Post-build Actions -> Post build task (via the named post task plug in) is always evaluated for run (regardless of the build exit status). Additional setting via phrase in the log ("Build was aborted") works reliable for me.
My problem was to run something even on an aborted build and post build task sorted out this problem.