I have a Jenkins job, which do Git syncs and build the source code.
I added and created a "Post build task" option.
In 'post build task', I am searching for keyword "TIMEOUT:" in console output (this part is done) and want to declare job as Failed and Aborted if keyword matches.
How can I raise / declare the Job as Aborted from batch script if keyword matches. Something like echo ABORT?
It is easier if you want mark it as "FAIL"
Just exit 1 will do that.
It is tricky to achieve "Abort" from post build task plugin, it is much easier to use Groovy post build plugin.
The groovy post build provide rich functions to help you.
Such as match function:
def matcher = manager.getLogMatcher(".*Total time: (.*)\$")
if(matcher?.matches()) {
manager.addShortText(matcher.group(1), "grey", "white", "0px", "white")
}
Abort function:
def executor = build.executor ?: build.oneOffExecutor;
if (executor != null){
executor.interrupt(Result.ABORTED)
}
Br,
Tim
you can simply exit the flow and raise the error code that you want:
echo "Timeout detected!"
exit 1
Jenkins should detect the error and set-up the build as failed.
The error code must be between 1 and 255. You can chose whatever your want, just be aware that some code are reserved:
http://tldp.org/LDP/abs/html/exitcodes.html#EXITCODESREF
You can also consider using the time-out plugin:
https://wiki.jenkins.io/display/JENKINS/Build-timeout+Plugin
And another option is to build a query to BUILD ID URL/stop. Which is exactly what is done when you manually abort a build.
echo "Timeout detected!"
curl yourjenkins/job_name/11/stop
Related
I am trying to implement Machine learning in my jenkins pipeline.
For that I need output data of pipeline for each build.
Some parameters that i need are:
Which user triggered the pipeline
Duration of pipeline
Build number with its details
Pipeline pass/fail
If fail, at which stage it failed.
Error in the failed stage. (Why it failed)
Time required to execute each stage
Specific output of each stage (For. eg. : If a stage contains sonarcube execution then output be kind of percentage of codesmells or code coverage)
I need to fetch these details for all builds. How can get it?
There is jenkins api that can be implemented in python but i was able to get only JOB_NAME, Description of job, IS job Enabled.
These details werent useful.
There are 2 ways to get some of data from your list.
1. Jenkins API
For first 4 points from the list, you can use JSON REST API for a specific build to get those data. Example API endpoint:
https://[JENKINS_HOST]/job/[JOB_NAME]/[BUILD_NUMBER]/api/json?pretty=true
1. Which user triggered the pipeline
This will be under actions array in response, identyfi object in array by "_class": "hudson.model.CauseAction" and in it you will have shortDescription key which will have that information:
"actions": [
{
"_class": "hudson.model.CauseAction",
"causes": [
{
"_class": "hudson.triggers.SCMTrigger$SCMTriggerCause",
"shortDescription": "Started by an SCM change"
}
]
},
2. Duration of pipeline
It can be found under key: "duration". Example
"duration": 244736,
3. Build number with its details
I don't know what details you need, but for build number look for "number" key:
"number": 107,
4. Pipeline pass/fail
"result": "SUCCESS",
If you need to extract this information for all builds, run GET request for job API https://[JENKINS_HOST]/job/[JOB_NAME]/api/json?pretty=trueand extract all builds, then run above-mentioned request per build you have extracted.
I will write later a dummy python script to do just that.
2. Dump data in Jenkinsfile
There is also a possibility to dump some that information from Jenkinfile in post action.
pipeline {
agent any
stages {
stage('stage 1') {
steps {
sh 'echo "Stage 1 time: ${YOUR_TIME_VAR}" > job_data.txt'
}
}
}
post {
always {
sh 'echo "Result: ${result}" > job_data.txt'
sh 'echo "Job name: ${displayName}" > job_data.txt'
sh 'echo "Build number: ${number}" > job_data.txt'
sh 'echo "Duration: ${duration}" > job_data.txt'
archiveArtifacts artifacts: 'job_data.txt', onlyIfSuccessful: false
}
}
}
List of available global variables for pipeline job can be found:
https://[JENKINS_HOST]/pipeline-syntax/globals#env
For rest, you will need to implement your own logic in Jenkinsfile.
Ad. 5
Create a variable which holds information about current stage. At the beginning of each stage change its value to the ongoing stage. At the end dump to file like rest variables. If pipeline will fail let's say on stage foo in post action this variable will have exact same value because if pipeline fails it won't go to next stage.
Ad. 6
I'm not sure what you want, a traceback, error code?
I guess you will probably need to implement your own logging function.
Ad. 7
Make a function for measuring time for each stage and dump value at the end.
Ad. 8
Also not sure what you mean. Like, build artifacts?
At the end of each build this file job_data.txt will be archived as build artifact which can be later downloaded.
If i will find more elegant and simple solution I'll edit this post.
Hope it helps in any way
EDIT 1
Here is the script I've mentioned earlier.
import requests
username = "USERNAME"
password = "PASSWORD"
jenkins_host = "JENKINS_HOST"
jenkins_job = "JOBNAME"
request_url = "{0:s}/job/{1:s}/api/json".format(
jenkins_host,
jenkins_job,
)
job_data = requests.get(request_url, auth=(username, password)).json()
builds = []
for build in job_data.get('builds'):
builds.append(build.get('number'))
for build in builds:
build_url = "{0:s}/job/{1:s}/{2:d}/api/json".format(
jenkins_host,
jenkins_job,
build,
)
build_data = requests.get(build_url, auth=(username, password)).json()
build_name = build_data.get('fullDisplayName')
build_number = build_data.get('number')
build_status = build_data.get('result')
build_duration = build_data.get('duration')
for action in build_data.get('actions'):
if action.get("_class") == "hudson.model.CauseAction":
build_trigger = action.get('causes')
print(build_name)
print(build_status)
print(build_duration)
print(build_number)
print(build_trigger)
Please note you might need to authorize with API Token depending on your security settings.
Jenkins build gets failed, when unit test cases are failed,
Here am using pipeline script in jenkins, Need to generate HTML report using mocha-awesome, I can get the HTML report only when all test cases are passed, Build fails if any functions failed in my testcases.Here you can see the screenshot
The call to the unit test run returns with exit status 1.
What you can do is use the returnStatus option for the sh or bat step which will then not fail the build itself but leaves that up to you:
def exitStatus = sh returnStatus: true, script: 'unittests'
or:
def exitStatus = bat returnStatus: true, script: 'unittests.exe'
Having this you can selectively fail the build, i.e. if exitStatus == 1 then ignore as it signals a test fail, if it is anything but 0 or 1 then fail the build using the error step.
Is there a way to not fail with a declarative pipeline step but display a warning instead? At the moment I am circumventing it by adding || exit 0 to the end of the sh command lines so it always exits ok.
Example currently:
sh 'vendor/bin/phpcs --report=checkstyle --report-file=build/checkstyle.xml --standard=phpcs.xml . || exit 0'
I feel there should be a better way of doing this? How can you control the result returned so the user knows with out digging through result logs but with out blocking/failing too?
Thank you
Edit:
So I have got this far to be able to mark it as unstable. In Blue Ocean it just marks every stage as unstable though and you have to go digging through to find it. Am I trying to do the impossible (but feels like I should be able to do it)?
Also it just displays it as 'Shell script' in the heading on Blue Sky now instead of showing the command being run so you don't even know what it's doing with out expanding each of them.
script {
def RESULT = sh returnStatus: true, script: 'vendor/bin/phpcs --report=checkstyle --report-file=build/checkstyle.xml --standard=phpcs.xml .'
if ( RESULT != 0 ) {
currentBuild.result = 'UNSTABLE'
}
}
Re marking it all yellow/unstable: Just found this which explains that I am not going mad. Several years of discussion and no progress :( https://issues.jenkins-ci.org/browse/JENKINS-39203
Re it just showing 'Shell script' now on Blue Ocean view: I'll go and play some more and see if I'm better with || exit 0 or a script block and using echo to display some useful info.
FYI: JENKINS-39203 has been resolved in the meantime.
Instead of setting currentBuild.result = 'UNSTABLE' we use the corresponding basic pipeline step unstable, which also takes a message:
unstable("There are Checkstyle issues")
You can find options for any steps in your jenkins itself under http://jenkins-url/pipeline-syntax/. sh with returnStatus: true can be used to achieve your goal.
sh returnStatus: true, script: 'ls -2' # returnStatus: true, just set $? but not fail the execution
Below pasted the screenshot from pipeline-syntax page:
I have a Jenkins job with 100+ builds. I need to search through all the builds of that job to find builds that have a certain string in the console output. Is there any plugin for that? How do I do that?
I often use the Jenkins Script Console for tasks like this. The Groovy plugin provides the Script Console, but if you're going to use the Script Console for periodic maintenance, you'll also want the Scriptler plugin which allows you to manage the scripts that you run.
From Manage Jenkins -> Script Console, you can write a groovy script that iterates through the job's builds looking for the matching string:
JOB_NAME = "My Job"
BUILD_STRING = "Hello, world"
def job = Jenkins.instance.items.find { it.name == JOB_NAME }
for (build in job.builds) {
def log = build.log
if (log.contains(BUILD_STRING)) {
println "${job.name}: ${build.id}"
}
}
If there is no additional requirements I would do it simply in the shell, e.g.:
find $JENKINS_HOME/jobs/haystack -name log -exec grep -l needle {} \; \
| sed 's|.*/\(.*\)/log|\1|'
To search in logs of all jobs:
I enhanced #DaveBacher 's code to be run in the Jenkins script console.
Helped me to locate a sporadic error happening in multiple jobs.
NEEDLE = "string_i_am_looking_for"
for (job in Jenkins.instance.getAllItems(Job.class)) {
for (build in job.builds) {
def log = build.log
if (log.contains(NEEDLE)) {
println "${job.name}: ${build.id}"
}
}
}
Thanks everyone for your valuable solutions. After a bit of additional research i found that there is a plugin in Jenkins to do this.
https://wiki.jenkins-ci.org/display/JENKINS/Lucene-Search
This will save the console output results and users can do search in search box.
There is the Log Parser Plugin
highlighting lines of interest in the log (errors, warnings,information)
dividing the log into sections displaying a summary of number of errors, warnings and information lines within the log and its sections.
linking the summary of errors and warnings into the context of the full log, making it easy to find a line of interest in the log
showing a summary of errors and warnings on the build page
If it is old logs then #jil has the answer assuming you are on Linux.
Just to throw another plugin out there, this blog post pointed me at the TextFinder plugin which allows you to search for text in either a workspace file or the console output, and override the build status as success/failure when the text is found.
The original poster doesn't say what should happen when the text is found, but it was searching for this functionality that brought me here.
To search for a regex pattern in all Jenkins jobs, and print the first matching line:
for (job in Jenkins.instance.items) {
for (build in job.builds) {
try {
def log = build.log
def match = log =~ "\n(.*${PATTERN}.*)\n"
if (match) {
println "Job [${job.name}] - Build [${build.id}]: ${match[0][0]}"
}
}
catch (Exception e) {
println e
}
}
}
For example, searching in my builds for PATTERN = "(TLS|Build).*timeout" I found:
Job [OSP-AWS] - Build [83]: Build timeout: dial tcp
[::1]:6443: connect: connection refused
Job [OSP-GCP] - Build [21]: Unable to
connect to the server: net/http: TLS handshake timeout
Just use Jenkins std search (top right corner) with keyword "console":
console:"whatever you are looking for"
I would like to have a post-build hook or similar, so that I can have the same output as e. g. the IRC plugin, but give that to a script.
I was able to get all the info, except for the actual build status. This just doesn't work, neither as a "Post-build script", "Post-build task", "Parameterized Trigger" aso.
It is possible with some very ugly workarounds, but I wanted to ask, in case someone has a nicer option ... short of writing my own plugin.
It works as mentioned with the Groovy Post-Build Plugin, yet without any extra quoting within the string that gets executed. So I had to put the actual functionality into a shell script, that does a call to curl, which in turn needs quoting for the POST parameters aso.
def result = manager.build.result
def build_number = manager.build.number
def env = manager.build.getEnvironment(manager.listener)
def build_url = env['BUILD_URL']
def build_branch = env['SVN_BRANCH']
def short_branch = ( build_branch =~ /branches\//).replaceFirst("")
def host = env['NODE_NAME']
def svn_rev = env['SVN_REVISION']
def job_name = manager.build.project.getName()
"/usr/local/bin/skypeStagingNotify.sh Deployed ${short_branch} on ${host} - ${result} - ${build_url}".execute()
Use Groovy script in post-build step via Groovy Post-Build plugin. You can then access Jenkins internals via Jenkins Java API. The plugin provides the script with variable manager that can be used to access important parts of the API (see Usage section in the plugin documentation).
For example, here's how you can execute a simple external Python script on Windows and output its result (as well as the build result) to build console:
def command = """cmd /c python -c "for i in range(1,5): print i" """
manager.listener.logger.println command.execute().text
def result = manager.build.result
manager.listener.logger.println "And the result is: ${result}"
For this I really like the Conditional Build Step plugin. It's very flexible, and you can choose which actions to take based on build failure or success. For instance, here's a case where I use conditional build step to send a notification on build failure:
You can also use conditional build step to set an environment variable or write to a log file that you use in subsequent "execute shell" steps. So for instance, you might create a build with three steps: one step to compile code/run tests, another to set a STATUS="failed" environment variable, and then a third step which sends an email like The build finished with a status: ${STATUS}
Really easy solution, maybe not to elegant, but it works!
1: Catch all the build result you want to catch (in this case SUCCESS).
2: Inject an env variable valued with the job status
3: Do the Same for any kind of other status (in this case I catch from abort to unstable)
4: After you'll be able to use the value for whatever you wanna do.. in this case I'm passing it to an ANT script! (Or you can directly load it from ANT as Environment variable...)
Hope it can help!
Groovy script solution:-
Here I am using groovy script plugin to take the build status and setting it to the environmental variable, so the environmental variable can be used in post-build scripts using post-build task plugin.
Groovy script:-
import hudson.EnvVars
import hudson.model.Environment
def build = Thread.currentThread().executable
def result = manager.build.result.toString()
def vars = [BUILD_STATUS: result]
build.environments.add(0, Environment.create(new EnvVars(vars)))
Postscript:-
echo BUILD_STATUS="${BUILD_STATUS}"
Try Post Build Task plugin...
It lets you specify conditions based on the log output...
Basic solution (please don't laugh)
#!/bin/bash
STATUS='Not set'
if [ ! -z $UPSTREAM_BUILD_DIR ];then
ISFAIL=$(ls -l /var/lib/jenkins/jobs/$UPSTREAM_BUILD_DIR/builds | grep "lastFailedBuild\|lastUnsuccessfulBuild" | grep $UPSTREAM_BUILD_NR)
ISSUCCESS=$(ls -l /var/lib/jenkins/jobs/$UPSTREAM_BUILD_DIR/builds | grep "lastSuccessfulBuild\|lastStableBuild" | grep $UPSTREAM_BUILD_NR)
if [ ! -z "$ISFAIL" ];then
echo $ISFAIL
STATUS='FAIL'
elif [ ! -z "$ISSUCCESS" ]
then
STATUS='SUCCESS'
fi
fi
echo $STATUS
where
$UPSTREAM_BUILD_DIR=$JOB_NAME
$UPSTREAM_BUILD_NR=$BUILD_NUMBER
passed from upstream build
Of course "/var/lib/jenkins/jobs/" depends of your jenkins installation