I have a Jenkins project with a job that gets triggered whenever something new is pushed on the jenkinstest branch of the associated GitHub repo.
What does the job, specifically? It builds a Visual Studio Solution on Debug and Release (both x86 and x64). So, it runs MSBuild.exe four times. After that, it runs a Google test command using an utilities_test.exe executable file.
These is the command that is executed at the first step:
echo "Building solution file on Debug x64"
"C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\MSBuild\Current\Bin\MSBuild.exe" "D:\Jenkins\workspace\CFT_TCA_IMS\SAB_Test\Repository\Utilities\solution_windows\utilities.sln" /p:Configuration="Debug" /p:Platform="x64"
IF %ERRORLEVEL% EQU 0 (echo "Build status of the solution file with <Debug x64> configuration is PASSED") Else (echo "Build status of the solution file with <Debug x64> configuration is FAILED")
(there are three more batch scripts for every platform). The test that follows looks like this:
echo off
echo "Running test on Debug x64"
"D:\Jenkins\workspace\CFT_TCA_IMS\SAB_Test\Repository\Utilities\solution_windows\x64\Debug\utilities_test.exe" --gtest_filter=CategoryName.MyTest
exit 0
Everything seems to work fine. The build process is triggered on push, and the test seems to pass, but there's one problem. I also use Jenkins' Log Parser and at the Post-build Actions, I've added a Console Output (build log) parsing action, that uses a project rule looking like that:
info /0 Error/
info /0 Warning/
info /(?i)errorreport/
error /(?i)error/
error /(?i)failed/
warning /(?i)warning/
info /(?i)building/
info /(?i)done/
Because of this, my build seems not to stop. It usually takes around 2-3 minutes for my solution to build, but I've waited like 20 minutes and nothing happened.
The solution is getting built, and the test passes, but the build process does not stop. The last output of my console log is:
13:35:10 Note: Google Test filter = CategoryName.MyTest
13:35:10 [==========] Running 1 test from 1 test case.
13:35:10 [----------] Global test environment set-up.
13:35:10 [----------] 1 test from CategoryName
13:35:10 [ RUN ] CategoryName.MyTest
13:35:10 [ OK ] CategoryName.MyTest (0 ms)
13:35:10 [----------] 1 test from CategoryName (0 ms total)
13:35:10
13:35:10 [----------] Global test environment tear-down
13:35:10 [==========] 1 test from 1 test case ran. (0 ms total)
13:35:10 [ PASSED ] 1 test.
There's nothing about my log parser outputted here.
When I remove the Log Parser post-build action, everything seems to work fine. Also tried to remove everything from the parser's config file, but nothing gets fixed.
Any idea about what it could be? Any help would be appreciated.
Thank you.
Does your build run on a slave ? If so, Jenkins copies log files and other artifacts back to the master after completing the build steps but before marking the build as complete. You may have a network or I/O bottleneck here.
If you can't figure out the root cause and just want to have the build terminate without intervention, you can use the build timeout plugin.
Related
There are 4 kind gerrit command in gerrit plugin settings.
Success / Fail / Unstable / Not Built.
I could return "success/fail" from my job to gerrit plugin, by using exit 0/1,
And "success/fail" gerrit command was executed.
But I don't Understand how to return "Not Built" by shellscript.
Please teach me.
sorry for my poor english.
Thanks.
Success / Failure / Unstable
If the job has no test reports configured:
If the last build command executed by Jenkins exits with failure
(<> 0) exit code, the build is considered to be failed.
If the last build command exits with success (= 0) exit
code, the build is successful.
If the job has one or more test reports configured:
If the last build command executed by Jenkins exits with failure
(<> 0) exit code, the build is considered to be failed.
If the last build command exits with success (= 0) exit
code but test reports have some failed tests, the build is unstable.
If the last build command exits with success and all test have
passed, the build is stable.
The only other way to get "Unstable" result is through the use of some post-build plugins (like static code analysis for example).
Not Built
I'm not sure about this item... but I think (guess) the build is considered not build when the job finishes by a timeout.
I found out way of force stopping by following command
curl http://127.0.0.1:8080/job/${JOB_NAME}/${BUILD_NUMBER}/stop
and gerrit command for "Not Build" was executed .
I have a job A in Jenkins for my automated testing that is triggered if another job B build is successful. The job A run several tests. Some of the test are flaky so I would like to run them again few times and let them the chance to pass so my build won't be unstable/failed.
Is there any plugin I can use?
I would suggest to fix your tests or rewrite them so they will only fail if something is broken. Maybe you can mock away the things that tend to fail. If you are depnending on a database connection, maybe you could use a sqlite or smething which is local.
But there is also a plugin which can retry a build:
https://wiki.jenkins-ci.org/display/JENKINS/Naginator+Plugin
Simply install the plugin, and then check the Post-Build action "Retry build after failure" on your project's configuration page.
If you want to rerun tests in JUnit-context, take a look here: SO: How to Re-run failed JUnit tests immediately?
Don't know of any plugin to run just the flaky/failed tests again, only the whole build. It should be possible, I just have not found any (and don't have enough time on my hand to write one). Here's what we did on a large java project where the build was ant based:
The build itself was pretty simple (using xml as formatter inside the junit ant task):
ant clean compile test
The build also accepted a single class name as parameter (using batchtest include section inside the junit ant task):
ant -Dtest.class.pattern=SomeClassName test
At the end of the jenkins job, we used the "Execute shell" build step. The idea was to search for all test results that had errors or failures, figure out the name of the class, then run that particular test class again. The file containing the failure will be overwritten, and the test collector at the end of the build will not see the flaky test failure, during the post build steps.
#!/bin/bash +x
cd ${WORKSPACE}
for i in $(seq 1 3); do
echo "Running failed tests $i time(s)"
for file in `find -path '*/TEST-*.xml' | xargs grep 'errors\|failures' | grep '\(errors\|failures\)="[1-9]' | cut -d ':' -f 1`; do
class=`basename ${file} .xml | rev | cut -d '.' -f 1 | rev`
ant -Dtest.class.pattern=${class} test
done
done
After getting the build back under control, you definitely need to address the flaky tests. Don't let the green build fool you, there's still work to be done.
I have a lots of tests on several server. I use Jenkins to manage all of them.
On one server (Slave Windows), when I launch a test in cmd, I got something like :
c:/tests/cucumber --tag #dev -p ie
...
1 scenarios (1 passed)
12 steps (12 passed)
0m31.761s
echo %errorlevel%
0
No error in the tests, and cucumber seems good.
When jenkins launch exactly the same tests, I get :
c:/jenkins_folder/cucumber --tag #dev -p ie
...
1 scenarios (1 passed)
12 steps (12 passed)
0m28.453s
Build step 'Execute Windows batch command' marked build as failure
Finished: FAILURE
The test passed, but is marked failed by jenkins.
The command "echo %errorlevel%" is aborted : the job fail before this point.
The same job played on another slave work.
Same problem with all profile and all tags.
Same problem when I replace profile by real value
I don't use the --strict flag
Jenkins, plugins : all up-to-date
Code of the windows batch :
cd /test8folder
cucumber --tag #dev -p ie
echo %errorlevel%
What did I missed ?
There doesn't seem to be anything wrong with the configuration.
The fact that you get Build step 'Execute Windows batch command' marked build as failure after your cucumber command without actually seeing the echo %errorlevel% executed only reaffirms that there was an error in cucumber (more on that later).
However, in Execute Windows Batch Command, even a command in error would not exit the batch script (unlike Jenkins's default Execute Shell implementation). You should be seeing at least some exit code, 0, 1, or anything.
The only time this would happen is if something within your buildstep executed exit /b [exit_code_num]. I don't know "cucumber", but if that is in-fact a cucubmer.bat and inside there is an exit /b statement, this is what's causing it drop out of the buildstep without continuing.
Solution
You can use call cucumber [whateverparams] so that even when it quits with exit /b, the control will return to the calling process, the Execute Windows Batch command script.
Try that first. And you will probably see that your echo %errorlevel% will probably return a non-zero value when executed under Jenkins, but at least you will see it now.
Now, as for why it succeeds on command prompt, but fails within Jenkins, there could be a lot of reasons, the most common one being environment variables and paths. We can tackle that later once we actually see the exit code of cucumber. You also said it worked on another node: even more reason to believe this is an environment issue, maybe a non-existent folder...
Edit:
The reason that even "successful" test execution exits the calling script is because exit /b 0 would still quit the calling script, even though cucumber exited with "success" 0
I'm going through the tutorial on YSlow and Phantom js in Jenkins here: http://yslow.org/phantomjs/
Everything appears to be working great except the Jenkins builds are failing. I think this is due to the violations that YSlow is finding (6 for the particular site I am measuring). I'd rather have the build be successful (or unstable) vs. failed though
Is that possible with this or will I have to resort to something like the postgroovy or text finder plugin?
This is the console output:
phantomjs.exe yslow.js -i grade -t 50 --format junit http://www.somesite.com 1>yslow.xml
D:\Apps\Jenkins\workspace\YSlow_Test>exit 6
Build step 'Execute Windows batch command' marked build as failure
Thanks
Any non-zero exit code at the end of your Execute Windows batch command build step will result in build step being marked as failure.
To have the build step marked as success, you need an exit code of 0. I don't know anything about "yslow" or "phantomjs" and why they are giving you exit code of non-zero, but from "batch" side of things, you need only write exit 0 at the end of your build step if you want to overwrite the exit code of your phantomjs command.
You can then use Text Finder plugin to parse the console log and mark build as unstable when certain conditions are met.
Reading over this answer, Configuring yslow on Jenkins looks like you need TAP plugin to have the functionality of unit testing marking the build as unstable automatically
Today I started working with jenkins and I successfully added my projects to jenkins and it says all works fine . one of the build takes more than 5 hours but didn't finish either so aborted it(while manual build takes less than 1 hour).. and while i tried to check with the log the log was not detailed. so i tried to get the logs of the perl script by running it as a shell command
usr/local/bin/perl perlscript.pl>logfile.txt
there was no log written and there was no evidence of the build triggered either cases.i'm not aware of what the problem is as both the perlscript(works fine while manually triggered) and jenkins are working properly except this project. I would like to have your help.thanks in advance
A few things you should understand about Jenkins:
Jenkins shows STDOUT as the log of the Job,
so if you redirect it to a file - nothing will be shown in the log.
Depending on how you have set it up, Jenkins may run as its own user,
which may change the behavior of your scripts.
You can confirm this by echo-ing the username at the beginning of your Execute Shell block,
for example:
echo $USER
Each Jenkins-Job is run from its own workspace -
you can confirm that location by simply printing the current working directory
at the beginning of your Execute Shell block, for example:
echo my current directory is
pwd