I've setup Cobertura successfully to run through Jenkins and generate coverage reports, now I'm stuck on configuring Violations to report results for pep8 tests.
Right now I have it so that I run PEP8 with a batch command and it generate a .txt file that stores all the violations which ends up in the workspace.
Do I have to define batch commands to create an xml file for these violations to send to the Report Violations Post-build action? If so, how would I define that command?
As Vajk points out, you do not need an XML file. You can simply pipe the output of the pep8 utility to a file and use that as the violations file in Jenkins.
For example in your Jenkins build configuration under:
Build -> Execute Shell -> Command
you can write:
pep8 path/to/code > pep8_report.txt
And then under:
Post-build Actions -> Report Violations -> pep8 / XML filename pattern
add:
pep8_report.txt
Note that even though the UI asks for an XML file and the file is not an XML file, this worked fine for me on Jenkins version 1.499.
I see this a very old question but I had a lot of trouble myself to find an answer to this, so hopefully this will help others.
After a lot of research I found that Jenkins Violations plugin does not like the ./ bit before each line, so just parsing the output of pep8 (it is also valid for flake8) will do the trick:
pep8 . | 's|^./||' | tee pep.txt
The important bit in there is sed which changes the occurrences of ./ at the beginning of the string(^) to '', effectively removing them
The tee bit is some workaround to avoid Jenkins halting if the output of the pep8 command is non-zero.
Related
I am trying to create report using ant in jmeter, meaning the full report.
I downloaded ant and it is installed as expected.
first I want to understand if ant command need to perform after test plan ran in the past? meaning it is offline process that creates the html reports? after the test plan finished?
Or is it command that actually used to run the test plan and create the html, meaning I do not need to run jmeter before?.
I used this command
jmeter -n -t C:\JMETER\Framework\Test_Fragment\Kung_Fu.jmx -l C:\Users\stackoverflow\Desktop\Jmeter_reports\results22_05_2018.csv
to run jmeter from command line, and create csv, so do I need two commands? one for creating csv and one for the ant? and if I create the csv where can I find the jtl of the testplan.
Name of test plan kung_fu
name of csv results22_05_2018.csv
what are the processes to run he ant, since I rename the Kung_Fu.jmx to test.jmx and put it in extras folder and when I command ant, it says test.jtl is not found.
can someone give a full explanation about the whole process
Rename results22_05_2018.csv into results22_05_2018.jtl and copy it to "extras" folder of your JMeter installation
Execute the following command in "extras" folder of your JMeter installation:
ant -Dtest=results22_05_2018 xslt-report
HTML report will be available as results22_05_2018.html
For more details see:
build.xml - reference Ant build file, by default it:
looks for Test.jmx file in the current folder
executes it and stores the result into Test.jtl file
applies XSLT transformation to the Test.jtl file and generates HTML file out of it.
JMeter Ant Task
Five Ways To Launch a JMeter Test without Using the JMeter GUI
In my project workspace, the test-results.xml file exists inside the target\surefire-reports\testng-results.xml directory. But Jenkins fails to read the XML file and gives below error on console.
TestNG Reports Processing: START
Looking for TestNG results report in workspace using pattern: **\target\surefire-reports\testng-results.xml
Did not find any matching files.
To ensure the file isn't too old, I had checked that the test-results.xml (and other files) belong to the latest test run. The Jenkins server is running on Ubuntu 14.04 LTS.
I'm running my tests in this manner: My project root directory has a run_tests.sh script which looks like this:
#!/bin/bash
if [ "$1" = "" ]; then
echo "Please provide a valid suite XML file name."
else
mvn clean
mvn compile
mvn clean test -Dsurefire.suiteXmlFiles="$1"
fi
I just pass the suite XML file name as a parameter to this script in Jenkins (execute shell).
Please help.
I found the solution for this.
Go to Configure of your Job
in General Tab, you may find
Advanced Button, Click on this
Check the check box of "Use custom
workspace", under this you see the Directory text box, here you copy
your Selenium Workspace Folder, for example mine is
"E:\eclipse\eclipse-workspace\WebDriveTest\"
Scroll down the page
under the Post-build Actions, Publish TestNG Results, TestNG XML
report pattern : give like this
"**/target/surefire-reports/testng-results.xml" (check this path in
the same workspace).
I hope this will help you!.
You should be using / instead of \ (since you mentioned that your Jenkins is running on a UNIX box)
Krishnan, in the testng-users Google Group, pointed out that it could be an issue with my Jenkins project workspace, and it was the same.
I changed the default workspace in my Jenkins project.
So I've added the path "$HOME/myWorkspace/myProject/" in my Jenkins project workspace, and "**/target/surefire-reports/testng-results.xml" in my TestNG setting in the same Jenkins project, and it works!
Thank you Krishan for your help.
Please see my answer in another post here, it should be very clear.
In short, it is caused by the current directory was changed to the default Jenkins workspace, you need set your custom workspace in the Job's Config.
I agree with Krishnan Mahadevan usage of '\' instead of '/' while providing the path for TestNG Report also solved my problem.
Extremely important thing to note here:
When providing path for Root POM in the build section '\' is used
C:\Users\harsh\eclipse-workspace\ProjTwo3\pom.xml
When providing path for TestNG XML report pattern in Publish TestNG Result section
'/' is used
C:/Users/harsh/eclipse-workspace/ProjTwo3/target/surefire-reports/testng-results.xml
Console Output:
channel stopped
TestNG Reports Processing: START
Looking for TestNG results report in workspace using pattern:
C:/Users/harsh/eclipse-workspace/ProjTwo3/target/surefire-reports/testng-results.xml
Saving reports...
Processing 'C:\Users\harsh.jenkins\jobs\MyApplication\builds\12\testng\testng-results.xml'
11.688312% of tests failed, which exceeded threshold of 0%. Marking build as UNSTABLE
I want to access and grep Jenkins Console Output as a post build step in the same job that creates this output. Redirecting logs with >> log.txt is not a solution since this is not supported by my build steps.
Build:
echo "This is log"
Post build step:
grep "is" path/to/console_output
Where is the specific log file created in filesystem?
#Bruno Lavit has a great answer, but if you want you can just access the log and download it as txt file to your workspace from the job's URL:
${BUILD_URL}/consoleText
Then it's only a matter of downloading this page to your ${Workspace}
You can use "Invoke ANT" and use the GET target
On Linux you can use wget to download it to your workspace
etc.
Good luck!
Edit:
The actual log file on the file system is not on the slave, but kept in the Master machine. You can find it under: $JENKINS_HOME/jobs/$JOB_NAME/builds/lastSuccessfulBuild/log
If you're looking for another build just replace lastSuccessfulBuild with the build you're looking for.
Jenkins stores the console log on master. If you want programmatic access to the log, and you are running on master, you can access the log that Jenkins already has, without copying it to the artifacts or having to GET the http job URL.
From http://javadoc.jenkins.io/archive/jenkins-1.651/hudson/model/Run.html#getLogFile(), this returns the File object for the console output (in the jenkins file system, this is the "log" file in the build output directory).
In my case, we use a chained (child) job to do parsing and analysis on a parent job's build.
When using a groovy script run in Jenkins, you get an object named "build" for the run. We use this to get the http://javadoc.jenkins.io/archive/jenkins-1.651/hudson/model/Build.html for the upstream job, then call this job's .getLogFile().
Added bonus; since it's just a File object, we call .getParent() to get the folder where Jenkins stores build collateral (like test xmls, environment variables, and other things that may not be explicitly exposed through the artifacts) which we can also parse.
Double added bonus; we also use matrix jobs. This sometimes makes inferring the file path on the system a pain. .getLogFile().getParent() takes away all the pain.
You can install this Jenkins Console log plugin to write the log in your workspace as a post build step.
You have to build the plugin yourself and install the plugin manually.
Next, you can add a post build step like that:
With an additional post build step (shell script), you will be able to grep your log.
I hope it helped :)
Log location:
${JENKINS_HOME}/jobs/${JOB_NAME}/builds/${BUILD_NUMBER}/log
Get log as a text and save to workspace:
cat ${JENKINS_HOME}/jobs/${JOB_NAME}/builds/${BUILD_NUMBER}/log >> log.txt
For very large output logs it could be difficult to open (network delay, scrolling). This is the solution I'm using to check big log files:
https://${URL}/jenkins/job/${jobName}/${buildNumber}/
in the left column you see: View as plain text. Do a right mouse click on it and choose save links as. Now you can save your big log as .txt file. Open it with notepad++ and you can go through your logs easily without network delays during scrolling.
I found the console output of my job in the browser at the following location:
http://[Jenkins URL]/job/[Job Name]/default/[Build Number]/console
This is designed for use when you have a shell script build step. Use only the first two lines to get the file name.
You can get the console log file (using bash magic) for the current build from a shell script this way and check it for some error string, failing the job if found:
logFilename=${JENKINS_HOME}/${JOB_URL:${#JENKINS_URL}}
logFilename=${logFilename//job\//jobs\/}builds/${BUILD_NUMBER}/log
grep "**Failure**" ${logFilename} ; exitCode=$?
[[ $exitCode -ne 1 ]] && exit 1
You have to build the file name by taking the JOB_URL, stripping off the leading host name part, adding in the path to JENKINS_HOME, replacing "/job/" to "/jobs/" to handle all nested folders, adding the current build number and the file name.
The grep returns 0 if the string is found and 2 if there is a file error. So a 1 means it found the error indication string. That makes the build fail.
Easy solution would be:
curl http://jenkinsUrl/job/<Build_Name>/<Build_Number>/consoleText -OutFile <FilePathToLocalDisk>
or for the last successful build...
curl http://jenkinsUrl/job/<Build_Name>/lastSuccessfulBuild/consoleText -OutFile <FilePathToLocalDisk>
My ant files are already present and I am not allowed to change those for various reasons. I have created a batch file and I am writing all those ant commands in that batch file just to automate the process. The code looks something like this
1. cd abc
2. ant realclean && ant
3. cd..
4. cd pqr
5. ant realclean && ant
6. cd..
However now I have to check if the build fired on line 2 is successful then only goto line 3 otherwise exit.
I googled a bit and found %errorlevel% as one option but it is not working in my case.
Any suggestions?
Thanks in advance.
How about adding another ANT file which contains the tasks to execute the existing ANT files?
That would give you error handling without doing anything, you could reuse that file on different OSs and your CI server and the syntax would be saner than batch...
You can have ANT execute, say project.xml, with ant -f project.xml
[EDIT] You don't even have to change the existing projects: Just create a new project, assign paths to the old projects to properties in the new build.xml and then you can build the old projects from the new one.
This gives you:
A sane[*] syntax that you already know (no need to learn the ugly/buggy/handicapped rules of DOS batch programming; you did know that DOS was once named QDOS "Quick and Dirty OS", yes?)
Proper error handling
A simple way to pass arguments to the sub-builds
Cross-platform support
Useful error messages when something goes wrong
[*]: Compared to DOS...
I added an 'Execute shell' build step in Jenkins to run the cpplint.py
python /var/lib/jenkins/scripts/cpplint.py --counting=detailed `find path -name *.cpp
I also added 'Scan for compiler warnings' and added CppLint.
However it always gets 0 warnings even though it displayed in the Console output some warnings such as
filename.cpp:18: Missing space after , [whitespace/comma] [3]
If you run cpplint.py with --output=vs7 it will produce the format expected by the Jenkins warnings plugin.
I use the Cppcheck Plugin and cpplint_to_cppcheckxml.py to convert cpplint.py output to the XML format expected by Cppcheck Plugin. This works really well. I can click on the offending issue in Cppcheck Results displayed on the Jenkins job page and it will display the source code with offending line highlighted. Very cool.
You must provide cpplint an absolute path to your source code directory in order for the hyperlink generation to work on Cppcheck Results page. The only con I see is your Cppcheck and cpplint results are combined and not separated.
If using Linux bash scripts, here is how I turn a relative path into absolute in order for the cpplint to generate absolute paths in it's output:
# Build cpplint reports and transform to cppcheck compatible XML format
# Convert relative path to absolute path so that Jenkins job can easily display the source code errors
srcPathAbsolute=${PWD}/../dicegame/src/main
srcPathAbsolute=$(readlink -f ${srcPathAbsolute})
cpplint.py --counting=detailed ${srcPathAbsolute}/*.cpp 2>&1| cpplint_to_cppcheckxml.py &> cpplint-cppcheck-result.xml
In my Jenkins job configuration to locate both ccplint-cppcheck-result.xml files and my normal cppcheck-result.xml.
Publish cppcheck results
Cppcheck report XMLs **/*cppcheck-result.xml
Thanks to original developer of cpplint_to_cppcheckxml.py. This script serves as an example on how to connect output of other tools into existing Jenkins plugins. Very nice!