Using Jenkins I am trying to pass the result of the Nunit task and the OpenCover task to SonarQube.
SonarQube displays properly the test coverage but there is no trace of unit test results.
Here is what I do :
Build Visual Studio project with MSBuild
Batch windows for opencover via nunit
Launch SonarQube scanner
Batch windows for deployement
Here is my actual properties for the Sonar Scanner task:
sonar.projectKey=FC
sonar.projectName=FC
sonar.projectVersion=%SVN_REVISION%
sonar.sources=src
sonar.binaries=build
sonar.exclusions=src/FC.Test/**/*.*
sonar.tests=src/FC.Test
sonar.cs.nunit.reportsPaths=TestResult.xml
sonar.cs.opencover.reportsPaths=reports/opencovertests.xml
I'm using SonarQube 5.6, Jenkins 2.10, SonarQube plugin 2.4.3, OpenCover 4.5 and Nunit 2.6.4
Is there any known problems (compatibility issues...) ?
Note that I tryed to wrap my process with the Prepare and Terminate Sonar analysis tasks but it systematicaly deletes the coverage panel. So i removed it.
EDIT 2017
I stopped doing this way. I understood that the good way to do this was using the SonarQube Scanner for MSBuild which is integrated at the Jenkins's plugin.
The coverage panel was not shown because i didn't configure my quality profiles properly.
But...
I still don't have my tests' result in Sonar.
I checked the absolute path of my report and its parsing by sonarQube : "INFO: Sensor C# Unit Test Results Import [csharp] (done) | time=25ms". I also tryed with a .nunit file. I saw that apply a filter to opencover could help, so I did.
Do you have a clue?
Scanning C# projects requires to use the Sonar Scanner for MSBuild and follow a specific 3 steps process which I guess you don't follow for now. See doc about the 3 steps process at: http://docs.sonarqube.org/display/SCAN/Analyzing+with+SonarQube+Scanner+for+MSBuild and http://docs.sonarqube.org/display/SCAN/From+Jenkins
Your properties look correct. Yet they shall not be provided in a sonar properties file but instead in the command line arguments of the MSBuild.SonarQube.Runner.exe begin step, for example:
MSBuild.SonarQube.Runner.exe begin <allMandatoryParameters> /d:sonar.cs.nunit.reportsPaths=TestResult.xml /d:sonar.cs.opencover.reportsPaths=reports\opencovertests.xml
Olivier
Related
I have a ASP.Net Web Application using .Net Framework 4.5.1 and it's building on the TFS Buildserver (on premise), but SonarQube doesn't perform the analysis. All other buildsteps are successfull. SonarQube 6.0 is a fresh installation, the Runner was installed around a year ago.
The necessary .Net Frameworks for SonarQube and Builds are installed on the build machine.
On a WebAPI Project the analyzer works like a charm. What could it be?
******************************************************************************
Starting task: Fetch the Quality Profile from SonarQube
******************************************************************************
Executing the powershell script: C:\Agents\Agent1\tasks\SonarQubePreBuild\1.0.34\SonarQubePreBuild.ps1
SonarQube Scanner for MSBuild 1.1
Default properties file was not found at C:\Agents\Agent1\tasks\SonarQubePreBuild\1.0.34\MSBuild.SonarQube.Runner-1.1\SonarQube.Analysis.xml
Pre-processing started.
Preparing working directories...
Checking for updates...
SonarQube Scanner for MSBuild Begin Step 2.1
14:41:58.919 Default properties file was not found at C:\Agents\Agent1\_work\14\.sonarqube\bin\SonarQube.Analysis.xml
14:41:58.929 Updating build integration targets...
14:41:58.939 Fetching analysis configuration settings...
14:41:59.039 Generating rulesets...
14:42:00.119 Provisioning analyzer assemblies...
14:42:00.119 Installing required Roslyn analyzers...
Pre-processing succeeded.
******************************************************************************
Finishing task: SonarQubePreBuild
******************************************************************************
... building solution, publish symbols & drop ...
******************************************************************************
Starting task: Finish the analysis and upload the results to SonarQube
******************************************************************************
Executing the powershell script: C:\Agents\Agent1\tasks\SonarQubePostTest\1.0.36\SonarQubePostTest.ps1
SonarQube Scanner for MSBuild 1.1
Default properties file was not found at C:\Agents\Agent1\tasks\SonarQubePreBuild\1.0.34\MSBuild.SonarQube.Runner-1.1\SonarQube.Analysis.xml
Post-processing started.
SonarQube Scanner for MSBuild End Step 2.1
14:40:50.486 WARNING: Failed to find the code coverage command line tool. Possible cause: Visual Studio is not installed, or the installed version does not support code coverage.
The exclude flag has been set so the project will not be analyzed by SonarQube. Project file: C:\Agents\Agent1\_work\14\s\TdbWeb\TdbWeb\TdbWeb.csproj
No analysable projects were found. SonarQube analysis will not be performed. Check the build summary report for details.
Generation of the sonar-properties file failed. Unable to complete SonarQube analysis.
14:40:50.526 Creating a summary markdown file...
Post-processing failed. Exit code: 1
Unexpected exit code received from batch file: 1
The build was not set to fail if the associated quality gate fails.
******************************************************************************
Finishing task: SonarQubePostTest
******************************************************************************
You need Visual Studio Enterprise edition for code coverage and you are using VS Pro. Have a look at this page and expand the 'tesing tools' section. Code coverage is only listed for the Enterprise edition.
I'm using SonarQube and Jenkins. In Jenkins, I check the unit tests with nUnit and the coverage with opencover. The opencover's report is displayed in Sonar but the nunit's report is not. The Unit Tests count is not even displayed in the home page of the project.
Here is what I do :
Build my VisualStudio project with MSBuild
Run nunit and opencover with a batch command line
Run SonarQube scanner's analysis
Package the project
The Step 2's command line :
MKDIR "%WORKSPACE%\reports"
MKDIR "%WORKSPACE%\reports-history"
SET COV_PTH="D:\Programmes\opencover.4.5.3522"
SET TOOL_PATH="D:\Programmes\NUnit-2.6.4"
SET RPT_PATH="D:\Programmes\ReportGenerator"
copy src\Foo.Bar.Tests\app.config build\Foo.Bar.Tests.dll.config
%COV_PTH%\OpenCover.Console.exe "-filter:+[Foo.Bar*]* -[Foo.Bar.Tests]* -[Foo.Bar.Service]* -[Foo.Bar.Domain]* -[Foo.Bar.Dal]Foo.Bar.Dal.Dao.* -[Foo.Bar.Controller]Foo.Bar.Controller.*Comparer -[Foo.Bar.Controller]Foo.Bar.Controller.External.*" -register "-target:%TOOL_PATH%\bin\nunit-console.exe" -targetargs:"build\Foo.Bar.Tests.dll /noshadow /result=reports\TestResult.xml" -output:reports\opencovertests.xml
%RPT_PATH%\ReportGenerator.exe "-reports:reports\opencovertests.xml" "-targetdir:reports-ReportGenerator" "-historydir:reports-history"
The Step 3's parameters :
sonar.projectKey=FooBar
sonar.projectName=FooBar
sonar.projectVersion=%SVN_REVISION%
sonar.sources=src/Foo.Bar.Controller, src/Foo.Bar.Dal, src/Foo.Bar.Domain, src/Foo.Bar.Fichier, src/Foo.Bar.Generation, src/Foo.Bar.Reporting, src/Foo.Bar.Service, src/Foo.Bar.Web
sonar.binaries=Build/Foo.Bar.Controller.dll, Build/Foo.Bar.Dal.dll, Build/Foo.Bar.Domain.dll, Build/Foo.Bar.Service.dll, Build/Foo.Bar.Fichier.exe, Build/Foo.Bar.Generation.exe, Build/Foo.Bar.Reporting.exe
sonar.tests=src/Foo.Bar.Tests
sonar.cs.opencover.reportsPaths=reports/opencovertests.xml
sonar.cs.nunit.reportsPaths=reports/TestResult.xml
EDIT
I also tried the following process (same results) :
Prepare SonarQube scanner's analysis
Build my VisualStudio project with MSBuild
Run nunit and opencover with a batch command line
End SonarQube scanner's analysis
Package the project
END EDIT
The nUnit's report is well generated in workspace/reports/TestResult.xml.
I tested the analysis generation by command line and via the sonarqube plugin for jenkins. I got the same results.
Can you see anything wrong?
Is there any SonarQube's logs that I could checked ?
I am using Jenkins 2.10, MSBuild 12, SonarQube 5.6, SonarQube scanner for Jenkins plugin 2.4.3, opencover 4.5 and nunit 2.6.4
EDIT
I updated my config : Jenkins 2.18, MSBuild 14, SonarQube scanner for Jenkins plugin 2.4.4. Same results.
END EDIT
Take a look at: SonarQube documentation: Unit Test Execution Results Import (C#, VB.NET)
It says:
Drilldown on Test Execution Results is not supported
My understanding is that you can only see the results (number of Tests passing, errors, etc) but not detail from the tests.
You also ask about Logs. You have:
Jenkins Job log
SonarQube Background tasks: Login as Admin -> Administration -> Projects -> Background tasks -> far right of each execution ---> http://docs.sonarqube.org/display/SONAR/Background+Tasks
SonarQube logs: sonar_install_directory\logs
I am running Sonar task through Ant, triggered by Jenkins in RHEL environment. I am successfully using Cobertura for Junit code coverage and Surefire for reporting. Sonar imports the Surefire reports fine.
However, now I am running Selenium tests using Ant in Jenkins. I would like to report code coverage and test results to Sonar. Apparently I need the JaCoCo plugin which analyses code coverage and reports tests. I presume like for unit tests, Jenkins does the job and Sonar only imports the reports into its own repository.
I am puzzled on how to actually do this. The web page http://www.sonarsource.org/measure-coverage-by-integration-tests-with-sonar-updated/ references to the JaCoCo page http://www.eclemma.org/jacoco/trunk/doc/ant.html. I am not sure what the steps are to be done. Do I need the coverage target? Should I only start the agent? Where in Ant do I start the agent? Do I need to dump stuff?
I really appreciate all the help I can get, thanks :)
Sonar 3.3 has a new feature for combine code coverage metrics generated by both unit tests and integration tests. This is done by using two properties to detail the two different report files generated by the jacoco too:
#Tells Sonar where the unit tests code coverage report is
sonar.jacoco.reportPath=reports/jacoco/jacoco-ut.exec
#Tells Sonar where the integration tests code coverage report is
sonar.jacoco.itReportPath=reports/jacoco/jacoco-it.exec
The Sonar examples project has an integration test example for ANT:
https://github.com/SonarSource/sonar-examples/tree/master/projects/code-coverage/it/ant/it-jacoco-ant
Unfortunately it doesn't give an example of running the actual tests, instead it just shows how to configure an ANT build to load data.
Finally the Sonar documentation has more details with links to the example projects.
I agree that this question is asked many a times by many people and many solutions are already been provided. However, after referring to umpteen links, not being able to display the Emma report in Sonar is driving me crazy. Below is a description of what I am trying to do.
My java project is a multi module maven project. We are using EMMA as the code coverage tool and Jenkins to build the project, Sonar to analyze the code and configured Sonar for our build job using the Jenkins sonar plugin. Up to this point, the story is great. The problem comes when I try to reuse the reports generated by Emma during Sonar analysis. There is no apparent error in the Sonar analysis logs during the build, however, in Sonar project dashboard the coverage report is 0%. Below is configuration data that I provide in the Sonar configuration section of my Jenkins build job.
-Dsonar.core.codeCoveragePlugin=emma
-Dsonar.emma.reportPath=/target
-Dsonar.dynamicAnalysis=reuseReports
-Dsonar.surefire.reportsPath=/target/surefire-reports
Sonar Version : 3.0
Emma plugin versions:
emma-maven-plugin : 1.0-alpha-2
Sonar Emma Plugin :1.0.1
emma : 2.0.5312
The Maven build generates the coverage.em files in the individual modules' target folder. No coverage.ec file is generated. Is it mandatory to have both coverage.ec and coverage.em files for Sonar?
The coverage.es,html,txt,xml files are generated in the project build directory/emma folder
The Sonar analysis logs for a module looks like below.
[INFO] [15:06:02.829] Sensor org.sonar.plugins.emma.EmmaSensor#bfc33...
[WARN] [15:06:02.844] Resource will be ignored in next Sonar versions, index is locked: com.mycompany.mypackage.tests.util.Activator
[INFO] [15:06:02.876] Sensor org.sonar.plugins.emma.EmmaSensor#bfc33 done: 47 ms
The above logs make me think that Sonar is indeed doing something with the generated Emma reports, but not displaying anything on the Sonar project dashboard.
Please let me know where I may be going wrong or what additional thing needs to be done to get this working.
I think this thread might me useful. http://sonar.15.n6.nabble.com/Receiving-quot-index-is-locked-quot-warning-when-Emma-sensor-runs-td4676490.html
It describes the same problem and gives you some workarounds
I followed the instructions here
I am NOT using Maven.
My Jenkins job output says:
12:32:33.951 INFO Sensor JaCoCoSensor...
12:32:33.961 INFO Project coverage is set to 0% as no JaCoCo execution data has been dumped: /var/lib/jenkins/workspace/SeqGen/SeqGen/jacoco.exec
12:32:35.152 INFO Sensor JaCoCoSensor done: 1201 ms
Here are my properties:
project.home=SeqGen
sonar.projectKey=com.skyboximaging:seqgen
sonar.projectName="SeqGen"
sonar.projectVersion=1.0
sonar.dynamicAnalysis=true
sources=src/java
tests=test/java
binaries=classes
sonar.jacoco.reportPath=jacoco.exec
sonar.jacoco.antTargets=test-with-coverage
On the Sonar server, I set General Settings > Code Coverage > Code coverage plugin to jacoco
I am very confused by the documentation at http://docs.codehaus.org/display/SONAR/Code+coverage+plugins I am particularly puzzled by this sentence:
"During Sonar analysis, the Sonar Jacoco plugin will take care to attach the Jacoco agent to the JVM and to launch the unit tests."
As far as I can tell, my Ant target is not getting invoked. How does the Sonar Jacoco plugin know where to find my build.xml?
What am I doing wrong?
They just released what you need : https://wiki.jenkins-ci.org/display/JENKINS/JaCoCo+Plugin
Providing an answer for the next poor soul who spends hours trying to find out why this doesn't work. Apparently this feature has been removed:
Remove "sonar.jacoco.antTargets" property and related classes in the plugin
One possibility is that the name of your ant task for junit does not match the name specified in the properties.
You have specified
sonar.jacoco.antTargets=test-with-coverage
As per the documentation, there should be an ant target with exactly the same name. Also in this target, junit should be set to fork as well.