I have a jenkins multi-pipeline project setup doing sonar scans with PR decoration.
When a pull request is scanned sonar correctly runs the test coverage report sensor and coverage is reported correctly.
However when this is merged back to our main the sensor just isn't being ran.
PR Branch:
....
INFO: Importing 34 Roslyn reports
INFO: Sensor C# [csharp] (done) | time=8804ms
INFO: Sensor C# Tests Coverage Report Import [csharp]
INFO: Parsing the OpenCover report C:\Jenkins\workspace\Build-API_PR-273\.\Tests\SharedDataServices.Tests\coverage.opencover.xml
....
Main Branch:
....
INFO: Importing 34 Roslyn reports
INFO: Sensor C# [csharp] (done) | time=13859ms
INFO: Sensor JavaSecuritySensor [security]
....
As you can see it completely omits the sensor.
Things I've checked:
Pull request works and coverage is being reported fine
The reports are being generated in the same relative locations on Main
The contents of the reports are not empty
The contents of the reports refer correctly to the tested .cs files
Any ideas or tips please?
Related
I followed this video https://www.youtube.com/watch?v=jh7utASgKj4 to integrate sonarqube with jenkins. However, I get the insufficient privileges error when scanning the code by sonarqube server. The version of sonarqube server is 6.7. how can I solve this problem?
Here is the analysis properties in the jenkins project
# Required metadata
sonar.projectKey = test-sonar
# Path to source directory
sonar.sources = /var/jenkins_home/workspace/$JOB_NAME/test-sonar/src
# bind sonar.java.binaries property
sonar.java.binaries=/var/jenkins_home/workspace/$JOB_NAME/test-sonar/target/classes
Here is the output of console.
INFO: SCM provider for this project is: git
INFO: 16 files to be analyzed
INFO: 16/16 files analyzed
INFO: 5 files had no CPD blocks
INFO: Calculating CPD for 8 files
INFO: CPD calculation finished
INFO: Analysis report generated in 132ms, dir size=124 KB
INFO: Analysis reports compressed in 48ms, zip size=61 KB
INFO: ------------------------------------------------------------------------
INFO: EXECUTION FAILURE
INFO: ------------------------------------------------------------------------
INFO: Total time: 48.028s
INFO: Final Memory: 15M/304M
INFO: ------------------------------------------------------------------------
ERROR: Error during SonarQube Scanner execution
ERROR: Insufficient privileges
ERROR:
ERROR: Re-run SonarQube Scanner using the -X switch to enable full debug logging.
WARN: Unable to locate 'report-task.txt' in the workspace. Did the SonarScanner succedeed?
ERROR: SonarQube scanner exited with non-zero code: 1
If you are running the Sonar Analysis via Execute shell plugin for Jenkins, you need to supply the Username Password of a User which has the Execute Analysis rights on the SonarQube server you are trying to access.
To supply the username in the shell, you can use
-Dsonar.login=yourLoginName -Dsonar.password=*****
To set the Execute Analysis permission you can ask your DevOps team to give you access or if you are looking into that,
Goto Administration > Security > User/Groups.
Please make sure that you configure the Sonarqube UserID and Password having privileges to access sonar server (execute analysis permissions) in the Sonar plugin for Jenkins. This error might appear for 5.3 and higher versions when the anonymous access to sonar is disabled.
Our Sonar Build Environment details as follows:
* SonarQube Server Version - 5.6.7 (64-Bit).
* Sonar Client Build Operating System – Ubuntu-14.04 (LTS - 64-Bit).
* Sonar Build machine total RAM: 16-GB.
* Sonar-Scanner- Version - 3.0.3.778.
* sonar-cxx-plugin-0.9.7.jar
* Source Code Language: C++
* Client Machine Java Version: 1.8
* Source Code Size: 62-GB.
Problem:
Through Jenkins sonar-scanner finished successfully (Log as follows).
Jenkins Sonar-Scanner Build Log:
12:24:14 INFO: CPD calculation finished
12:26:35 INFO: Analysis report generated in 136049ms, dir size=1 GB
12:49:25 INFO: Analysis reports compressed in 1369236ms, zip size=385 MB
12:49:43 INFO: Analysis report uploaded in 18811ms
12:49:43 INFO: ANALYSIS SUCCESSFUL, you can browse http://sonarqube-server/dashboard/index/scm.project.4.0
12:49:43 INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
12:49:43 INFO: More about the report processing at http://sonarqube-server/api/ce/task?id=AV-af9So2HbsBuSrsvrw
12:50:22 INFO: ------------------------------------------------------------------------
12:50:22 INFO: EXECUTION SUCCESS
Whereas Sonarqube project dashboard shows "No analysis has been performed since creation. The only available section is the configuration." And also under SonarQube Web--> Project--> Administration-->Background Tasks-->Logs shows below errors.
Error:
2017.11.07 13:04:23 ERROR [o.s.s.c.t.CeWorkerCallableImpl] Failed to execute task AV-VUEIr2HbsBuSrsvnY
java.lang.OutOfMemoryError: GC overhead limit exceeded
I have tried the following, still same problem persists.
Tried with SONAR_RUNNER_OPTS option with -Xmx10240m -XX:MaxPermSize=1024m -XX:ReservedCodeCacheSize=128m
Then in sonar-project.properties file I have added sonar.ce.javaOpts=-Xmx1280m & sonar.web.javaOpts=-Xmx1280m
Or still I need to increase the build machine RAM (or) 16-GB is sufficient for 62-GB Code?
You need to upgrade the memory allocated for Compute Engine on the server side not on the scanner side (for this particular issue).
So on your SonarQube server, update the following parameter :
sonar.ce.javaOpts=-Xmx2G -Xms128m -XX:+HeapDumpOnOutOfMemoryError -Djava.net.preferIPv4Stack=true
It will allow Compute Engine which will integrate your report to consume 2Gb of memory (by default it's 512Mb).
I am running SonarQube server v6.4 with v6.1 of the C# plugin. We have v3.0.0 of the SonarQube TFS extension installed in our TFS 2015 (Update 3) instance.
I am having issues with SonarQube scanning .cs files. There are a few warnings in the output of a few build steps that I'm not entirely sure what to make of. I haven't been able to find much on the messages.
Visual Studio Build step output:
OverrideCodeAnalysisProperties:
Skipping FxCop analysis: the SonarQube ruleset does not exist. Ruleset: C:\BuildAgents\Agent-01\_work\111\.sonarqube\conf\\SonarQubeFxCop-cs.ruleset
SonarQube Scanner for MSBuild - End Analysis (new) step output:
I'm seeing this at the beginning of the output for the step...
SonarQube Scanner for MSBuild 3.0
Default properties file was found at C:\BuildAgents\Agent-01\tasks\SonarQubeScannerMsBuildBegin\3.0.0\SonarQubeScannerMsBuild\SonarQube.Analysis.xml
Loading analysis properties from C:\BuildAgents\Agent-01\tasks\SonarQubeScannerMsBuildBegin\3.0.0\SonarQubeScannerMsBuild\SonarQube.Analysis.xml
Post-processing started.
17:58:54.965 17:58:54.965 WARNING: Failed to find the code coverage command line tool. Possible cause: Visual Studio is not installed, or the installed version does not support code coverage.
WARNING: File is not under the project directory and cannot currently be analysed by SonarQube. File: C:\Users\TFS2015Build\AppData\Local\Temp\.NETFramework,Version=v4.5.2.SqlClrAttributes.cs, project: C:\BuildAgents\Agent-01\_work\111\s\FunProject.Db\FunProject.Db.sqlproj
SONAR_SCANNER_OPTS is not configured. Setting it to the default value of -Xmx1024m
Calling the SonarQube Scanner...
INFO: Scanner configuration file: C:\BuildAgents\Agent-01\tasks\SonarQubeScannerMsBuildBegin\3.0.0\SonarQubeScannerMsBuild\sonar-scanner-3.0.3.778\bin\..\conf\sonar-scanner.properties
INFO: Project root configuration file: C:\BuildAgents\Agent-01\_work\111\.sonarqube\out\sonar-project.properties
INFO: SonarQube Scanner 3.0.3.778
INFO: Java 1.8.0_121 Oracle Corporation (64-bit)
INFO: Windows Server 2012 R2 6.3 amd64
INFO: SONAR_SCANNER_OPTS=-Xmx1024m
INFO: User cache: C:\Users\TFS2015Build\.sonar\cache
INFO: Load global settings
INFO: Load global settings (done) | time=125ms
INFO: User cache: C:\Users\TFS2015Build\.sonar\cache
INFO: Load plugins index
INFO: Load plugins index (done) | time=0ms
INFO: Download sonar-csharp-plugin-6.1.0.2359.jar
INFO: Download sonar-python-plugin-1.8.0.1496.jar
INFO: Download sonar-java-plugin-4.11.0.10660.jar
INFO: Download sonar-flex-plugin-2.3.jar
INFO: Download sonar-scm-git-plugin-1.2.jar
INFO: Download sonar-xml-plugin-1.4.3.1027.jar
INFO: Download sonar-php-plugin-2.10.0.2087.jar
INFO: Download sonar-scm-svn-plugin-1.4.0.522.jar
INFO: Download sonar-javascript-plugin-3.1.1.5128.jar
INFO: SonarQube server 6.4.0
INFO: Default locale: "en_US", source code encoding: "windows-1252" (analysis is platform dependent)
INFO: Process project properties
INFO: Load project repositories
INFO: Load project repositories (done) | time=141ms
INFO: Load quality profiles
INFO: Load quality profiles (done) | time=47ms
INFO: Load active rules
INFO: Load active rules (done) | time=281ms
INFO: Load metrics repository
INFO: Load metrics repository (done) | time=78ms
INFO: Publish mode
...then I'm seeing a set of output like this for each project in my solution:
INFO: ------------- Scan FunProject.Shared
INFO: Load server rules
INFO: Load server rules (done) | time=156ms
INFO: Initializer GenericCoverageSensor
INFO: Initializer GenericCoverageSensor (done) | time=0ms
INFO: Base dir: C:\BuildAgents\Agent-01\_work\111\s\FunProject.Shared
INFO: Working dir: C:\BuildAgents\Agent-01\_work\111\.sonarqube\out\.sonar\mod13
INFO: Source paths: FunProjectFolder1/FunProjectClassFile1.cs, FunProjectFolder2/FunProjectClassFile2.cs, ... Properties/AssemblyInfo.cs, packages.config
INFO: Source encoding: UTF-8, default locale: en_US
INFO: Index files
INFO: Analyzer working directory does not exist
INFO: 12 files indexed
INFO: Quality profile for cs: Sonar way
INFO: Sensor C# [csharp]
INFO: Importing analysis results from C:\BuildAgents\Agent-01\_work\111\.sonarqube\out\FunProject.Shared_3389\output-cs
WARNING: WARN: Protobuf file not found: C:\BuildAgents\Agent-01\_work\111\.sonarqube\out\FunProject.Shared_3389\output-cs\encoding.pb
INFO: Importing Roslyn report
WARNING: WARN: Protobuf file not found: C:\BuildAgents\Agent-01\_work\111\.sonarqube\out\FunProject.Shared_3389\output-cs\metrics.pb
WARNING: WARN: Protobuf file not found: C:\BuildAgents\Agent-01\_work\111\.sonarqube\out\FunProject.Shared_3389\output-cs\token-type.pb
WARNING: WARN: Protobuf file not found: C:\BuildAgents\Agent-01\_work\111\.sonarqube\out\FunProject.Shared_3389\output-cs\symrefs.pb
WARNING: WARN: Protobuf file not found: C:\BuildAgents\Agent-01\_work\111\.sonarqube\out\FunProject.Shared_3389\output-cs\token-cpd.pb
INFO: Sensor C# [csharp] (done) | time=78ms
INFO: Sensor SonarJavaXmlFileSensor [java]
INFO: Sensor SonarJavaXmlFileSensor [java] (done) | time=0ms
INFO: Sensor Analyzer for "php.ini" files [php]
INFO: Sensor Analyzer for "php.ini" files [php] (done) | time=0ms
Is there something obvious that I'm missing?
Edit:
I am using Visual Studio Professional, so the warning about code coverage is caused by the fact that I do not have Visual Studio Enterprise installed on the build server.
Suggest you first try to analyze solution in the command line locally to see if it works.
Run your analysis by executing the following commands from the root directory of the project:
SonarQube.Scanner.MSBuild.exe begin /k:"org.sonarqube:sonarqube-scanner-msbuild" /n:"Project Name" /v:"1.0"
MSBuild.exe /t:Rebuild
SonarQube.Scanner.MSBuild.exe end
This will narrow down it's your environment issue or just related to your build definition settings.
According to the error message:
WARNING: Failed to find the code coverage command line tool. Possible
cause: Visual Studio is not installed, or the installed version does
not support code coverage.
Double check your VS edition on your build agent. Note: you need Visual Studio Enterprise edition for code coverage. Have a look at this page and expand the 'testing tools' section. Code coverage is only listed for the Enterprise edition.
Besides, you could also enable verbose Debug Mode with system.debug=true to get a more detail build log for troubleshooting.
I downgraded the C# plugin from 6.1 (build 2359) to 5.11 (build 1721), and the problem went away. There must be some issue with that release that was causing the issue.
I'm just getting started with SonarQube and it does run so far.
The IT department has set up SonarQube 6.2 for our Team Foundation Server and I have installed and configured the msbuild scanner on my BuildAgent.
I call the msbuild scanner in the XAML-Template of my build definition and it does run, but it gives me a warning, which makes my build "partially succeeded".
This is the error message:
07:53:01.658 WARN: Protobuf file not found:
d:\b\210\474.sonarqube\out\ReferenceManager_x86_Release_6561\output-cs\issues.pb
The issues.pb file is really missing, but why do I get an error/warning for this? I do not get this, when I run the scanner and msbuild from command line, only when in runs in team build. But when running on command line there also is no issues.pb file.
When running the scanner from the command line, I get this output:
09:34:05.959 INFO: 97 files indexed
09:34:05.985 INFO: Quality profile for cs: Sonar way
09:34:05.992 DEBUG: Code colorizer, supported languages:
09:34:07.008 DEBUG: 'Generic Coverage Report' skipped because one of the required properties is missing
09:34:07.009 DEBUG: 'Generic Test Executions Report' skipped because one of the required properties is missing
09:34:07.018 DEBUG: 'SurefireSensor' skipped because there is no related file in current project
09:34:07.023 DEBUG: 'JaCoCoSensor' skipped because there is no related file in current project
09:34:07.025 DEBUG: 'JavaSquidSensor' skipped because there is no related file in current project
09:34:07.025 DEBUG: 'JavaScript Squid Sensor' skipped because there is no related file in current project
09:34:07.027 DEBUG: Sensors : Lines Sensor -> SCM Sensor -> C# -> Coverage Report Import -> Coverage Report Import -> Unit Test Results Import -> XmlFileSensor -> Zero Coverage Sensor -> Code Colorizer Sensor -> CPD Block Indexer
09:34:07.029 INFO: Sensor Lines Sensor
09:34:07.279 INFO: Sensor Lines Sensor (done) | time=250ms
09:34:07.279 INFO: Sensor SCM Sensor
09:34:07.279 INFO: SCM Publisher is disabled
09:34:07.281 INFO: Sensor SCM Sensor (done) | time=2ms
09:34:07.281 INFO: Sensor C#
09:34:07.292 INFO: SonarAnalyzer.Scanner needs to be executed: false
09:34:07.292 INFO: Importing analysis results from C:\Users\Build-SW-Dev\Desktop\.sonarqube\out\ReferenceManager_x86_Release_5730\output-cs
09:34:08.749 INFO: Importing Roslyn report
09:34:08.998 INFO: Sensor C# (done) | time=1717ms
When running the scanner during team build, I get this output:
11:12:51.390 INFO: 97 files indexed
11:12:51.422 INFO: Quality profile for cs: Sonar way
11:12:51.422 DEBUG: Code colorizer, supported languages:
11:12:52.468 DEBUG: 'Generic Coverage Report' skipped because one of the required properties is missing
11:12:52.468 DEBUG: 'Generic Test Executions Report' skipped because one of the required properties is missing
11:12:52.468 DEBUG: 'SurefireSensor' skipped because there is no related file in current project
11:12:52.468 DEBUG: 'JaCoCoSensor' skipped because there is no related file in current project
11:12:52.468 DEBUG: 'JavaSquidSensor' skipped because there is no related file in current project
11:12:52.468 DEBUG: 'JavaScript Squid Sensor' skipped because there is no related file in current project
11:12:52.484 DEBUG: Sensors : Lines Sensor -> SCM Sensor -> C# -> Coverage Report Import -> Coverage Report Import -> Unit Test Results Import -> XmlFileSensor -> Zero Coverage Sensor -> Code Colorizer Sensor -> CPD Block Indexer
11:12:52.484 INFO: Sensor Lines Sensor
11:12:52.562 INFO: Sensor Lines Sensor (done) | time=78ms
11:12:52.562 INFO: Sensor SCM Sensor
11:12:52.562 INFO: SCM Publisher is disabled
11:12:52.562 INFO: Sensor SCM Sensor (done) | time=0ms
11:12:52.562 INFO: Sensor C#
11:12:52.562 INFO: SonarAnalyzer.Scanner needs to be executed: false
11:12:52.562 INFO: Importing analysis results from d:\b\210\474\.sonarqube\out\ReferenceManager_x86_Release_1481\output-cs
11:12:53.078 WARN: Protobuf file not found: d:\b\210\474\.sonarqube\out\ReferenceManager_x86_Release_1481\output-cs\issues.pb
11:12:53.718 INFO: Sensor C# (done) | time=1156ms
Didn't find anything about this error using google. Any help appreciated!
By looking at the code in CSharpSencor.java, I think the difference between command line build and team build is caused by the RoslynReportPath:
void executeInternal(SensorContext context) {
boolean requiresAnalyzerScannerExecution = !config.isReportsComingFromMSBuild();
LOG.info("SonarAnalyzer.Scanner needs to be executed: " + requiresAnalyzerScannerExecution);
String roslynReportPath = settings.getString(config.getRoslynJsonReportPathProperty());
boolean hasRoslynReportPath = roslynReportPath != null;
Path protobufReportsDirectory;
if (requiresAnalyzerScannerExecution) {
// MSBuild 12 or MSBuild 14 with old scanner
analyze(!hasRoslynReportPath, context);
protobufReportsDirectory = protobufReportPathForMSBuild12(context);
} else {
protobufReportsDirectory = config.protobufReportPathFromScanner();
}
LOG.info("Importing analysis results from " + protobufReportsDirectory.toAbsolutePath().toString());
importResults(context, protobufReportsDirectory, !hasRoslynReportPath);
if (hasRoslynReportPath) {
LOG.info("Importing Roslyn report");
importRoslynReport(roslynReportPath, context);
}
}
command line build:
...
Generating SonarQube project properties file to C:\Users\Build-SW-Dev\Desktop\.sonarqube\out\sonar-project.properties
The supplied Code Analysis ErrorLog file is a valid json file and does not need to be fixed: d:\b\210\474\src\TPA\DevTools_Source\internal\ReferenceManager\main\ReferenceManager\bin\Release\ReferenceManager.exe.RoslynCA.json
...
08:41:57.276 INFO: Sensor C#
08:41:57.291 INFO: SonarAnalyzer.Scanner needs to be executed: false
08:41:57.291 INFO: Importing analysis results from C:\Users\Build-SW-Dev\Desktop\.sonarqube\out\ReferenceManager_x86_Release_6194\output-cs
08:41:59.948 INFO: Importing Roslyn report
08:42:00.448 INFO: Sensor C# (done) | time=3172ms
team build:
...
Generating SonarQube project properties file to d:\b\210\474\.sonarqube\out\sonar-project.properties
No Code Analysis ErrorLog file found at d:\b\210\474\bin\ReferenceManager.exe.RoslynCA.json.
...
11:12:52.562 INFO: Sensor C#
11:12:52.562 INFO: SonarAnalyzer.Scanner needs to be executed: false
11:12:52.562 INFO: Importing analysis results from d:\b\210\474\.sonarqube\out\ReferenceManager_x86_Release_1481\output-cs
11:12:53.078 WARN: Protobuf file not found: d:\b\210\474\.sonarqube\out\ReferenceManager_x86_Release_1481\output-cs\issues.pb
11:12:53.718 INFO: Sensor C# (done) | time=1156ms
While I do not have a solution to that yet, a workaround is to just redirect the output of the msbuild scanner into a file. TFS then does not see the warning and does not make the build partially succeeded.
edit:
Now I have found the root cause :) During team build, all output files get moved into a subfolder by a script. That's why SonarQube cannot find the file 'd:\b\210\474\bin\ReferenceManager.exe.RoslynCA.json'.
The file is located in a subfolder...
10:43:16.645 INFO - 0 files indexed
10:43:16.646 INFO - 0 files ignored because of inclusion/exclusion patterns
10:43:16.666 INFO - Quality profile for java: Sonar way
10:43:21.609 INFO - Sensor Lines Sensor
10:43:21.610 INFO - Sensor Lines Sensor (done) | time=1ms
10:43:21.610 INFO - Sensor SCM Sensor
10:43:21.610 INFO - No SCM system was detected. You can use the 'sonar.scm.provider' property to explicitly specify it.
10:43:21.611 INFO - Sensor SCM Sensor (done) | time=1ms
10:43:21.611 INFO - Sensor XmlFileSensor
See "SonarQube doesn't show any issues with Maven project", and check your Sonar server: log in as administrator, click on Settings, then System, then Update Center.
Check if the Sonar Java plugin is installed. If not, click on Available Plugins to download and install it.