I'm using TFS 2017 and SonarQube 5.6.2 5.6.4 6.2.
I'm trying to setup SonarQube integration into pull requests. On pull requests that don't appear to have any issues the sonarqube analysis runs fine. It looks like it only fails when there are issues found and it tries to read the sonar-report.json to post the issues to the pull request. I'm receiving the following error on builds that appear to find issues:
------------------------------------------------------------------------
EXECUTION SUCCESS
------------------------------------------------------------------------
Total time: 5:25.577s
Final Memory: 56M/600M
------------------------------------------------------------------------
The SonarQube Scanner has finished
Creating a summary markdown file...
Analysis results: http://somedomain:9000/dashboard/index/CP
Post-processing succeeded.
Fetching code analysis issues and posting them to the PR...
System.Management.Automation.RuntimeException: Could not find the SonarQube issue report at D:\agent2-TFS-Build01\_work\13\.sonarqube\out\.sonar\sonar-report.json. Unable to post issues to the PR. ---> System.IO.FileNotFoundException: Could not find the SonarQube issue report at D:\agent2-TFS-Build01\_work\13\.sonarqube\out\.sonar\sonar-report.json. Unable to post issues to the PR.
--- End of inner exception stack trace ---
at System.Management.Automation.Runspaces.PipelineBase.Invoke(IEnumerable input)
at System.Management.Automation.PowerShell.Worker.ConstructPipelineAndDoWork(Runspace rs, Boolean performSyncInvoke)
at System.Management.Automation.PowerShell.Worker.CreateRunspaceIfNeededAndDoWork(Runspace rsToUse, Boolean isSync)
at System.Management.Automation.PowerShell.CoreInvokeHelper[TInput,TOutput](PSDataCollection`1 input, PSDataCollection`1 output, PSInvocationSettings settings)
at System.Management.Automation.PowerShell.CoreInvoke[TInput,TOutput](PSDataCollection`1 input, PSDataCollection`1 output, PSInvocationSettings settings)
at Microsoft.TeamFoundation.DistributedTask.Handlers.PowerShellHandler.Execute(ITaskContext context, CancellationToken cancellationToken, Int32 timeoutInMinutes)
at Microsoft.TeamFoundation.DistributedTask.Worker.JobRunner.RunTask(ITaskContext context, TaskWrapper task, CancellationTokenSource tokenSource)
As far as I can tell, there is no way to configure the location of this report. One thing I read was that if you aren't failing the build on quality gate failures then you should also not enable the option to include full analysis report in the build summary. There were no specifics on why this is the case. I currently have the option to include the report enabled. Could this be the cause of my problem? We are not failing builds on quality gate failures yet because I was trying to give devs some time to adjust to the changes. Anyone know what's going on here? Here is a screenshot of my prepare analysis settings.
You can find a copy of the logs for the end analysis step here. I've retracted the domain and username. Everything else in the log is left untouched.
Edit 1/19:
Since I don't have fail on quality gate failure enabled I am not getting an error message if the build does fail the quality gate. I'm receiving the error message that I posted above about the sonar-report.json missing. Here is what I am seeing for pull requests with zero issues.
Fetching code analysis issues and posting them to the PR...
SonarQube found 0 issues out of which 0 are new
True
Processing 0 new messages
No new messages were posted
Uploading the legacy summary report. The new report is not uploaded if not enabled, if the SonarQube server version is 5.2 or lower or if the build was triggered by a pull request
The build was not set to fail if the associated quality gate fails.
This is why I think it only happens on builds that do have some issues to post to the PR. When is the sonar-report.json that it's looking for supposed to be written? I looked in the workspace on the build server and the file is definitely not there.
Edit 1/30/17:
Here is some additional info that may help figure out what it is. Currently I have 26 projects that run a SQ analysis on both PR and CI builds. All CI builds are working as expected. 24/26 PR builds are working as expected. The only two builds that are failing are both webapp projects. One of them is made up of C#, TypeScript, and JavaScript. The other is C#, VB, TypeScript, and Javascript. All the projects that are succeeding are strictly C# applications. What I have noticed is that on all of the C# applications I see a log message similar to this:
INFO: Performing issue tracking
INFO: 1033/1033 components tracked
INFO: Export issues to D:\agent2-TFS-Build01\_work\35\.sonarqube\out\.sonar\sonar-report.json
INFO: ANALYSIS SUCCESSFUL
This little Export bit is missing from the failed builds. Here is a log snippet from one of the failed PR builds. I would expect the logs to be in somewhat similar order but the Export issues to ... section is missing. Also it looks like it's uploading a full analysis report even though this is a PR build:
INFO: Analysis report generated in 2672ms, dir size=10 MB
INFO: Analysis reports compressed in 2750ms, zip size=4 MB
INFO: Analysis report uploaded in 583ms
INFO: ANALYSIS SUCCESSFUL, you can browse http://~:9000/dashboard/index/AppName
I'm getting tons of Missing blame information for the following files: [long list of files] errors during the analysis for the JS files that are being generated by the TypeScript compiler. I don't know if those errors would have anything to do with why this is failing but it's the only errors I see in the logs for that build step.
So, now I guess the problem to figure out is why are the issues for the two web app projects not being exported to the sonar-report.json. As you can see in the following log messages it appears that both projects are being started as PR builds with analysis mode set to issues and export path set to sonar-report.json but by the time the scan is done it skips the export step.
##[debug]Calling InvokeGetRestMethod "/api/server/version"
##[debug]Variable read: MSBuild.SonarQube.HostUrl = http://somedomain:9000/
##[debug]Variable read: MSBuild.SonarQube.ServerUsername = ********
##[debug]Variable read: MSBuild.SonarQube.ServerPassword =
##[debug]GET http://somedomain:9000/api/server/version with 0-byte payload
##[debug]received 3-byte response of content type text/html;charset=utf-8
##[debug]/d:sonar.ts.lcov.reportPath="C:\TFS-Build02-Agent2\_work\11\s\Source\AppName.Web\CodeCoverage\lcov\lcov.info" /d:sonar.ts.tslintconfigpath="C:\TFS-Build02-Agent2\_work\11\s\Source\AppName.Web\tslint.json" /d:sonar.ts.tslintruledir="C:\TFS-Build02-Agent2\_work\11\s\Source\AppName.Web\tslint-rules\" /d:sonar.ts.tslintpath="C:\TFS-Build02-Agent2\_work\11\s\Source\AppName.Web\node_modules\tslint\bin\tslint" /d:sonar.analysis.mode=issues /d:sonar.report.export.path=sonar-report.json
And the second one that is failing:
##[debug]Calling InvokeGetRestMethod "/api/server/version"
##[debug]Variable read: MSBuild.SonarQube.HostUrl = http://somedomain:9000/
##[debug]Variable read: MSBuild.SonarQube.ServerUsername = ********
##[debug]Variable read: MSBuild.SonarQube.ServerPassword =
##[debug]GET http://somedomain:9000/api/server/version with 0-byte payload
##[debug]received 3-byte response of content type text/html;charset=utf-8
##[debug]/d:sonar.ts.lcov.reportpath="D:\agent2-TFS-Build01\_work\13\s\Source\AppName.WebApp\CodeCoverage\lcov\lcov.info" /d:sonar.ts.tslintconfigpath="D:\agent2-TFS-Build01\_work\13\s\Source\AppName.WebApp\tslint.json" /d:sonar.ts.tslintruledir="D:\agent2-TFS-Build01\_work\13\s\Source\AppName.WebApp\tslint-rules\" /d:sonar.ts.tslintpath="D:\agent2-TFS-Build01\_work\13\s\Source\AppName.WebApp\node_modules\tslint" /d:sonar.analysis.mode=issues /d:sonar.report.export.path=sonar-report.json
I have finally had some time to circle back around to figuring out this problem. It appears that the SonarQube VSTS task doesn't export the issues report if it encounters an error during the end analysis step. I went through the build logs and made sure to exclude any of the js files that are generated by the typescript compiler. This helped to get rid of the many missing blame information errors that were happening. Once I had those files excluded from analysis I updated the SonarTsPlugin to the latest version along with all the associated build variables and PR integration started working again.
I know this worked because I no longer see the RuntimeException: Could not find the SonarQube issue report error. Now I see this in the logs:
SonarQube found 9264 issues out of which 102 are new
Processing 102 new messages
102 message(s) were filtered because they do not belong to files that were changed in this PR
If you are experiencing a problem like this I suggest you look at fixing any build errors you see in the logs for the end analysis step. Once I had them all resolved it started working as intended.
Related
I've set up an automatic "pull request check" via jenkins/github/sonarqube integration.
The workflow is as follows:
Github pull request created by user → Github Webhook triggers, and calls Jenkins API to execute sonarqube scanner → reports to sonarqube server → sonarqube server calls github API(create commit statuses : ref https://developer.github.com/v3/repos/statuses/) and posts a comment about the PR.
The issue is that it marks the PR as check failed just because it didn't pass its code health checks. The build passed, but the code is "dirty" - and that causes the PR to be marked as unacceptable. I'd like to find a way to prevent code quality checks from appearing as an actual status of the commit, and only allow commenting.
Additional images to provide some context:
SonarQube uses a techuser account token to post its analysis summary as a comment on the PR thread. (Sorry for the black boxes, corporate stuff..)
This functionality is everything we need, nothing more.
However... the plugin does one more thing, which is marking the commit as a failure. Note that we're already using something else to check for actual build failures. Although it didn't fail, sonarqube marking the commit as failure because of code quality makes the whole commit display as a failure. I'd like to prevent sonarqube from setting branch check statuses, while letting it comment on the issue. I couldn't find an option for anything like that neither in jenkins plugin configuration nor sonarqube admin page nor sonarqube scanner script documentation.
Thanks in advance.
What you want to achieve is currently not possible when using the SonarQube GitHub plugin, since this behaviour is hardcoded in the plugin and there is no configuration option to customize this.
In upcoming versions of SonarQube and SonarCloud, pull request will have a built-in support and the behaviour will be the following:
The status will be red if there is at least an open issue on the PR analyzed by SonarQube/SonarCloud
Teams will have the ability to mark those issues as "Confirmed" in SonarQube/SonarCloud (to acknowledge that they accept this technical debt), in which case the status will be automatically turned to green in GitHub
Our pipeline failed, but the graph in the developers console still shows it as been successful. However, the results were not written to BigQuery, and the job log clearly shows it failed.
Shouldn't the graph show it as failed too?
Another example:
Another example:
This was a bug with the handling of BigQuery outputs that was fixed in a release of the Dataflow service. Thank you for your patience.
We have a whole bunch of reports sitting in TFS 2010, then we decide to directly upgrade to TFS 2013.
Upgrading and configuration and have been done successfully.
However those old reports are not working, I fixed the data source connection string and rebuild the data warehouse successfully.
But I get the error:
An error has occurred during report processing. (rsProcessingAborted)
Query execution failed for dataset 'IterationParam'. (rsErrorExecutingCommand)
The dimension '[Iteration]' was not found in the cube when the string, [Iteration].[Parent_ID].[XXX], was parsed.
I looked in the analysis service database and could not find Itration Dimension.
Is something wrong? Please give me advice
cheers,
Did you rebuild just the warehouse, or the cube also? You need to do both. And you need to rebuild the warehouse first, then the cube afterwards.
I use the TFS Web Services to do this:
Fire up the web service page, for me its: http://localhost:8080/tfs/TeamFoundation/Administration/v3.0/WarehouseControlService.asmx
Execute GetProcessingStatus passing false for the includeOnlineHostsOnly
Make sure that all the jobs are currently Idle
Execute ProcessWarehouse leaving the args blank
Rerun GetProcessingStatus until all the warehouse jobs are idle again
Execute ProcessAnalysisDatabase passing in Full for the processingType
Rerun GetProcessingStatus until all the jobs are idle again
I have a Jenkins build that takes JSON output from a jruby/cucumber test and generates reports using the Cucumber Reports plugin. The plugin is only giving me sensible reports on a feature file basis: that is, it can tell me whether a feature file passed or not, but not any given step.
When I expect the steps, every step has this error message Result was missing for this step
I've heard about this happening with cucumber-jvm, but I am using jruby, which, as far as I know, has nothing to do with cucumber-jvm.
Any insight?
This has been reported as fixed in cucumber/gherkin 2.0.0 beta. You can follow the thread here: github.com/cucumber/gherkin/issues/165
My company is developing a web application that builds in ant. I've been tasked with getting CruiseControl.net to differentiate between a build failure and a unit test failure, which it can't do natively. ( It currently lumps both together but doesn't help developers understand what's broken )
I have CC.net call a script that returns specific exit codes depending on the nature of an ant task failure. I'd like these exit codes to be reflected in the CC.net failure report / dashboard but am having some trouble finding resources on how this might be done.
Any suggestions?
Not directly. All the reports and display works from information in the logs which are XML files. The display and reports work by applying XSLT to these XML files.
Take a look at your build logs and unit test logs, to see if each of those process write the failure information to their respective log files.
If they do, you should be able to write a custom XSLT or modify the existing XSLT to display that information.
Edit:
A different approach based on your comment. You could probably redirect the ANT error code to a file. Then you could have a seperate ccnet task that takes the error code from that file and re-format and display it (depending on how/where you want it displayed)