I'm trying to make a Jenkins job that only scans the test source files, so everything under /src/test/java (using Maven). I use the SonarQube Jenkins post-action for this.
When we used to configure Sonar in the pom file directly we could do this in a profile:
<sonar.sources>/src/test/java</sonar.sources>
<sonar.tests/>
That worked fine.
But in the Jenkins job I have to specify these as 'Additional properties' and I can't seem to specify an emtpy sonar.tests element. I tried -Dsonar.tests, -Dsonar.tests=,-Dsonar.tests="", nothing works. When this element is not empty Sonar will attempt to scan the test files twice and crash.
The post-build step is specifically and explicitly a Maven operation. Your problem comes from trying to use Maven to do something un-Mavenish; i.e. ignore the convention that files in the tests directory should be treated as tests.
Since you want to scan your tests as code, your best bet is to use the build step (which uses SonarQube Scanner) and set your scanner properties manually. That will make it easy to set your sources directory and to omit the tests directory.
Related
Some context
I am trying to configure the plugin 'Sonar Scanner' in our Jenkins, the trigger and the project validation is "working", I added the " " because it is actually validating everything, even the test files and it is not checking the coverage.
Things you should know before
I have no access to Jenkins machine
I have no access to SonarQube machine
All plugins installed from Jenkins are the most recent versions
My Config
My project folder structure looks like this (Java project):
.git
.scannerwork
bl
commons
datafeed
keyfiles
persistence
post-aggregator
project-setup
webapp
webapp-server
.gitignore
pom.xml
My Sonar Scanner config is:
#required metadata
sonar.projectKey=inventory
sonar.projectName=Inventory1
sonar.language=java
sonar.sourceEncoding=UTF-8
#sonar.tests=**/.*Test.java
sonar.exclusions=**/*Dto*, **/*Entity*
sonar.binaries=build/classes
sonar.java.binaries=/data/application/jenkins/jenkins/workspace/$JOB_NAME
#Adding more info to the log
sonar.verbose=true
#Path to source directory
sonar.sources=/data/application/jenkins/jenkins/workspace/$JOB_NAME
The problems
Test coverage is 0
Checking the report I see that the current Scanner config checks everything, even the tests
So.. the questions are:
My test files are spread all around in the project, how can (or if it is even possible) to add a regex to the "sonar.tests"? i have been trying to do it but no success, all my tests ends with Test.java
How can I avoid properly Sonar to scan the test files.
Do I need jacoco as well in order to make this work or its possible to do it without it? (we are configuring the jacoco-maven in the project)
Thank you for your time!
I have inherited a system which uses Jenkins Job DSL to build the jobs for all our projects, I have little experience with configuring Jenkins and none at all with Jenkins Job DSL, so please be gentle.
Some of these projects are Gradle projects. There is a function, createGradleJob() which creates the gradle job. In this function we build the task list for the job, as a string, based upon some features of the project. e.g. if it is being built from the master branch we append the 'publish' task. All of these conditional tasks are currently appended based upon the projects branch name, or the presence , or absence of certain files in the projects repo.
I would like to now add a new task into this task list conditional upon the contents of some of these files. Specifically if certain keywords are detected in the projects build.gradle file then certain tasks need to be appended to the task list.
So, is there a way in Jenkins Job DSL to check the contents of a file and use that as a conditional expression?
I have found that I can execute arbitrary shell commands using the shell function, so I thought I could just grep the file, but I can't locate the documentation for this function, so I'm not clear how I can able to access the output of the commands, so as to use them in a conditional expression.
I have found the textFinder function, but this appears to only allow you to fail (or mark as unstable) the build as a result of finding or failing to find, the text, not use the result as a conditional expression.
It sounds like you want to readFileFromWorkspace. It returns the contents of the file as a string. Simply read your file and parse the string as needed using the Groovy and/or Java string utils.
It's not quite clear from your question, but if you're talking about reading files out of the repo to be checked out by the job you're generating, this function won't help. But if the file is already somewhere in the workspace (i.e. it's one of the files checked out by the seed job), you'll be fine.
The shell command you found adds an "Execute Shell Script" build step to the job being generated. It doesn't actually execute the script there and then, it just copies the contents of the parameter verbatim into the build step ready to be executed when Jenkins runs the job.
For your continued sanity, here is a link to the Job DSL API Documentation
I would like to run Sonar plugin on Jenkins without any build process (my intent was to integrate Sonar analysis within Jenkins and take advantage of the subversion plugins and configurations we already had on there).
I do not want to run the build process since that would take up unnecessary time; I would only like to have a Jenkins job dedicated for Sonar analysis.
You can do that. You have to triggering the analysis with the SonarQube Runner.
Go to the Build section, click on Add build step and choose Invoke Standalone Sonar Analysis
Configure the SonarQube analysis. You can either point to an existing sonar-project.properties file or set the analysis properties directly in the Project properties field
When you analyse with SonarQube Runner , then you should give the following mandatory properties:
sonar.projectKey=my:project
sonar.projectName=My project
sonar.projectVersion=1.0
# Path to the parent source code directory.
# Path is relative to the sonar-project.properties file. Replace "\" by "/" on Windows.
# Since SonarQube 4.2, this property is optional. If not set, SonarQube starts looking for source code
# from the directory containing the sonar-project.properties file.
sonar.sources=src
In this case you may miss some rule violations (like FindBugs), because .class files are not provided. You have to build the project manually and set the sonar.binaries property to your class files. If you never want to build the project, then you can use the SourceMeter plugin for SonarQube too. It only needs the source files, but can produce more metrics and issues if you needed.
I am looking at using Jenkins on Windows.
I currently have an ant script. It works pretty well. Except for instance, when the build breaks because of a syntax error, I can not see the error in the Jenkins console log.
So I am thinking may be ant is not the best tool for use on Windows.
What do most of you use for Jenkins on Windows?
If it is ant, how do you send the build output, VStudio, to the Jenkins console?
Thanks
Primarily I use Jenkins with maven projects, although you should see the ANT output in the online logs regardless.
What are you building? Are you building Java projects? Ant is what you use. If you are bulding a C project, you should use Make. If you're building a VisualStudio project, you should use msbuild. You use the build tool for your project. Jenkins will execute them without a problem.
Take a look at the build in question. On the left side of the screen, there's a Console output item. Click on that. Is there any output. No matter what tool you use, Jenkins captures the STDOUT and STDERR in that console output. If nothing else, you should see the exact commands Jenkins is executing to checkout and to build your project. Try executing those commands.
Still, you didn't give us much to go on. No idea what you're building or what you're doing with Ant. You didn't state any error, the console output, or even what the Jenkins error log is stating.
Jenkins does two things:
It watches your repository for changes.
Once it detects the changes, it executes the very commands you'd execute to build the project.
Jenkins doesn't care whether you use Ant, Maven, Make, or simply do a del /s/q. Jenkins will simply execute the commands you tell it to execute.
Addendum
It is c, c++, Java and InstallShield. I use ant to do file copy and move, call msdev.exe project. Some Javac calls, InstallShield command line builds..
Jenkins can execute multiple step builds in a single job. After you specify the build step, you can press the Add button to add another build step. There's no reason that all the build steps even have to be of the same type. Just select a "Freestyle" build, and use the right build tools for the job.
There's an optional MSBUILD plugin in Jenkins that should do your MS Build. This should give you the complete output from MSBuild, so you can see any errors.
After you do your MSBuild step, you can create a second build step to run an Ant task to build your InstallShield. After that, you could run another build step to do the copying you need either on the command line or through something like Ant (or Nant.
Whatever the output of the various tools is the output you'll get in the build console.
dev.cmd shows the output and I continue to use ant.
I am new to Jenkins and just started configuring it. This is what i have done till now:
Installed and configured Jenkins to display the home page. Added PMD plugin.
Set the HUDSON_HOME to a specific directory > C:\Work\Jenkins
Configured a test build to run a simple do-nothing ant script. It runs successfully
Written an independent pmdbuild.xml to run checks on a set of files in C:\myview (I am using clearcase). This xml also copies the output pmd_results.xml to the workspace directory in $HUDSON_HOME/[job-name]/workspace
Now I added the pmdbuild.xml as a step in my primary build. So my build has 2 steps:
a. Run a simple script, do-nothing.
b. Run pmdbuild.xml which generate pmd_results.xml and place it in $HUDSON_HOME/[job-name]/workspace (HARD-CODED as Jenkins PMD plugin expects the file there)
Jenkins picks up the pmd_results.xml automatically with the plugin and displays warnings and everything.
Now the problem:
If I click on a filename in the PMD results, it gives a filenotfound exception as it is looking for the source file in $HUDSON_HOME/[job-name]/workspace.
My java code files are placed in C:\myview (a clearcase snapshot view)
My question is, do I need all my code files to be present inside $HUDSON_HOME/[job-name]/workspace ?? Meaning can't I tell Jenkins to look for the PMD input files in C:\myview or any other directory instead of $HUDSON_HOME/[job-name]/workspace ??
Sorry for the extremely long description.
Jenkins expects that all the code is in the workspace. Usually Jenkins is used to check out a copy of the code into the workspace, and then runs all build steps on the Sources in the Workspace.
Might seem restraining at first, but it saves you a lot of trouble if you need to move Jenkins to another server, or create a slave instance.
So I would suggest you let Jenkins check out your code (there should be a clearcase plugin) into the workspace, and run the analysis on the checked out code.
If there are compelling reasons why your code has to stay where it is (C:\myview in your case) you can still set the workspace of your build to that directory (find this in the job configuration page, you need to click on the 'extended' button to see the option).