I want to integrate the Specs2 test results with Jenkins.
I was added the below properties in sbt:
resolver:
"maven specs2" at "http://mvnrepository.com/artifact"
libraryDependencies:
"org.specs2" %% "specs2" % "2.0-RC1" % "test",
System Property:
testOptions in Test += Tests.Setup(() => System.setProperty("specs2.outDir", "/target/specs2-reports")) //Option1
//testOptions in Test += Tests.Setup(() => System.setProperty("specs2.junit.outDir", "/target/specs2-reports")) //Option2
testOptions in Test += Tests.Argument(TestFrameworks.Specs2, "console", "junitxml")
If I run the below command, it is not generating any specs reports in the above mentioned directory("/target/specs2-reports").
sbt> test
If I run the below command, it is asking for the directory as shown in the below error message:
sbt> test-only -- junitxml
[error] Could not run test code.model.UserSpec: java.lang.IllegalArgumentException: junitxml requires directory to be specified, example: junitxml(directory="xxx")
And it is working only if I give the directory as shown below:
sbt> test-only -- junitxml(directory="\target\specs-reports")
But sometimes its not generating all the specs report xmls (some times generating only one report, sometimes only two reports etc.).
If I give test-only -- junitxml(directory="\target\specs-reports") in the jenkins it is giving the below error.
[error] Not a valid key: junitxml (similar: ivy-xml)
[error] junitxml(
[error] ^
My main goal is, I want to generate the consolidated test reports in junit xml format and integrate with Jenkins. Kindly help me to solve my problem.
Best Regards,
Hari
The option for the junitxml output directory is: "specs2.junit.outDir" and the default value is "target/test-reports".
So if you don't change anything you could just instruct Jenkins to grab the xml files from "target/test-reports" which is what I usually do.
Also you might have to enclose your sbt commands in Jenkins with quotes. This is what I typically do:
"test-only -- html junitxml console"
Related
I have integrated Tosca with jenkins to run in local .The test runs successfully on jenkins .However I see below error in result.xml file .The tests run fine in Tosca its only in Jenkins that I see this error .What do I do?
This is the result.xml output
<testcase name="Shipping Costs" time="12.7" timestamp="2022-10-07T16:04:47.1086587+05:30" log="- Failed Shipping Costs {LogInfo='Execution will be continued with the next TestCase, because there was an error in the current TestCase.'} + Passed Open Web Shop - Failed Navigate to Log In Page {LogInfo='No matching tab was found with the following properties: Title=Demo Web Shop*'} ">
- Failed Shipping Costs {LogInfo='Execution will be continued with the next TestCase, because there was an error in the current TestCase.'} + Passed Open Web Shop - Failed Navigate to Log In Page {LogInfo='No matching tab was found with the following properties: Title=Demo Web Shop*'}
I'm running a Jenkins 2.25 server on Windows Server 2012. At the moment we're using the Maven Integration Plugin 2.12.1 and the Job DSL Plugin 1.57.
I've written DSL scripts for around 200 existing jobs on our server.
For any jobs that use Maven, either as a build step or as an actual Maven, I'm having a really frustrating issue. When I run the generated jobs, they fail with the following output.
12:17:12 [ERROR] Failed to execute goal com.googlecode.maven-download- plugin:download-maven-plugin:1.3.0:wget (default) on project myprojecy: The parameters 'url' for goal com.googlecode.maven-download-plugin:download-maven-plugin:1.3.0:wget are missing or invalid -> [Help 1]
12:17:12 [ERROR]
12:17:12 [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
12:17:12 [ERROR] Re-run Maven using the -X switch to enable full debug logging.
12:17:12 [ERROR]
12:17:12 [ERROR] For more information about the errors and possible solutions, please read the following articles:
12:17:12 [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginParameterException
Initially we thought we identified the issue as being that we were missing the XML snippet, even thought it appeared these settings were as they should be in the UI.
<settings class="jenkins.mvn.DefaultSettingsProvider"/>
<globalSettings class="jenkins.mvn.DefaultGlobalSettingsProvider"/>
<injectBuildVariables>false</injectBuildVariables>
So added this to the scripts:
configure { node ->
node / settings (class: 'jenkins.mvn.DefaultSettingsProvider') {
}
node / globalSettings (class: 'jenkins.mvn.DefaultGlobalSettingsProvider') {
}
node / injectBuildVariables ('false') {
}
}
But the jobs still fail when I try to run them, even though the XML now contained this snippet as expected.
Now two very bizarre things that I can't work out which are clearly related. Firstly, after the jobs fail, if I manually select "configure" for the job, then save it (i.e. don't make any actual changes), the job runs fine forever more (until a seed job us run and then it fails again).
Secondly, in the job config history after I run the seed job, I see the changes made when the seed job runs under the System user. However, within a matter of seconds, every time, another configuration change is recorded under my username, despite the fact that I have not made any changes to the job config - this is independent of me saving the job without making changes, by the way, it happens instantly.
I should add that further inspection suggests to me that there is some default settings for Maven which are not being applied to my DSL generated jobs. When adding the -X switch to the Maven goals, I could see more information about where these jobs are failing. The output is:
15:06:31 [DEBUG] Goal: com.googlecode.maven-download-plugin:download-maven-plugin:1.3.0:wget (default)
15:06:31 [DEBUG] Style: Regular
15:06:31 [DEBUG] Configuration: <?xml version="1.0" encoding="UTF-8"?>
15:06:31 <configuration>
15:06:31 <cacheDirectory>${download.cache.directory}</cacheDirectory>
15:06:31 <checkSignature default-value="false">${checkSignature}</checkSignature>
15:06:31 <failOnError default-value="true"/>
15:06:31 <outputDirectory default-value="${project.build.directory}">D:\data\jenkins\workspace\project\target</outputDirectory>
15:06:31 <outputFileName>${jarsigner.keystore.filename}</outputFileName>
15:06:31 <overwrite>${download.overwrite}</overwrite>
15:06:31 <readTimeOut default-value="0"/>
15:06:31 <retries default-value="2"/>
15:06:31 <session>${session}</session>
15:06:31 <skip default-value="false">${download.plugin.skip}</skip>
15:06:31 <skipCache default-value="false"/>
15:06:31 <unpack default-value="false">false</unpack>
15:06:31 <url>${jarsigner.keystore.url}</url>
15:06:31 </configuration>
Where in the successful run of the job (post fake config change) some of those fields are full, for example a URL for the keystore. This is obviously the problem, but I don't know what to do. As far as I can tell this should be resolved by including the configure block above in the groovy, but somehow my jobs are missing this (but they have it after saving the job again with no changes).
Can anyone see what I am doing wrong here?
The issue is this code in the XML which is automatically generated:
<jvmOptions></jvmOptions>
It seems that despite being empty this is overriding any default Maven options but then when the job gets saved again this is taken out because it is empty. Resolved by adding this to the groovy script:
configure({
it.remove(it / 'jvmOptions')
})
This seems likely to be a bug in the DSL but it's surprising that my colleagues and I have been unable to find any mention of this. Anyway, the above resolved this for me.
One of my Jenkins job is executing MSTest. I am passing the following command to
Execute Windows batch command:
del TestResults.trx
"C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\MSTest.exe" /testcontainer:D:\Projects\Jenkins\TestResultVerificationFromJenkins\TestResultVerificationFromJenkins\bin\Debug\TestResultVerificationFromJenkins.dll /resultsfile:TestResults.trx /nologo /detail:stdout
At the time of execution, Console Output is displaying the following values:
Starting execution...
Results Top Level Tests
------- ---------------
Passed TestResultVerificationFromJenkins.UnitTest1.PassTest
[stdout] = Test is passed*
1/1 test(s) Passed
Summary
Test Run Completed.
Passed 1
Total 1
Results file: C:\Program Files (x86)\Jenkins\jobs\JenkinsTestResultReader\workspace\TestResults.trx
Test Settings: Default Test Settings
In the post build step, I have to pass the MS test result "Test is passed" to a HTTP Request.
Is there any way to save this result in a Jenkins variable so that I can pass that to HTTP Request?
Regards,
Umesh
Since you are in the postbuild step, would parsing the console output for the test result and sending it off to the HTTP Request be an option for you?
For example, using Groovy Postbuild plugin, you could write a small script that could do this.
Perhaps something like:
if(manager.build.logFile.text.indexOf("Test Run Completed. Passed") >= 0)
manager.listener.logger.println (new URL("http://localhost?parameter=Test+is+passed")).getText()
My project has a number of packages ("models", "controllers", etc.). I've set up Jenkins with the Cobertura plugin to generate coverage reports, which is great. I'd like to mark a build as unstable if coverage drops below a certain threshold, but only on certain packages (e.g., "controllers", but not "models"). I don't see an obvious way to do this in the configuration UI, however -- it looks like the thresholds are global.
Is there a way to do this?
(Answering my own question here)
As far as I can tell, this isn't possible -- I haven't seen anything after a couple days of looking. I wrote a simple script that would do what I want -- take the coverage output, parse it, and fail the build if coverage of specific packages didn't meet certain thresholds. It's dirty and can be cleaned up/expanded, but the basic idea is here. Comments are welcome.
#!/usr/bin/env python
'''
Jenkins' Cobertura plugin doesn't allow marking a build as successful or
failed based on coverage of individual packages -- only the project as a
whole. This script will parse the coverage.xml file and fail if the coverage of
specified packages doesn't meet the thresholds given
'''
import sys
from lxml import etree
PACKAGES_XPATH = etree.XPath('/coverage/packages/package')
def main(argv):
filename = argv[0]
package_args = argv[1:] if len(argv) > 1 else []
# format is package_name:coverage_threshold
package_coverage = {package: int(coverage) for
package, coverage in [x.split(':') for x in package_args]}
xml = open(filename, 'r').read()
root = etree.fromstring(xml)
packages = PACKAGES_XPATH(root)
failed = False
for package in packages:
name = package.get('name')
if name in package_coverage:
# We care about this one
print 'Checking package {} -- need {}% coverage'.format(
name, package_coverage[name])
coverage = float(package.get('line-rate', '0.0')) * 100
if coverage < package_coverage[name]:
print ('FAILED - Coverage for package {} is {}% -- '
'minimum is {}%'.format(
name, coverage, package_coverage[name]))
failed = True
else:
print "PASS"
if failed:
sys.exit(1)
if __name__ == '__main__':
main(sys.argv[1:])
I am seeing this error for quite sometime now.
I am running ant build on CYGWIN which inturn runs on WindowsXP.
The resolution(bad one) I found was to delete my gcct/first directory and run ant build again (which runs from another directory). It runs successfully but if I modify some code under gcct/first, I do not want to delete it because of this error.
I did see this link. The resolution here does not apply to me since I do not have .cvspass defined anywhere in the build.xml.
C:\svn\CEL_v3681\buildCore.xml:1883: cvs exited with error code 1
Command line was [Executing 'cvs' with arguments:
'checkout'
'-A'
'-rfirst_v2_126'
'gcct/first'
The ' characters around the executable and arguments are
not part of the command.
environment:
ALLUSERSPROFILE=C:\Documents and Settings\All Users
ANT_HOME=C:/Apps/Apache/apache-ant-1.7.0
APPDATA=C:\Documents and Settings\shankarc\Application Data
CLASSPATH=./;C:/Program Files/Java/jre1.5.0_07/lib/ext/QTJava.zip
COMMONPROGRAMFILES=C:\Program Files\Common Files
COMPUTERNAME=NYKPWM2035798
COMSPEC=C:\WINNT\system32\cmd.exe
CUSTPROF=Roaming700Live
CVSROOT=:pserver:shankarc#amcvs2.lehman.com:/home/eqcvs/cmte
CVS_RSH=/bin/ssh
FP_NO_HOST_CHECK=NO
HOME=C:\Apps\CYGWIN\home\shankarc
HOMEDRIVE=F:
HOMEPATH=\
HOSTNAME=nykpwm2035798
IDEA_PROPERTIES=C:\Documents and Settings\shankarc\idea.properties
INFOPATH=/usr/local/info:/usr/share/info:/usr/info:
JAVA_HOME=C:/Program Files/Java/jdk1.6.0_21/
JDK_HOME=C:\Program Files\Java\jdk1.6.0_21\
LOGONSERVER=\\NYKPSM00069
MANPATH=/usr/local/man:/usr/share/man:/usr/man::/usr/ssl/man
NUMBER_OF_PROCESSORS=2
OS=Windows_NT
PATH=C:\Apps\CYGWIN\usr\local\bin;C:\Apps\CYGWIN\bin;C:\Apps\CYGWIN\bin;C:\Apps\CYGWIN\usr\X11R6\bin;C:\Apps\Apache\apache-ant-1.7.0\bin;C:\Program Files\Java\jdk1.6.0_21\bin\;C:\Apps\CYGWIN\bin;C:\Program Files\VisualSVN Server\bin;C:\Program Files\Sudowin\Clients\Console;C:\Program Files\Fortify Software\Fortify 360 v2.5.0\bin
PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.PSC1
PRINTER=\\NYKPSM04020\NYKLPR1301-03-03C05
PROCESSOR_ARCHITECTURE=x86
PROCESSOR_IDENTIFIER=x86 Family 6 Model 15 Stepping 6, GenuineIntel
PROCESSOR_LEVEL=6
PROCESSOR_REVISION=0f06
PROFGROUP=FONP
PROGRAMFILES=C:\Program Files
PROMPT=$P$G
PWD=/cygdrive/c/svn/CEL_v3681/gcct/cel
QHOME=c:\q
QTJAVA=C:\Program Files\Java\jre1.5.0_07\lib\ext\QTJava.zip
SESSIONNAME=Console
SHLVL=1
SITECODE=NYK
SITEIDENT=NYK
SVN_ASP_DOT_NET_HACK=1
SYSTEMDRIVE=C:
SYSTEMROOT=C:\WINNT
TEMP=C:\TEMP
TERM=cygwin
TMP=C:\TEMP
UATDATA=C:\WINNT\system32\CCM\UATData\D9F8C395-CAB8-491d-B8AC-179A1FE1BE77
USER=shankarc
USERDNSDOMAIN=INTRANET.BARCAPINT.COM
USERDOMAIN=INTRANET
USERNAME=shankarc
USERPROFILE=C:\Documents and Settings\shankarc
WINDIR=C:\WINNT
CVS_PASSFILE=C:\Apps\CYGWIN\home\shankarc\.cvspass]
Total time: 58 seconds
How I resolve this?
I had the same issue and found that even though I was not using .cvspass I did have a build property of cvs.pass set which needed to be reset to OVERRIDE to function depending on how you set up your cvs access (though it looked similar from your post). This needed to be changed in build.properties and .build.properties. Hope this helps!