I finally got Hudson to build my project and the corresponding test project (using the XMLTestRunner2 unit provided in the Embarcadero forum). Running the test executable manually correclty produces a "dunit-report.xml" file with the test results.
I can't get Hudson to call my executable and produce this file though.
What I did is to create a build step as a Windows batch command and just call the executable. I tried several things:
bin\Test.exe
start bin\Test.exe
start /wait bin\Test.exe
start /wait /b bin\Test.exe
I cannot get it to work. It either returns immediately with some random exit code or it does not produce the XML output file. This must be very simple but I'm a little bit frustrated by now, because I don't get it to work.
What's the right way to execute the unit tests from Hudson?
Related
I am completely new with Jenkins and have this question. So I created two projects on Jenkins. The first project will fetch from github if it notices any kind of change on my repo and will store a local copy on my machine.
The second project will start only if the first project is stable. If it is stable, the second project will first compile the all the java files (which I put in a compile.bat file), then it will run the testng.xml file which runs junit test and selenium test (run.bat file).
So what I want to do is if there is no compilation errors on compile.bat, then proceed on run.bat, but right now even though there are some errors on the java file and when the compile.bat runs, it catches those errors, Jenkins still proceeds to the run.bat file and at the end passes the build. I want the build to fail when there is any kind of errors.
Here is the link to my repo for the batch file and other files if that helps:
https://github.com/na2193/Demo
I had figured it out. You need to use conditional step(single) and there you can define what you want to do
I have my selenium tests written using SpecFlow(+SpecRun) and NUnit framework (v.3.8.1.0). I've configured Jenkins to run these tests. My Jenkins Windows Batch Command is as follows:
"C:\Program Files (x86)\NUnit.ConsoleRunner\3.7.0\tools\nunit3-console.exe"
C:\Projects\Selenium\ClassLibrary1\PortalTests\bin\Debug\PortalTests.dll
--test=TransactionTabTest;result="%WORKSPACE%\TestResults\TestR.xml";format=nunit3
When I trigger build test seems to start running as I'm getting as far as end of NUNIT3-CONSOLE [inputfiles] [options] with spinner indicating that test is running but it actually never ends and estimated remaining time is: N/A.
Now, when I run this script with windows cmd.exe:
"[PATH to Console.exe]\nunit3-console.exe" PortalTests.dll -- test=TransactionTabTest
this test pass successfully and so does in VS.
Now, I know this is very generic question but any clues will be much appreciated.
As you are using SpecFlow+Runner/Specrun, you can find the documentation how to configure it for the different build servers here: http://specflow.org/plus/documentation/SpecFlowPlus-and-Build-Servers/
I'm relatively new to using build files and Ant to compile and execute from a command line so i wanted to know how it would be possible to modify the build-impl.xml file auto generated from netbeans so that after it compiles it also executes the program.
Currently when i type just "ant" in the command window where the build.xml file is listed it compiles and etc but then says
-do-jar-copylibs:
[copylibs] Nothing to copy.
[echo] To run this application from the command line without Ant, try:
[echo] java -jar "C:\Users\Eric\Documents\Word Documents\Java Test Code\NetbeansVersion\Cops\dist\Cops.jar"
I can run that command and it will run fine, but i want the program to execute by just typing ant.
The build.xml file - Pastebin
Build-impl.xml file - Pastebin
There are a couple "tasks" available in ant that you could use to accomplish this.
You can use either of these:
Java Task,
Exec Task
Those documentation pages provide examples as well. Before you go there though, you might want to start at the basic manual to get an idea of the structure of an ant build file to determine where and how you want to add this execution step in.
It feels a little "off" to me to have your build script executing the program, but I'm sure you've got a reason, so I might suggest adding a new target that does the build steps and then also runs this command to kick off the program. That way you've still got the option of running the build without running the program.
I invoke our gtest suite for iOS in Jenkins using the shell script
#!/bin/sh
pkill -a "iPhone Simulator"
ios-sim launch ${WORKSPACE}/source/apple/build/Debug-iphonesimulator/MyAppTest.app --args --gtest_output=xml:${WORKSPACE}/JUnitTestResultsIOS.xml
exit $?
This always successfully runs the tests, and when the tests pass the xml file gets generated as expected. However, when the tests fail, no xml file is generated, and the "Execute shell command" build step terminates but does not fail the job. I echoed the exit code and it comes back 0 even when the tests fail.
This is even more confusing to me since we have a basically identical script in the same job for running tests on our OSX version. This always writes the xml and successfully fails the job when the tests fail.
This behavior seems totally arbitrary and everything about our configuration seems to be exactly as it should be. What am I missing?
Thanks!
There were two things at work here.
First of all, we had the break_on_failure gtest option enabled, which works great when running tests on a local machine but isn't useful within Jenkins, so we disabled it on the build machine.
The second issue was around how we used the exit code. Since ios-sim launch ... always succeeds we were always getting an exit code of 0, regardless of whether the tests passed or failed. I ended up using grep to determine if the resulting xml file indicated any failures, and generated an exit code based on that.
My ant files are already present and I am not allowed to change those for various reasons. I have created a batch file and I am writing all those ant commands in that batch file just to automate the process. The code looks something like this
1. cd abc
2. ant realclean && ant
3. cd..
4. cd pqr
5. ant realclean && ant
6. cd..
However now I have to check if the build fired on line 2 is successful then only goto line 3 otherwise exit.
I googled a bit and found %errorlevel% as one option but it is not working in my case.
Any suggestions?
Thanks in advance.
How about adding another ANT file which contains the tasks to execute the existing ANT files?
That would give you error handling without doing anything, you could reuse that file on different OSs and your CI server and the syntax would be saner than batch...
You can have ANT execute, say project.xml, with ant -f project.xml
[EDIT] You don't even have to change the existing projects: Just create a new project, assign paths to the old projects to properties in the new build.xml and then you can build the old projects from the new one.
This gives you:
A sane[*] syntax that you already know (no need to learn the ugly/buggy/handicapped rules of DOS batch programming; you did know that DOS was once named QDOS "Quick and Dirty OS", yes?)
Proper error handling
A simple way to pass arguments to the sub-builds
Cross-platform support
Useful error messages when something goes wrong
[*]: Compared to DOS...