In Jenkins job, my grails target does:
clean compile "test-app -unit -integration"
And it outputs the tests results twice.
I check .jenkins/jobs/myjob/target/test-reports
and there are XML corresponding to the tests but there is no duplication. So everything look likes it only executes once. Same with the console log - I can only see the test execute once.
However, when I look at the build results on Jenkins all the tests are duplicated.
I go to:
.jenkins/myjob/builds/buildnumber/junitResult.xml
and I can see the tests duplciated in this.
So it is as if when Jenkins creates the junitResult.xml file it copies tests.
Any ideas why?
Related
I have a simple application with some tests.
Actually, there is 1 JUnit test and some Specifications.
The thing is that when I run each of them separately they work fine.
But when I run:
grails test-app
It's not executing all the tests. All the tests are unit tests but they are in separate packages. But even into the package from the one it's executing, there is another test almost equal, but this one is not being executed as well...
Running grails test-app -unit I get the same problem.
Someone know if I need to do something else to be able to execute them all executing the grails test-app command?
Hm my previous answer was deleted for some reason, but be sure that the Spock test ends with Spec, rather than Test. If it ends with Test, then it will not be picked up.
e.g. MyTest -> MySpec
We just hit an issue with yeoman-generator tests when they would pass when run in isolation but fail when run in parallel with other tests.
Specifically, we call require('yeoman-generator').test.run() to run the generator and then use require('yeoman-generator').assert.file to check that the correct files were generated, which is what the documentation says. However, the assert would sometimes fail saying the files don't exist.
How does the interaction between test.run() and assert.file work? Where are the files written? Is is a global variable / temp file that is always the same and therefore can be overwritten by other tests running at the same time?
This is the test, and an example of a failing build.
There's a github issue with detailed discussion and here's a discussion on how the tests suddenly started passing when run in isolation.
We are using the Jest testing framework which runs tests in parallel.
Looks like Yeoman tests can't be run in parallel.
require('yeoman-generator').test.run() does create a temp directory but then changes the current working directory to that directory. This interferes with other tests that also rely on the CWD and therefore the Yeoman tests can't be run in parallel with other tests.
Relevant comment in run-context.js and process.chdir in helpers.js.
I am using Spock plug-in in my grails-2.3.4 application for automated unit and integration tests. When I run grails test-app, all the test cases run two times. Also in test report, every spec file is listed twice. As the application grew, number of test cases also grew, and all of them run twice. This takes double time to execute all of the test cases while development and deploying through Jenkins. Can anyone help me fix it (any help will be appreciated)?
http://grails.github.io/grails-doc/2.3.4/guide/upgradingFromPreviousVersionsOfGrails.html -> Spock included by default
You no longer need to add the Spock plugin to your projects. Simply
create Spock specifications as before and they will be run as unit
tests. In fact, don't install the Spock plugin, otherwise your
specifications will run twice [...].
I use selenium WebDriver with junit, ant and jenkins.
I set up jenkins to use ant build.xml to run my tests. But currently I run only one tests. In build.xml I set variable which is used in each test. So to run test in Jenkins I set in Targets:
build MyTest1 -Dvariable="value"
I want to run all tests in sequence one after another. I try this:
build MyTest1 -Dvariable="value" MyTest2 -Dvariable="value"
But 2 tests began run in browser at the same time. How can I organize needed sequence. Maybe there are some ways to do it in build.xml? I guess I can create target, in which call targets which runs tests, but how set my variable in that case? I'm new to ant so please advice me solution.
I need to clarify - my tests are independent, I won't run them in some stable sequence. The problem is that tests are running in parallel in browser. I need to run first test and only after it finish - run second test.
First of all, tests should have little dependencies. In your case, your tests depend on a global variable - try to get rid of it. Use a "configuration" object that you can modify from tests in a safe way and which your application code then uses to configure itself.
Which reduces the problem above to "how do I collect a number of tests" to which the answer is: Use a test suite:
import org.junit.runner.RunWith;
import org.junit.runners.Suite;
#RunWith(Suite.class)
#Suite.SuiteClasses({
MyTest1.class,
MyTest2.class
})
public class JunitTestSuite {
}
I finally got Hudson to build my project and the corresponding test project (using the XMLTestRunner2 unit provided in the Embarcadero forum). Running the test executable manually correclty produces a "dunit-report.xml" file with the test results.
I can't get Hudson to call my executable and produce this file though.
What I did is to create a build step as a Windows batch command and just call the executable. I tried several things:
bin\Test.exe
start bin\Test.exe
start /wait bin\Test.exe
start /wait /b bin\Test.exe
I cannot get it to work. It either returns immediately with some random exit code or it does not produce the XML output file. This must be very simple but I'm a little bit frustrated by now, because I don't get it to work.
What's the right way to execute the unit tests from Hudson?