Im working on getting our CI (Jenkins) to use and run our qunit test, we have qunit-reporter, qunit.compposite and phantomjs to execute the whole. Im not ant expert so im using windows batch command as a prestep with the following command
phantomjs src\test\webapp\js\runner.js src\test\webapp\jquery.all.test.html > test-report.xml
Now this runs and generates the junit xml file, with a few issues, firstly since I use pipe phantomjs performance information is also put into the file so that's my first issue
My second issue is that I have no idea in the world how I can get Jenkins to actually read the xml file instead of just considering it random stuff, similar to it interpreting when we have junit test run.
Just for reference the project is a maven project.
All advice is welcome :)
In Jenkins, configure your job, add a post build action, select publish JUnit test result report then add the path to your xml file in the requested field.
A couple years ago, a colleague of mine and I worked on getting PhantomJS to run QUnit tests and output JUnit XML that Jenkins can consume:
http://www.cameronjtinker.com/post/2013/09/24/QUnit-JSCoverage-and-Jenkins.aspx
I had forgotten to post this on my blog after we worked on this so I just posted it today when I saw your question. Much has changed since 2011 when this was originally written, but it should have most of the same concepts involved.
Related
I have my selenium tests written using SpecFlow(+SpecRun) and NUnit framework (v.3.8.1.0). I've configured Jenkins to run these tests. My Jenkins Windows Batch Command is as follows:
"C:\Program Files (x86)\NUnit.ConsoleRunner\3.7.0\tools\nunit3-console.exe"
C:\Projects\Selenium\ClassLibrary1\PortalTests\bin\Debug\PortalTests.dll
--test=TransactionTabTest;result="%WORKSPACE%\TestResults\TestR.xml";format=nunit3
When I trigger build test seems to start running as I'm getting as far as end of NUNIT3-CONSOLE [inputfiles] [options] with spinner indicating that test is running but it actually never ends and estimated remaining time is: N/A.
Now, when I run this script with windows cmd.exe:
"[PATH to Console.exe]\nunit3-console.exe" PortalTests.dll -- test=TransactionTabTest
this test pass successfully and so does in VS.
Now, I know this is very generic question but any clues will be much appreciated.
As you are using SpecFlow+Runner/Specrun, you can find the documentation how to configure it for the different build servers here: http://specflow.org/plus/documentation/SpecFlowPlus-and-Build-Servers/
How to generate Html reports from using Apache ant in Jmeter.
Please attach screenshots
There won't be too much screenshots, however I believe the answer will still be helpful
Install Apache Ant. Make sure that /bin folder of Ant installation is in your PATH
Go to "extras" folder of your JMeter installation in command propmt.
Type ant
If everything goes well you should see "BUILD SUCCESSFUL" message
Ant will generate 2 artifacts:
Test.jtl - XML file with results
Test.html - HTML wile with results which can be viewed in browser
If steps 1-5 are successful you can now replace /extras/Test.jmx with your own JMeter test, delete Test.jtl file and execute step 3 for your test script.
See following references:
/extras/build.xml Ant build file
JMeter Ant Task manual
Five Ways To Launch a JMeter Test without Using the JMeter GUI guide which includes launching JMeter via Ant.
JMeter produces the result in XML format. You need an XSLT file to convert this to a nice HTML.
This site has the detailed steps on running JMeter tests with ANT + creating HTML reports + Creating charts.
http://www.testautomationguru.com/jmeter-continuous-performance-testing-part1/
I'm using karma to run unit tests and generate coverage reports. That all works fine but I want to publish the lcov.info file to Code Climate from Travis CI. I've done it before and it works great, but the url from that test runner was static.
The issue is that karma creates a subfolder for each instance it runs such as test/coverage/PhantomJS 1.9.7 (Mac OS X)/lcov.info. Is there a clean way to get that url to feed into travis? I don't want to have to remember to update a hardcoded value every time PhantomJS gets updated. Does anyone know if the .travis.yml file supports something like glob patterns.
Thoughts?
A glob pattern will work and I'm using it successfully with a karma/travis/codeclimate set up.
Using
codeclimate < test/coverage/**/lcov.info
should work assuming have the CODECLIMATE_REPO_TOKEN variable set.
Also, options to change the subdirectory structure is being discussed on https://github.com/karma-runner/karma-coverage/pull/62.
I have been trying to get Bamboo working with codeception tests. I am using codeception to test my code in a symfony project.
After some research I found an article on how to setup Jenkins with codeception.
Once read I figured out that I should use Ant to run the codeception commands that run the tests.
The problem is I don't really know where to put everything. This article explains all the fields for a new Ant task but nothing seem to work.
Can someone please help me?
In case anyone else comes across this, this is how I have gotten codeception working in bamboo.
In Admin > Server Capabilities. Add a new executable for Codeception with the path /usr/local/bin/codecept.
In your job, create a new task of type Command. Set the executable as Codeception. The arguments should be run --xml (and any others you need).
Next, create another task with the type of JUnit Parser. Set the custom results directory to tests/_log/*.xml.
This works perfectly for me.
Additional: If you do not have admin rights to the server, set the command executable as PHP and add ./vendor/bin/codecept run --xml as the arguments.
I have started using Qunit to test my JS code. I am looking into JSCoverage to generate the coverage reports later. We have a CI server (Jenkins) which already do a few things with our PHP code and I was wondering if anyone can comment on how I can integrate the report from my Qunit and JSCoverage into Jenkins
Thanks
Sparsh
QUnit: use QUnit API to generate junit XML files. Here's a sample.
In Post-build Actions for your job you then check Publish JUnit test result report and specify your junit XML files (or their file pattern). Jenkins will then mark builds that have failed tests as unstable and produce a nice trend graph of successful/failing tests.
A few more details, for those actually attempting this:
Putting together QUnit and Jenkins
If you want to run QUnit and publish the results in Jenkins, you'll need to do the following:
Step 1: Getting QUnit to generate an XML file compatible with JUnit.
If you're using Apache Ant, this question explains how to get
QUnit to generate XML.
If not, you can use Grunt and
grunt-qunit-junit, together with grunt-contrib-qunit, to
run your .html tests.
And if you're not into either Ant or Grunt, here is
a script for PhantomJS to run your tests directly and produce
JUnit-style XML.
Step 2: Processing that XML file
This is the easy step - look in "Post-build Actions" for your job in Jenkins, and add the path to the XML file.