I've seen various articles describing how to run Frank automation on Jenkins/Hudson using headless app running method. But is there a possibility to run the Frank Automation locally and generate the coverage files (gcno, gcda?)
How to make the test App quit gracefully with Frank and get the gcno and gcda ?
Related
I am trying to write an integration test for my iOS app & our API server. Both are in the same repo, so this test would prevent merging any changes that would break their ability to communicate.
The server team has created a bash script that I need to call to start a local copy of the API server. That script will run until it is terminated. How can I run that script when I run my iOS integration test? I can't put it as part of the build phase as the build will wait for it to terminate before it lets the build finish. I can't use Process directly in my tests to launch the script because that's only available on the Mac, not to iOS targets. Refactoring all my networking & model code to it's own framework that supports iOS & Mac targets would work, but that's a sizable refactor to do. Are there any other ways to run commands along with tests? I'm just using XCTest for all my tests.
As mentioned in the comments I found a way to run scripts on test start & finish. You have to do it for the entire test target not the individual test cases.
You have to edit the scheme, expand the tests and add pre-actions & post-actions.
Good morning,
I've got a problem when using GCOV within my working environment.
Gcov is working very well when I run some tests cases (up to 1000) but no gcda are generated when running more tests.
This is how I use it.
I compile my code with gcov flags correctly set
I boot a test server containing the gcov libs, and the variables GCOV_PREFIX and GCOV__PREFIX_STRIP
I launch my regression on this server
Once finished I stop the server and now all the gcda files are generated
use lcov and genhtml to generate the test coverage and the report.
This works very well when I've got few tests to lauch (up to 1000 cases I guess), but if I run more tests, I don't get any gcda files anymore...
I could not find any documentation on this part, is there a buffer somewhere, where all the gcdas files are stored waiting for the server to be released?
Is it possible to parameter this setting?
Is there any documentation on this subject somewhere?
Thanks a lot for your help.
Regards,
Thomas
I have my selenium tests written using SpecFlow(+SpecRun) and NUnit framework (v.3.8.1.0). I've configured Jenkins to run these tests. My Jenkins Windows Batch Command is as follows:
"C:\Program Files (x86)\NUnit.ConsoleRunner\3.7.0\tools\nunit3-console.exe"
C:\Projects\Selenium\ClassLibrary1\PortalTests\bin\Debug\PortalTests.dll
--test=TransactionTabTest;result="%WORKSPACE%\TestResults\TestR.xml";format=nunit3
When I trigger build test seems to start running as I'm getting as far as end of NUNIT3-CONSOLE [inputfiles] [options] with spinner indicating that test is running but it actually never ends and estimated remaining time is: N/A.
Now, when I run this script with windows cmd.exe:
"[PATH to Console.exe]\nunit3-console.exe" PortalTests.dll -- test=TransactionTabTest
this test pass successfully and so does in VS.
Now, I know this is very generic question but any clues will be much appreciated.
As you are using SpecFlow+Runner/Specrun, you can find the documentation how to configure it for the different build servers here: http://specflow.org/plus/documentation/SpecFlowPlus-and-Build-Servers/
I managed to install Google Test on Jenkins.
I use cmake to build the test executable and everything works fine.
The stupid question I have now is:
How do I automatically let Jenkins run google test?
Do I have to write a shell script for this or is there a better way?
I know that one could run it in ant but since I use cmake I doubt that this is the right way to go.
There's no builtin or pluggable "Googletest automation" for Jenkins.
You've built the test executable with CMake in a build step.
Execute the test executable in a subsequent build step
with xml test reporting enabled
and configure the build to archive or otherwise publish the test reports.
I'm using psake, msbuild and nUnit to automate my build and testing of an MVC web app, which will be carried out (kindly), by Jenkins, once I have it working.
My build steps work fine, creating two DLL's in the build\bin dir:
MyWebApplication.dll
MyWebApplication.Tests.dll
I'm using nunit-console.exe to run the automated tests as part of the psake build script, pointing it at the newly built MyWebApplication.Test.dll. However, the tests fail due to is saying it could not load file or assembly MyWebApplication.dll, despite it being in the same directory as the test dll file.
How do I go about executing tests using nunit in this scenario?
It's most likely looking for the application .dll in the workspace root, which is the current directory by default in Jenkins. Try changing the current directory to %WORKSPACE%\build\bin before launching the test.
I found the answer to my own question.
I had to compile a debug version of my projects as part of the build script, then run the Nunit console exe against my csproj file for the test project. With this, it executes the tests properly.