I am trying to write an integration test for my iOS app & our API server. Both are in the same repo, so this test would prevent merging any changes that would break their ability to communicate.
The server team has created a bash script that I need to call to start a local copy of the API server. That script will run until it is terminated. How can I run that script when I run my iOS integration test? I can't put it as part of the build phase as the build will wait for it to terminate before it lets the build finish. I can't use Process directly in my tests to launch the script because that's only available on the Mac, not to iOS targets. Refactoring all my networking & model code to it's own framework that supports iOS & Mac targets would work, but that's a sizable refactor to do. Are there any other ways to run commands along with tests? I'm just using XCTest for all my tests.
As mentioned in the comments I found a way to run scripts on test start & finish. You have to do it for the entire test target not the individual test cases.
You have to edit the scheme, expand the tests and add pre-actions & post-actions.
Related
we are currently using Wallaby.js for javascript unit testing. Works fine and is great. But within our development pipeline we of course want to run the same tests on the build server - in our case a tfs.
Is it possible to use wallaby on a tfs build server? Anf if yes how?
If not, what is the way to go to run the wallaby configured unit tests on the build server?
As we used the karma test runner earlier, I tried to execute the new test configuration with it but then I get
Can't find variable: wallaby
as in our main/ starting test file it is written
wallaby.delayStart();
require.config({
baseUrl: 'app',
(Originally from a karma/ requirejs configuration)
How to get around this?
Has anyone experience in this scenario?
Wallaby.js main idea is to integrate with editors, run tests for the code that you change and display the results in the editor. You can't use Wallaby.js in a CI build.
You may consider invoking other test runners, or use grunt/gulp task instead for javascript unit testing.
In TFS 2012 and later (might work in 2010 but not sure) you can extend the testing capabilities of the build system.
Check out these posts -
http://www.aspnetperformance.com/post/Unit-testing-JavaScript-as-part-of-TFS-Build.aspx
https://blogs.msdn.microsoft.com/visualstudioalm/2012/07/09/javascript-unit-tests-on-team-foundation-service-with-chutzpah/
I want to use Jenkins CI for integration testing with Play framework. My scenario is as following:
I have 2 projects, Project A and B.
Project A depends on Project B. The dependency is as such that to run tests on Project A, I need to start Project B first.
I already have unit tests in Project A but I need to test the integration of Project A and B.
I am using SBT plugin to execute the SBT and the Project A and Project B are working fine separately.
I could not figure out a proper way to do it. The issue I am facing is that I need to run Project B as a pre-build step but the Project B must be kept in running state but Project B is ended as soon as the build step executes run action of sbt and finishes the build which I don't want.
The command I execute to run Project B is clean compile run which executes as an action to SBT launcher.
I tried SBT stage and then run the jar but that is also causing the issue that the jar halts the control of the build and Project A doesn't get a chance to start.
I also checked Spawning a process in Jenkins but I couldn't make it work too. I am using Ubuntu and I tried using nohup instead of daemonize as described in the link by adding it as Execute shell script build step and it starts the Project B server process and kills it after some time. I also don't think that it might be the only way to do what I want to do.
May be I am using Jenkins wrong or may be I need to look in another direction so any help on this is much appreciated.
I ran into a similar problem where I needed to free the console for running other stuff. I did something similar (i.e. creating a script with the sbt commands), then running the script with a nohup like so:
nohup ./myScript.sh &
and the Play! app runs just fine in the background.
Remember to use different ports in your case, since you're running two Play! apps.
I finished writing Atuomation scripts(using selenium webdriver) for IOS Application.
Now how can i release it to client (Or) how to release a build and how to run scripts once again
Selenium is code. You should follow standards for releasing/branching of code and then setup your client with access to the code. Since it is code I would recommend creating an instruction doc on how to get the latest and how to execute the code through whatever execution mechanism you guys are using...i.e. unit test framework.
I have a bunch of jasmine tests that I would like to run on a jenkins CI server.
At the moment, we use an html page that runs the specs, that a developper can open in a browser on its own machine.
The transition to CI would be easy if I had access to some kind of server side test runner (like karma), however for some undisclosable reasons, I can not run nodejs on our CI server.
So in the spirit of creativity-under-constraints, what could I use to automate jasmine tests without node ? (But anything that can run with maven and a jdk is probably fine...)
You can make your test automatically spawn a browser with the page that runs your unit test. The tricky part tough is to get the result back to the main test runner. The solution that I have found for that is to use a custom jasmine reporter (you just need to implement the same function has to other reporter) and when a spec has finished to run you do an AJAX call to write that result in a file. The main runner just needs to wait until something is written in that file to see the results. Once the test are finish, just don't forget to kill the browser, otherwise your CI server will be flooded by window.
We use TFS 2010 and Automated builds.
We also make use of MSTests.
I would like some concrete information about the build server's test execution method.
Will the test engine (on build server) run the unit tests sequentially or in parallel?
By default it will run them sequentially. You can customize the build workflow by adding a Parallel activity and running different sets of tests in each. Or if you want to parallelize the test run across multiple build machines you can have the build use multiple RunOnAgent activities to do so (http://blogs.msdn.com/b/jimlamb/archive/2010/09/14/parallelized-builds-with-tfs2010.aspx).
Note: If you execute the tests across multiple test runs you will end up with multiple test reports (.trx files) that will not be merged together without further customization of the build.
#Dylan Smiths answer is correct, but does not cover option # 3.
Executing Unit Tests in parallel on a multi-CPU/core machine
DANGER WILL ROBINSON: This is only applicable to VS2010 and mstest.exe. VS2012 has a new test runner that does not support parallel test execution Visual Studio UserVoice Run unit tests in parallel The VS2012 test system can use the legacy testrunner, and you can make it work if you specify a .testsettings file using the MSTest/SettingsFile element. Configuring Unit Tests by using a .runsettings File
How to: Enable parallel test execution
Ensure you have a multi-core/CPU machine
Ensure you are running only unit tests
Ensure your tests are thread-safe
Ensure you do not have any data adapters on
Ensure you are running locally (cannot use TestController/TestAgent)
Modify your test settings file.
Right-click the test setting file and select "Open With" -> Open as Xml
Set the parallelTestCount attribute on the Execution element
Options are:
blank = 1 CPU/Core - this is the default
0 = Auto configure: We will use as many tests as we can based on your CPU and core count
n = The number of tests to run in parallel
Save your settings file