I am a tester writing cucumber test scripts using a Calabash/Ruby solution.
Here is my setup. I have two iOS devices plugged into a mac-mini. I have Xcode running and connected to a physical device. I also have an Oracle Virtualbox VM running on it that has another copy of Xcode running and that VM has another physical iOS device connected to it.
My dilemma is this. What I need is to write one test scenario that can test communication like calling and text between the two...so from within that test scenario I need to find a way to switch between the two devices.
I have been able to successfully write cucumber tests for Android doing this by adding special tags in the app_installation_hooks file. I referenced this link.
I have an idea it involves redefining the BUNDLE_ID, DEVICE_TARGET, and DEVICE_ENDPOINT every time I switch to a device, but I'm not sure how to write that if that is the solution.
I'm currently executing just one of my tests by calling 'bundle exec cucumber' which I can do on both machines (phsycial and virtual) individually.
Related
I want to open two chrome browser instances parallel in real android device using appium. i have started two appium servers with two ports, but when i tried to start second session, first one is killing. Please help me in doing this.
If you are trying to start port by using terminal then i had issues too. Open two appium windows and start server on two different ports. Make sure you change the bootstrap port for both.
I have an iOS project that comes from an older Xcode (4.6.3) and that did not include a test target by default, as projects in latest Xcode versions seem to include. I now have Xcode 5.1.1. on a Mac that is running OS X 10.8.5. I've reading this Apple's document regarding unit testing in Xcode, but it looks like you need OS X Mavericks, at least for installing and configuring OS X Server, don't you?
I'd appreciate some guidelines to follow to add a test target to my project and how could I integrate a testing process in my workflow if I do not have OS X Maveriks.
Thanks in advance
EDIT: Is there anyway of run tests in normal running of the app? I mean, I need to test some cases that involve the update of data at certain hours, and the update of some parameters and the GUI after that data update... how could I do this? Thanks
You need Mavricks if you wish to use the Xcode service, which is Apple's continuous integration server. You do not need Mavricks to run unit tests, however you will miss out on the pretty in-xcode reports.
Without Mavericks, you can run unit tests manually by pressing CMD+U or going to Product->Test.
You can also setup your own continuous integration service and run the unit tests with a command line script. You'll have to pick a CI server, something like Jenkins or many alternatives, and look up a how-to for that system.
Do you redirect NSLog/stderr anywhere?
I encountered this problem too with no red marks and failing test of '1 == 1'.
My app was catching NSLog calls to a file and not letting the system get them. Once I disabled the redirecting of logs freopen(filepath, "a", stderr) test started passing and green checks and red crosses appeared.
I did this for an external accessory device that hooked up through the lightning port and couldn't send the logs to the mac. You mentioned how you want to run the test during the normal running of you app, so I figured you may have saved the logs so you could read them away from you mac.
I am using Calabash to perform UI tests in my iOS app.
Calabash is chosen because tests could be written using Gherkin-style and I can use RubyMine.
Good things: any time during writing gherkin scenarios I can easily get a list of already implemented functions (like: When user successfully logged in) and also RubyMine will create functions for every new scenario. These are actually huge benefits.
Bad things: In addition to UI tests in Calabash I also have native unit tests which I run using bots, but I didn't figure out how run Calabash tests using bots on OS X Server.
So in ideal world I would like to have something what KIF does: bots could run UI tests and give nice OS X Server web page results, but at the same time I'd like to have all benefits of using Calabash + RubyMine
Or maybe there is a way to run calabash cucumber tests on OS X Server using bots? And I am not really comfortable using OS X Server for native unit tests and Jenkins for cucumber tests. I just want one tool do whole thing.
Any suggestions?
Create a new bot and set its Schmeme to your Calabash scheme, e.g., MyXcodeScheme-cal. Then for example add an iPhone in the testing tab as target.
But unfortunately a successful integration does not mean that the Calabash tests has passed - only that the test were executed! So the Calabash test results itself has to be investigated manually afterwards :S
:)
For some of my iOS application projects, I would like my CI server to be able to report the following properties:
startup time
frame rate
both as a graph over time, and with "low water marks" so the build fails if the measured results aren't within certain criteria. I've already found some of the things I need.
The CI server will be Jenkins.
I can use Transporter Chief to get the built app onto an iPad.
To measure the startup time I could find the duration between launching main() and leaving application:didFinishLaunchingWithOptions:.
To measure frame rate I can put a CADisplayLink into the app and sample its duration property.
If those tests output JMeter XML, then Jenkins can display the output via the Performance plugin.
What I haven't worked out is, how should I embed those tests into my app and launch it on the iPad? As described above I can deploy the app to the iPad, but then I don't know how I would launch it to gather the results of the tests. My unit tests are running on the simulator - I don't want to run the performance tests there obviously :-).
I imagine that there's a solution involving jailbreaking the iPad and controlling the app over SSH, I'd prefer not to go down that route if it's possible. If you have done that and can explain how it works, I'd still like to hear about it.
I'm also using fruitstrap to install apps on the device in CI. In terms of booting the app, I know of two ways:
Use fruitstrap with the debugger attached
I know teams that have done this to run KIF integration tests on devices in CI. I have played around with fruitstrap to get it booting apps on the device, but haven't myself taken the extra step of automating the whole thing
shameless plug for my post on fruitstrap: http://www.stewgleadow.com/blog/2011/11/05/installing-ios-apps-on-the-device-from-the-command-line/
Use the instruments command line tool with UIAutomation
I know the instruments tool can boot apps on the device automatically in CI (I wish it also installed them, but we have fruitstrap for that until Apple fixes it). So you could write a really simple little UIAutomation test that gave your app enough time to do its performance analysis.
Jonathan Penn has a nice little demo project for UIAutomation and build script that could be integrated with an 'install' step using fruistrap to try it out on the device
In both cases, I uses a little wrapper around libusb to give me the device ID of attached devices, so the more devices I plug into my CI machine, the more devices it runs tests on, https://github.com/sgleadow/iphone_detect
Can you launch the app on the device using lldb?
If so, it may also be able to capture the log output.
Is there any tool in blackberry sdk like adb in android which can be used to install or run apps from command prompt on simulator.
Once you've packaged a .cod file to deploy, you basically have two options, which Howard and seand mentioned:
Using javaloader: javaloader -u load <path-to-cod>.cod
Using fledge and fledgecontroller
I'm guessing based on your question that this is part of an automated deployment scheme? I'd recommend using javaloader when deploying to real devices, and using fledge/fledgecontroller for deploying to the simulator.
javaloader will require that the simulator is simulating a usb connection, and a lot of people seem to have had trouble getting it to work correctly - when we tried there was about a 50% chance for any given run that it wouldn't connect or deploy correctly. But it works a bit better when trying to connect to a real device, at least in my experience. And if your project and requirements are fairly simple, you might be able to get away with using it on a simulator without too many problems.
Fledge/fledgecontroller have more commands that are useful for doing automated deployment and testing, which is primarily why I recommend them. They are a bit more complicated to use, however. To deploy to the simulator with fledge and fledgecontroller, you'd want to do the following:
Start fledge (located in the simulator folder) with fledge.exe /handheld=<handheld-id> /session=<session-name> /app=<simulator-folder-path>/Jvm.dll /automate
Use fledgecontroller (also located in the simulator folder) to load the cod with fledgecontroller.exe /session=<session-name> /execute=LoadCod("<path-to-cod>.cod")
<handheld-id> is the id of the device you want to test on - e.g. "8300" would run the 8300 simulator for BB 4.5
<session-name> can be anything, but should match between the call to fledge and the call to fledgecontroller
Note that it will take a little bit of time (anywhere from 15 seconds to 5 minutes, depending on which simulator you are using) for the simulator to boot after calling fledge. You'll need to wait for it to finish before calling fledgecontroller.
Fledge also has a bunch more commands that you can use to configure the session, for example by setting up an sd card or language options. If you run fledge.exe /help, it will describe the different parameters; and fledge.exe /controller-help will list the different commands you can pass through fledgecontroller.
In summary: if you are using real devices, or possibly if you are doing very simple deployments on a simualtor, you can probably get away with using javaloader. If you want more control over the simulator, you should use fledgecontroller.
Most probably the JavaLoader is the tool you are looking for.
Simulators are launched using the 'fledge.exe' CLI tool. Look at the built in scripts for launching simulators to see the myriad of options.