There are three basic layers of testing:
Unit test — making sure each function/unit of code works as expected
Functional test — making sure units interact with each other as expected
Integration test — making sure our app integrate with other app/api/services as expected
I can handle cases 1. and 2. using Xcode's iOS Unit Testing Bundle and iOS UI Testing Bundle I don't know how to write tests from point 3.
I would like to write to tests to check if me app correctly integrate with backend, BLE device, etc.
Note that I have already written unit tests which I run using CI.
Integration test should not be run in CI because it depends on external environment. I would like to run its only manually.
How to handle it in Xcode? Second Unit Testing Target? Any advice? How are you dealing with it in yours systems?
Depending on the style of integration test that you are after you may want to use dedicated a unit or UI tests target.
From your question I'm guessing the style you want to have leans towards black box testing with other real systems. If that's the case a dedicated UI tests target would do the job.
With UI tests you can do stuff like opening other apps on the device and test how your app interacts with them. For example, this post shows how to write a UI test that uses the Messages app to test Universal Links.
I'd reach for using a dedicated unit tests target only if you need to control part of the code in your app, for example stubbing network requests. But that doesn't really fit with the definition of integration test you have given.
Related
How I do I automate Xamarin.iOS unit test project.
For Android, I found this link which worked fine.
https://developer.xamarin.com/guides/android/troubleshooting/questions/automate-android-nunit-test/
Is there any references like this for Xamarin.iOS too?
At the time of writing this, I don't believe what you'd like to do is possible. If I take your meaning, you'd probably like to say "Run All" with some test runner (presumably in XS or VS) and then get the results immediately, but that's not how it works with Xamarin.iOS. I'm sure you've already glanced at Xamarin's iOS testing quick-start, but if not here that is.
You have to set up a Unit Test app that uses the Touch.Unit framework, fire up the test app, and "touch" the tests you'd like to run. My experience doing this has not been so great. The runner itself seems buggy and you're limited with in what other tools you can use (e.g., mocking frameworks won't work, assertions made with Shouldly won't register). I guess it's better than nothing, though!
Any frameworks available for writing test cases to Support iOS applications developed using Appcelerator Titanium?
There are quite a few frameworks for Unit Testing:
MockTi
Titanium-Jasmine
Ti-Mocha
Ti02
TiUnit
TiShadow
But the most of those frameworks listed above are either discontinued or don’t work anymore. And the frameworks that are still working mostly run in the Titanium container/runtime. This means the Titanium project needs to be build and run first which makes the execution process of your tests very slow. Besides that, most of them don’t provide mocking the Titanium namespace (e.g. manipulating/mocking Ti.Network).
We’re using the TiUnit toolset for our Unit Testing in combination with Istanbul (test/code coverage). TiUnit covers our needs within Unit Testing:
Fast and execution outside of the Titanium container/runtime
Mocking all dependencies (e.g. required CommonJS modules)
Generating a mock for all functions, constants and properties in the
Ti namespace (Ti)
Callback, L macro and $ testing
More information can be found on the TiUnit github page
We recommend ti-mocha (https://github.com/tonylukasavage/ti-mocha) which suited, test cases, supports asserts, skips, advanced validations and many more features.
I was using KIF Framework until now for iOS UI Automation. KIF (also Unit Test) targets runs the Unit/UI testing code in the same process as your application is running and we can easily Mock different objects/classes used by the app to show mock data.
When switching to Xcode 7 based new UI Automation, I found that UI Unit Test target runs in separate process and it launches the application in separate process. And thus it's not possible to access App classes/objects through unit tests and mock them.
Did anyone faced the same issue, and know about any workaround?
Please let me know if any more details are needed.
Answer: Please see the detailed answer below. Also here is link to reply from Apple devs: https://forums.developer.apple.com/thread/23552
Because you are running in a different process, there isn't really a way to inject mocks into your application via your tests. Having said that, you do get to control your App's process, and can thus have some effects on it.
I've gone back and forth on how I feel about this (I'm currently undecided), but you could add some code to your app that reads an environment variable (via NSProcessInfo) that makes the app behave differently (i.e., change out what your dependency injection is injecting into your class so it uses mocks).
The downside to this is the mock code isn't contained strictly in the test bundle, and you end up with that code in your app (unless you use #if statements to isolate it). Obviously it's also important to be prudent with your code branching, so you don't invalidate your testing.
You could setup an environmental variable prior to launching the app in your UI test:
let app = XCUIApplication()
app.launchEnvironment = ["UITestUseMocks" : "true"]
app.launch()
Then in your application you can check for that, possibly in a #if statement and alter your behavior:
#if TEST_TARGET
if let useMocks = NSProcessInfo().environment["UITestUseMocks"] where useMocks == "true" {
// Alter services used in dependency injection or some other testing behavior
}
#endif
I've been considering this approach for implementing a mock service layer that just replays some canned server responses so I can make my UI tests not depend on server responses (other things test the server, after all).
I am using Calabash to perform UI tests in my iOS app.
Calabash is chosen because tests could be written using Gherkin-style and I can use RubyMine.
Good things: any time during writing gherkin scenarios I can easily get a list of already implemented functions (like: When user successfully logged in) and also RubyMine will create functions for every new scenario. These are actually huge benefits.
Bad things: In addition to UI tests in Calabash I also have native unit tests which I run using bots, but I didn't figure out how run Calabash tests using bots on OS X Server.
So in ideal world I would like to have something what KIF does: bots could run UI tests and give nice OS X Server web page results, but at the same time I'd like to have all benefits of using Calabash + RubyMine
Or maybe there is a way to run calabash cucumber tests on OS X Server using bots? And I am not really comfortable using OS X Server for native unit tests and Jenkins for cucumber tests. I just want one tool do whole thing.
Any suggestions?
Create a new bot and set its Schmeme to your Calabash scheme, e.g., MyXcodeScheme-cal. Then for example add an iPhone in the testing tab as target.
But unfortunately a successful integration does not mean that the Calabash tests has passed - only that the test were executed! So the Calabash test results itself has to be investigated manually afterwards :S
:)
I would like to perform the following iPad/iPhone testing scenario automatically:
Tap Edit box A
Type text "abcd"
Verify button B is high-lightened
I understand UIAutomation 4.0 allow you to write a simple JavaScript to perform the above steps. However, UIAutomation does not have test infrastructure ready. For example it lacks testing macros to show if any tests failed and does not have a clear way to run setup and shutdown for each test cases.
That is why I look back to XCode unit testing. Logic tests won't work for me. How about Application tests?
Basically, I am looking for something that can do GUI testing and at the same time has test infrastructure. It is even better if it can be integrated to continuous build environment.
Check out Zucchini. It's just come out and I saw a demo at a recent YOW! conference. It's basically a BDD testing framework that uses coffeescript for scripting and runs against an actual device. It's also fully runnable from CI servers which makes it perfect for agile teams.
I haven't run it myself yet, but it seems to exactly what I'm looking for and No I don't work for PlayUp :-)