I would like to perform the following iPad/iPhone testing scenario automatically:
Tap Edit box A
Type text "abcd"
Verify button B is high-lightened
I understand UIAutomation 4.0 allow you to write a simple JavaScript to perform the above steps. However, UIAutomation does not have test infrastructure ready. For example it lacks testing macros to show if any tests failed and does not have a clear way to run setup and shutdown for each test cases.
That is why I look back to XCode unit testing. Logic tests won't work for me. How about Application tests?
Basically, I am looking for something that can do GUI testing and at the same time has test infrastructure. It is even better if it can be integrated to continuous build environment.
Check out Zucchini. It's just come out and I saw a demo at a recent YOW! conference. It's basically a BDD testing framework that uses coffeescript for scripting and runs against an actual device. It's also fully runnable from CI servers which makes it perfect for agile teams.
I haven't run it myself yet, but it seems to exactly what I'm looking for and No I don't work for PlayUp :-)
Related
There are three basic layers of testing:
Unit test — making sure each function/unit of code works as expected
Functional test — making sure units interact with each other as expected
Integration test — making sure our app integrate with other app/api/services as expected
I can handle cases 1. and 2. using Xcode's iOS Unit Testing Bundle and iOS UI Testing Bundle I don't know how to write tests from point 3.
I would like to write to tests to check if me app correctly integrate with backend, BLE device, etc.
Note that I have already written unit tests which I run using CI.
Integration test should not be run in CI because it depends on external environment. I would like to run its only manually.
How to handle it in Xcode? Second Unit Testing Target? Any advice? How are you dealing with it in yours systems?
Depending on the style of integration test that you are after you may want to use dedicated a unit or UI tests target.
From your question I'm guessing the style you want to have leans towards black box testing with other real systems. If that's the case a dedicated UI tests target would do the job.
With UI tests you can do stuff like opening other apps on the device and test how your app interacts with them. For example, this post shows how to write a UI test that uses the Messages app to test Universal Links.
I'd reach for using a dedicated unit tests target only if you need to control part of the code in your app, for example stubbing network requests. But that doesn't really fit with the definition of integration test you have given.
How I do I automate Xamarin.iOS unit test project.
For Android, I found this link which worked fine.
https://developer.xamarin.com/guides/android/troubleshooting/questions/automate-android-nunit-test/
Is there any references like this for Xamarin.iOS too?
At the time of writing this, I don't believe what you'd like to do is possible. If I take your meaning, you'd probably like to say "Run All" with some test runner (presumably in XS or VS) and then get the results immediately, but that's not how it works with Xamarin.iOS. I'm sure you've already glanced at Xamarin's iOS testing quick-start, but if not here that is.
You have to set up a Unit Test app that uses the Touch.Unit framework, fire up the test app, and "touch" the tests you'd like to run. My experience doing this has not been so great. The runner itself seems buggy and you're limited with in what other tools you can use (e.g., mocking frameworks won't work, assertions made with Shouldly won't register). I guess it's better than nothing, though!
I'm trying to use SpecFlow for a .net project. I'm new to SpecFlow. The Development team are using NUnit, so it would seem that SpecFlow would be a good option in conjunction with Cucumber. However, the Development team have come back say that SpecFlow cannot be used saying they do not have an API/Service that is available to use at the level required. Currently all of their automated tests are through the UI using Test Complete, I am keen to move to API level testing.
Can anyone explain to me why SpecFlow cannot be used, I'm sorry it's a newbie question but no one can answer it, I've asked everyone I can think of, surely the first steps would be to see if we can use SpecFlow with NUnit but perhaps not.
Can anyone give me a guide on my next steps, how can I be sure this isn't an option without righting it off without concern that it's just being blocked?
Thank you
SpecFlow has a unit test generator that generates unit test code for a variety of unit test frameworks. SpecFlow generates NUnit tests in its default configuration. The getting started page on specflow.org explains a quick way to get up and running with SpecFlow and NUnit, http://www.specflow.org/getting-started/.
If the UI is HTTP based, SpecFlow can be used with WebDriver or another browser automation framework to test the UI. This blog post provides an overview of how to get started with SpecFlow, NUnit and WebDriver, http://blogs.lessthandot.com/index.php/enterprisedev/application-lifecycle-management/using-specflow-to/
I am unclear on the API you want to test. If you could provide more information on the specific API and UI you are trying to test, I could possibly provide some code examples or references for you.
Is the API exposed through HTTP?
Is the UI a web, mobile, or desktop
application?
Have you tried to use SpecFlow at all?
SpecFlow doesn't run tests. It simply maps readable language to tests. If their test can be written as a nunit test, then SpecFlow is available to you to use. With no change, here is how it would look.
Scenario: Running 'testname'
Then I execute the test 'TestName'
You would map that to
[Then(#"I execute the test '(.*)'")]
public void ExecuteSpecificTest(string testname)
{
// Using reflection, execute the method listed
}
Obviously you would want to do better than that. You want a given, when, then so you clearly show the setup, action, and then compare expected verses actual result but it isn't necessary. Best practices however is another discussion.
To sum it up, code is code and SpecFlow simply maps to code. You can use WatiN, WebDriver, or anything else to hook into the UI or an API. SpecFlow doesn't care. It simple executes the methods without knowing what's inside.
I am using Calabash to perform UI tests in my iOS app.
Calabash is chosen because tests could be written using Gherkin-style and I can use RubyMine.
Good things: any time during writing gherkin scenarios I can easily get a list of already implemented functions (like: When user successfully logged in) and also RubyMine will create functions for every new scenario. These are actually huge benefits.
Bad things: In addition to UI tests in Calabash I also have native unit tests which I run using bots, but I didn't figure out how run Calabash tests using bots on OS X Server.
So in ideal world I would like to have something what KIF does: bots could run UI tests and give nice OS X Server web page results, but at the same time I'd like to have all benefits of using Calabash + RubyMine
Or maybe there is a way to run calabash cucumber tests on OS X Server using bots? And I am not really comfortable using OS X Server for native unit tests and Jenkins for cucumber tests. I just want one tool do whole thing.
Any suggestions?
Create a new bot and set its Schmeme to your Calabash scheme, e.g., MyXcodeScheme-cal. Then for example add an iPhone in the testing tab as target.
But unfortunately a successful integration does not mean that the Calabash tests has passed - only that the test were executed! So the Calabash test results itself has to be investigated manually afterwards :S
:)
G'day guys,
I've started a new role at a company and am working on a rails application that's used internally for particular management applications. At any rate, there are no unit tests associated with this system, and as I am building the system out I've realised there is a serious need to build unit tests so that I can have regression testing as I continue to add to this codebase.
It's a rails 1.2.7 app that I cannot migrate for reasons of compatibility, but I will be beginning to port components over it to 3.0 in the background.
There is currently only the built in tests that rails makes, and I would love some insights into how to get started with building small tests into the system and particularly at what aspects would be good to start?
This system builds a lot of individual config files, so I'm assuming a lot of the test generation would begin with testing those config files for individual aspects/elements and going from there.
I've had a look at cucumber and rspec, but just wanted to find something straightforward and easy to use that I could slowly build upon within the current system and build up a testing base over time.
Any insights, links or others would be really really appreciated.
Cheers!
A lot has changed since Rails 1.2.7, definitely one of the first things would be to right unit tests, but instead of going an all out unit test attack.. Attack the problem component,
Write tests
Upgrade the component
Run Tests ( mostly they'll fail / possibilities of errors )
Make them pass
Refactor ( May be the new features at your disposal )
I usually do the above steps with writing the integration tests, and functional tests and unit tests for the components in the integration tests.
All the best, this ain't going to be a simple task :).