Has anyone written (or know about) a .net unit-test runner, preference to NUnit, that runs on iOS ? or do I have to roll my own ?
My goal is to execute the unit tests on the simulator or devices. So far I've seen quite a few posts / blogs on mocking monotouch.dll (but running the tests on a PC) and one answer stating the lack of such tool.
As promised here's my own solution for my problem :-) I hope it can help other people too!
EDIT
Sounds like you'd have to write a MonoDevelop add-in that serves up an API for remote NUnit runners to send data to. Not really a trivial task.
The original Touch.Unit was updated to include support for network logging (albeit not inside MonoDevelop).
EDIT #2 : A similar runner now exists for Mono for Android.
FINAL EDIT: Touch.Unit is now an integreal part of MonoTouch releases (starting with version 5.2) and does not have to be downloaded seperately.
I've wanted the same thing myself for quite some time. I think building one is the only option...
Can't imagine porting this GUI to iOS:
I think the only reasonable solution might be to show a simple UITableView on the device/simulator with test name and red/green checkmarks, and post the full NUnit results via http to server software somewhere.
Sounds like you'd have to write a MonoDevelop add-in that serves up an API for remote NUnit runners to send data to. Not really a trivial task.
Related
I have used Appium in Mobile automation with IOS and Android. I wanted to know about advantages that offers Calabash and if there are common point in generated scripts of both tools?
I have followed this link : Appium VS Calabash
I have hands on in mobile automation quite a long time. let me tell my point of view on both tools.
both also have advantage and disadvantage find below:
I have used calabash and appium both for automation. Appium has more advantages than calabash.
Advantages of Appium over Calabash:
1. You can write your code in multiple language like Java,Python, C#, php etc. However, Calabash restrict to use RUBY only.
Appium comes with many predefined methods such as OpenNotification(), StartActivity(), InstallApp etc. Calabash does not provide such predefined methods.
Appium set up is very easy as compared to Calabash.
In calabash, it installs the app on the device every time you run the test. However, In appium you have a feature where in you don't need to install the app every time. It works with previously installed app on the device.
Appium works with both native and hybrid app.
There are many advantages of appium. I hope this helps.
Try them both for yourself.
I found calabash easier to get started. I had a complete user flow test working from setup to working test in Calabash in half the time it took for Appium.
I don't like using xpath. It makes tests brittle and subject to breakage if something in the layout changes. Appium requires mainly the use of xpath. Calabash allows you to drive actions and interactions through the selection of elements by their text . I prefer this because that same text is normally in the designs and/or requirements.
Some comments in other posts are not accurate. Calabash does not require the install of the app every time unless you don't look beyond the surface. There is a simple means to enable/disable it. I have logic that runs either way based on passing an argument. Calabash has more than enough existing methods to interact with your app. It has a built in debug that allows you to tinker with element interaction and those same calls can be copied and reused in your test code. Using it with getgauge.io provides reporting akin to the built in Cucumber but is easier to run multi-threaded and easier to write your feature/spec files.
Be fair to both tools and yourself. Create a simple project in each and try for yourself. Don't make a decision solely based on someone else's opinion.
There is no common part in the automation scripts since Calabash uses query for interacting elements but Appium uses selenium find functions.
The best part of the Calabash is that you can use cross-platform frameworks for automating ios and android application with same feature file and most part of automation code. However the worst part is that Calabash is Ruby only.
Look at this post for more details about the differences of Calabash and Appium.
Since you have already read the differnces , Few things to note that Calabash can also be used with Xamarin which is also a good tool to test hybrid apps and you can run the same script for ios and android apps with some minor changes.Also it has been now integrated into visual studio,so microsoft support is inevitable and some users might feel comfort factor.Lambda expressions are also a plus.A major setback with the Calabash however that it only uses ruby.
Appium on the other hand can support various languages which is plus and you can use webdriver as well.However it struggles with hybrid apps but you can overcome that functionality by using google chrome mobile view to find coordinates to click.one thing for sure is that sause labs have many bugs to fix in Appium yet.
I am new to automation and trying to use Appium(Mac app) for writing test scripts on iOS app.
I have managed to load my app successfully on Appium and also used Appium Inspector.
There is a Record button which records my actions in Appium inspector.
However, my question is how do I save these recorded scripts for reuse?
Secondly, I also want to manually write test scripts using Java, but currently don't know where to write code using Appium Mac app.
I have read the starter guide on Appium website and seen some videos but cannot find any app usage related information.
Any help(links, tutorial video) will be appreciated.
P.S. I do not use Selenium.
My suggestion is to not use the record feature in Appium, it's not stable and hard to maintain.
You can write your java-automation-code on Eclipse/intellij(and all java IDE)
You have a lot of information on internet, you can start here:
Appium - Git - Sample - java
Try to make it run, and take it from there.
You always can ask at discuss.appium.io and see examples.
Good luck, hope it's helps a little.
You can use any programming language supported by Appium where you feel better.
Setup environment and create test scripts. you can learn from:
Appium
Jave Sample
You can write your script in the same way as you used to right in Selenium means you can choose any IDE (in my case I'm working on Eclipse Juno) and then just pass all the capabilities if you will use TestNG framework then run as TestNG.
I'm trying to use SpecFlow for a .net project. I'm new to SpecFlow. The Development team are using NUnit, so it would seem that SpecFlow would be a good option in conjunction with Cucumber. However, the Development team have come back say that SpecFlow cannot be used saying they do not have an API/Service that is available to use at the level required. Currently all of their automated tests are through the UI using Test Complete, I am keen to move to API level testing.
Can anyone explain to me why SpecFlow cannot be used, I'm sorry it's a newbie question but no one can answer it, I've asked everyone I can think of, surely the first steps would be to see if we can use SpecFlow with NUnit but perhaps not.
Can anyone give me a guide on my next steps, how can I be sure this isn't an option without righting it off without concern that it's just being blocked?
Thank you
SpecFlow has a unit test generator that generates unit test code for a variety of unit test frameworks. SpecFlow generates NUnit tests in its default configuration. The getting started page on specflow.org explains a quick way to get up and running with SpecFlow and NUnit, http://www.specflow.org/getting-started/.
If the UI is HTTP based, SpecFlow can be used with WebDriver or another browser automation framework to test the UI. This blog post provides an overview of how to get started with SpecFlow, NUnit and WebDriver, http://blogs.lessthandot.com/index.php/enterprisedev/application-lifecycle-management/using-specflow-to/
I am unclear on the API you want to test. If you could provide more information on the specific API and UI you are trying to test, I could possibly provide some code examples or references for you.
Is the API exposed through HTTP?
Is the UI a web, mobile, or desktop
application?
Have you tried to use SpecFlow at all?
SpecFlow doesn't run tests. It simply maps readable language to tests. If their test can be written as a nunit test, then SpecFlow is available to you to use. With no change, here is how it would look.
Scenario: Running 'testname'
Then I execute the test 'TestName'
You would map that to
[Then(#"I execute the test '(.*)'")]
public void ExecuteSpecificTest(string testname)
{
// Using reflection, execute the method listed
}
Obviously you would want to do better than that. You want a given, when, then so you clearly show the setup, action, and then compare expected verses actual result but it isn't necessary. Best practices however is another discussion.
To sum it up, code is code and SpecFlow simply maps to code. You can use WatiN, WebDriver, or anything else to hook into the UI or an API. SpecFlow doesn't care. It simple executes the methods without knowing what's inside.
I need to know if I can do Continuous Integration with XCode server. In example: set run the tests every night or when someone commit changes, and more..
I am trying to decide one iOS Ui Automation tool to integrate with my Xcode server
Thanks
There are a few problems here:
UIAutomation has no built in support in Xcode Server. I've filed bugs, I've chased down people at WWDC. Most I've ever gotten on this problem is basically "shrug." I'm not sure UIAutomation is a priority for Apple right now. So you're not going to get any official support.
As was noted, you might be able to use a trigger. The trigger won't be able to add anything to the Xcode Server report, besides possibly the error logging. But you're not going to get anything added to the nice report table.
Running on actual devices has traditionally been a problem (if you care about that.) The loading the app part has been a problem for us, but Xcode Server might be able to preload the app for you. In addition, it seems like this might have changed in the iOS 8 SDK.
There is just a lot of uncertainty around this sort of workflow. I'm hoping Apple eventually makes an announcement or adds a new tool, but the best answer I've gotten is if you want to go down this path, use UI Unit Tests. That's a shame because it requires knowledge of Obj-C or Swift, and means interacting with the app at an API level instead of an abstract level, but if you're looking for the direction Apple wants to see people go, that's it.
Edit 7/4/2015: As of WWDC 2015, there is a new UI Testing component as part of Xcode 7 that, in my experience, seems totally supported, and is promising Xcode Server support. I would very strongly recommend using that, and not using the Instruments UIAutomation tool.
With Xcode6 right around the corner they are adding some features to XCode Server specifically it looks like "Triggers" will be helpful for running iOS UIAutomation tools. Since you can run UI automation scripts from cmd line it should be possible to utilize triggers to run your scripts post builds. This along side the logic for when a bot should run will let you decide if it should be nightly or on every commit.
https://developer.apple.com/library/prerelease/ios/documentation/DeveloperTools/Conceptual/WhatsNewXcode/Articles/xcode_6_0.html#//apple_ref/doc/uid/TP40014509-SW1
I wrote a framework around UIAutomation called Illuminator to handle tasks like nightly test runs, pull request tests, and other automated conveniences.
It provides a flexible and extensible command line that can execute any particular subset of tests that you'd like, and produces reports in JUnit format (used by Jenkins).
I would like to perform the following iPad/iPhone testing scenario automatically:
Tap Edit box A
Type text "abcd"
Verify button B is high-lightened
I understand UIAutomation 4.0 allow you to write a simple JavaScript to perform the above steps. However, UIAutomation does not have test infrastructure ready. For example it lacks testing macros to show if any tests failed and does not have a clear way to run setup and shutdown for each test cases.
That is why I look back to XCode unit testing. Logic tests won't work for me. How about Application tests?
Basically, I am looking for something that can do GUI testing and at the same time has test infrastructure. It is even better if it can be integrated to continuous build environment.
Check out Zucchini. It's just come out and I saw a demo at a recent YOW! conference. It's basically a BDD testing framework that uses coffeescript for scripting and runs against an actual device. It's also fully runnable from CI servers which makes it perfect for agile teams.
I haven't run it myself yet, but it seems to exactly what I'm looking for and No I don't work for PlayUp :-)