I want to test my iOS app along with a memory profiler. I hope to use XCode inbuilt memory profiler with Calabash testing tool. But I can't run both at once. What I want is to integrate the both and analyze memory usage while testing the app. If calabash can't do this I want to know atleast XCode Automation tool can integrate with the XCode memory profiler.
can someone help me out
I couldn't find a way to use Calabash tool and Allocations tool together. But I found a way to use Allocations tool and Automation tool together. In XCode profiler first choose Automation tool and then click library tool in the tool bar. After that add the tools you need(In this case Allocations tool)
This is possible, but the results are pretty hard to understand. I use this to track down leaks in the Calabash iOS Server.
You'll need an Automation + Leaks/Allocations template and you'll need to use the TEMPLATE environment variable.
The next time I use this technique, I will generate a document that describes the process.
Related
I have used Appium in Mobile automation with IOS and Android. I wanted to know about advantages that offers Calabash and if there are common point in generated scripts of both tools?
I have followed this link : Appium VS Calabash
I have hands on in mobile automation quite a long time. let me tell my point of view on both tools.
both also have advantage and disadvantage find below:
I have used calabash and appium both for automation. Appium has more advantages than calabash.
Advantages of Appium over Calabash:
1. You can write your code in multiple language like Java,Python, C#, php etc. However, Calabash restrict to use RUBY only.
Appium comes with many predefined methods such as OpenNotification(), StartActivity(), InstallApp etc. Calabash does not provide such predefined methods.
Appium set up is very easy as compared to Calabash.
In calabash, it installs the app on the device every time you run the test. However, In appium you have a feature where in you don't need to install the app every time. It works with previously installed app on the device.
Appium works with both native and hybrid app.
There are many advantages of appium. I hope this helps.
Try them both for yourself.
I found calabash easier to get started. I had a complete user flow test working from setup to working test in Calabash in half the time it took for Appium.
I don't like using xpath. It makes tests brittle and subject to breakage if something in the layout changes. Appium requires mainly the use of xpath. Calabash allows you to drive actions and interactions through the selection of elements by their text . I prefer this because that same text is normally in the designs and/or requirements.
Some comments in other posts are not accurate. Calabash does not require the install of the app every time unless you don't look beyond the surface. There is a simple means to enable/disable it. I have logic that runs either way based on passing an argument. Calabash has more than enough existing methods to interact with your app. It has a built in debug that allows you to tinker with element interaction and those same calls can be copied and reused in your test code. Using it with getgauge.io provides reporting akin to the built in Cucumber but is easier to run multi-threaded and easier to write your feature/spec files.
Be fair to both tools and yourself. Create a simple project in each and try for yourself. Don't make a decision solely based on someone else's opinion.
There is no common part in the automation scripts since Calabash uses query for interacting elements but Appium uses selenium find functions.
The best part of the Calabash is that you can use cross-platform frameworks for automating ios and android application with same feature file and most part of automation code. However the worst part is that Calabash is Ruby only.
Look at this post for more details about the differences of Calabash and Appium.
Since you have already read the differnces , Few things to note that Calabash can also be used with Xamarin which is also a good tool to test hybrid apps and you can run the same script for ios and android apps with some minor changes.Also it has been now integrated into visual studio,so microsoft support is inevitable and some users might feel comfort factor.Lambda expressions are also a plus.A major setback with the Calabash however that it only uses ruby.
Appium on the other hand can support various languages which is plus and you can use webdriver as well.However it struggles with hybrid apps but you can overcome that functionality by using google chrome mobile view to find coordinates to click.one thing for sure is that sause labs have many bugs to fix in Appium yet.
I need to know if I can do Continuous Integration with XCode server. In example: set run the tests every night or when someone commit changes, and more..
I am trying to decide one iOS Ui Automation tool to integrate with my Xcode server
Thanks
There are a few problems here:
UIAutomation has no built in support in Xcode Server. I've filed bugs, I've chased down people at WWDC. Most I've ever gotten on this problem is basically "shrug." I'm not sure UIAutomation is a priority for Apple right now. So you're not going to get any official support.
As was noted, you might be able to use a trigger. The trigger won't be able to add anything to the Xcode Server report, besides possibly the error logging. But you're not going to get anything added to the nice report table.
Running on actual devices has traditionally been a problem (if you care about that.) The loading the app part has been a problem for us, but Xcode Server might be able to preload the app for you. In addition, it seems like this might have changed in the iOS 8 SDK.
There is just a lot of uncertainty around this sort of workflow. I'm hoping Apple eventually makes an announcement or adds a new tool, but the best answer I've gotten is if you want to go down this path, use UI Unit Tests. That's a shame because it requires knowledge of Obj-C or Swift, and means interacting with the app at an API level instead of an abstract level, but if you're looking for the direction Apple wants to see people go, that's it.
Edit 7/4/2015: As of WWDC 2015, there is a new UI Testing component as part of Xcode 7 that, in my experience, seems totally supported, and is promising Xcode Server support. I would very strongly recommend using that, and not using the Instruments UIAutomation tool.
With Xcode6 right around the corner they are adding some features to XCode Server specifically it looks like "Triggers" will be helpful for running iOS UIAutomation tools. Since you can run UI automation scripts from cmd line it should be possible to utilize triggers to run your scripts post builds. This along side the logic for when a bot should run will let you decide if it should be nightly or on every commit.
https://developer.apple.com/library/prerelease/ios/documentation/DeveloperTools/Conceptual/WhatsNewXcode/Articles/xcode_6_0.html#//apple_ref/doc/uid/TP40014509-SW1
I wrote a framework around UIAutomation called Illuminator to handle tasks like nightly test runs, pull request tests, and other automated conveniences.
It provides a flexible and extensible command line that can execute any particular subset of tests that you'd like, and produces reports in JUnit format (used by Jenkins).
I'm curious if there is any tools available to record app session in iOS, so later I can use it for testing by running it automatically?!
I used to use this kind of tool in Visual Studio to test my website, I'm wondering if there is anything similar in Xcode.
After some research, I guess my only option here would be to use Automation tool from the Instruments, and just writing javascript code to run my app in a particular way.
I am working on automating my iOS project's build process using Jenkins and xcodebuild command line tool. I am also able to test the application on simulator using UIAutomation testing scripts (instruments command line utility). But I would like to use other instruments like Time Profiler, Leaks, etc.. along with the automation to automatically test for irregular behavior.
Although I can make a custom template and run it with automation, but the problem is I can not parse the results of these other Instruments. Is there a way, for example, to know that there is a leak in my code, and trigger an email (from jenkins)?
It's possible (I've done it), but it is terribly complicated to do well. You need something to translate the Instruments results plist to something Jenkins can understand. You can use XSLT (I wouldn't recommend it), or write a script in perl/python/etc.
The Jenkins XCode plugin is an excellent tool for automating Xcode from Jenkins.
https://wiki.jenkins-ci.org/display/JENKINS/Xcode+Plugin
That aside, building a CI environment around UIAutomation is doable, but there are a lot of gotchas - the Jenkins user has to have the correct set of permissions to use the dev tools, has to be in the right group, and has to be logged into the UI for the simulator to run correctly from Instruments. Make sure you can get that far before you invest too much time into parsing the output of the Instruments run.
I am looking for any Functional Automation testing tool for Testing my native Ipad application,whether its a paid tool or open source, but i am not looking for only record and Play option, the Tool should have option to write the scripts in any programming language, and should generate a Proper results after running the test.
You can look at the UI Automation documentation.
The Automation instrument, guided by your test scripts, exercises the
user interface elements of your application, allowing you to log the
results for your analysis. The Automation feature can simulate many
user actions supported by devices that support multitasking and run
iOS 4.0 or later. Your test script can run both on the iOS device and
in the iOS Simulator without modification.
An important benefit of the Automation instrument is that you can use
it with other instruments to perform sophisticated tests such as
tracking down memory leaks and isolating causes of performance
problems.
Hi I am also doing a similar thing and I found http://www.cloudmonkeymobile.com/monkeytalk-documentation/monkeytalk-faq
I am currently trying it out and it seems to be working fine. I was wondering if you found anything else that may be open source?
Thanks...