Performance testing of native iOs apps - ios

I have a native iOS app developed in house which need to be tested for performance. Any suggestions regarding this like the approach, tools that can be used for this etc.

I would disagree with the notion that if iPhone 3GS works it works with everything above. I have done lots of testing across devices and versions of iOS and find bugs all the time that don't manifest on iOS 5 but will on 6 or 7. Bugs that happen on one device but not another because of performance differs in the hardware.
I would suggest that identify who your primary audience is and what devices they are likely to be using. What device/os combinations do you care the most about making a "legendary" experience. Do you want to limit/complicate yourself to supporting an older SDK or simply build against the latest and greatest?
Once you have these questions answered, get a few physical devices if possible for "real" testing, and then use simulator to flush out and iOS version bugs. A good spread might be an iPad 2, iPad Air, iPhone 4, and iPhone 5 or 5s.
Hope this helps.

Related

How to set up development environment for a particular version of ios

I work for a company that offers software solutions to the care industry. I have written an Android app that the care worker use to provide home care.
I've been asked to replicate the same app on the iOS platform. They have ordered a mini Mac 2.6Ghz Core i7 16GB ram machine to come next week. I know the Hardware is suitable for writing for the latest iPhone but what if carers have an iPhone 4, which alot will.
How do i set up my development environment to ensure the app works on earlier phones?
I've looked at the latest Stanford Uni tutorials(2013/2014) and they differ from the 2010 tutorials, this makes me think the later code will not run on iPhone 4 say.
Thanks in advance
Matt
iOS applications can run on previous devices. In Xcode you can see that in the storyboards it allows you to use a single storyboard for all versions of the iPhone. The only difference is the screen size: so you have work with screen constraints to adjust to different sizes.
It should not be an issue if people have an iPhone 4.

Do I need to separately test on iPod touch, iPad and iPhone?

I would like to start developing for iOS. Coming from an Android developing background, I know that the more types of devices you can get your hands on, the better testing will be, as all devices have wildly different specs, and what may work perfectly in your test device may not even run in another one, let alone look good.
I know that testing on the actual device is very important, as there are many limitations on what you can test on an emulator, so I've decided to get an actual device.
However, there are also tons of devices available in the iOS world! There's the iPod touch, the iPad and the iPhone, each in several different generations and configurations (8GB version vs 16GB version, WiFi version, 3G version, etc.). Not also the screen sizes, but also the aspect ratio is very different across devices, and also the included sensors.
I think that getting an application to run in varied devices should not be difficult, but is it necessary to actually test on all the device types you plan to support? Apple is not renown for its low price, and I would like to keep the starting costs as low as possible.
So, to conclude: Is it necessary/recommended to test on as many device types as you can in the iOS development world?
A small clarification: I'm specially asking if it is possible for there to be compatibility issues related to a specific device/family-of-devices that I would not be able to catch either by testing on the emulator nor a totally different device.
Generally speaking, the major differences in capabilities between testing on the simulator and testing on a device are:
The simulator does not use exactly the same sandboxing as the device. So, for instance, if your provisioning profile is missing your Passbook credentials, this problem will show up on a physical device but not on the simulator.
The simulator doesn't generally support GPS, multitouch, push notifications, Bluetooth, and some other specific features.
On a non-retina display, the simulator view for an iPhone 5+ or (especially!) a retina iPad will be nigh unusable at 100% because its size will exceed the size of your screen.
There are a few, very rare, crashes that occur only on the simulator and a few that occur only on the device.
The simulator does not always support the earliest iOS versions your app supports. For instance, the current version of Xcode (which you must use if you want to build for the latest iOS version) only has simulators from 5.0+ available.
Certain profiling with Instruments is, as far as I can tell, only available in the Simulator.
Now, in my specific case, I try to test on one of each screen resolution I support and one of each major OS version I support.
This boils down to the following array of test devices:
(480x320) iPhone 3GS running 4.3.3
(1136x640) iPod 5gen running the latest 6.x
(960x640) iPhone 4S running 6.0
(1024x768) iPad 1st gen running 5.0
(2048x1536) iPad 4th gen running the latest 6.x
Note that the iPad mini is the same resolution as the iPad 1st gen.
(My choices are skewed towards later iOS versions since I like to implement integration with all of Apple's snazzy optional features as they roll them out. It would probably be a more balanced assortment if one of the 6.x devices were running 5.1 instead.)
If you don't need to support 4.x, I would personally advise against it, since iTunes Connect no longer collects crash reports for it and the simulator no longer offers it. Of course, only you can decide whether you really need to or not, and if you do, focus a lot of your testing there as Xcode does not warn you if you are using APIs that were only introduced in 5.0, which will crash any device running 4.x.
Please note that there are ways to (with significant preparation) downgrade the version of iOS on a device, so if you really want to test more versions than you have devices for, you can (with a lot of effort). But you're probably better off cultivating a strong pool of beta testers for this, anyway.
Whilst it's obviously great to test on all possible devices, the iOS ecosystem is much tighter than Android, so you can narrow down the field somewhat.
You can start by limiting your target iOS versions. That will anyway cut out a number of older devices. iOS6 share of all iOS devices is now probably around 75% 4 months after release; iOS5+iOS6 upwards of 90%. If you are just now starting to develop for iOS, you could probably just target iOS6.
That means your minimum hardware platform is iPad2 / iphone3GS / ipodTouch4
Total list of devices
iPad: 2 3 4 mini
iPhone: 3GS 4 4S 5
iPod: 4 5
Ten devices.
But you won't need to test them all. You could sensibly narrow it to...
iPad: mini + one of the retina models
iPhone: 3GS + 5
for everyday testing.
Obviously if you do want to be more back-compatible, just replace the lower-end testing model for a lower-spec device (iphone 3, iPad 1).
The difference in storage capacity (8GB vs 16GB for example) will be mostly immaterial.
There are some hardware features you will have to pay special attention to, depending on your project. The obvious one is retina vs non-retina displays. Hardware features for location services is particularly nuanced between models.
Lowendmac have a pretty thorough iphone comparison chart...
I think that getting an application to run in varied devices should
not be difficult, but is it necessary to actually test on all the
device types you plan to support?
That depends a lot of what kind of apps you intend to build. For example, universal apps run on both small- and large-screen devices but may present themselves differently on each, so you'd want to be able to try both. Many apps target iPad specifically, so obviously testing on small devices isn't necessary. iPod Touch and iPhone are very similar, so testing on one or the other is often sufficient.
In short, you don't have to own every version of every device, and you don't probably don't have to test on every single version of iOS that's ever been released. But you do want to get your product tested on as many different devices and operating system versions as you can. So, cultivate a group of beta testers who will help you out by trying your app on their devices. The iOS developer program lets you add up to 100 test devices to your account precisely so that you can get your app tested in lots of different circumstances.
Testing on the iPod touch is generally not worth it.
In the iOS world, there is generally the iPhone/iPod, the iPhone 5/iPod 5th generation, and the iPad.
So, that is a total of 3 screen sizes.
If you plan on targeting more than the latest OS (iOS 6), that is where the complexity of testing comes in. Simply targeting iOS 5, and iOS 6 nearly doubles the amount of targets you need to test for.
I try and keep it simple. I test on my iPhone 5, and my iPad. Both of those run iOS 6. For iOS 5 and the small iPhone, I rely on the simulator.
I develop for both Android and iOS, so I get where this question is coming from. I have the luxury of being able to develop on most of the different iOS devices and I would say that in most cases I would feel comfortable testing on the least advanced device my user will be using. If it runs smooth on a iPOd 3rd gen then it is going to run very smooth on an iPhone 4S, 5, etc. For the different screen height for the iPhone 5, the simulator works very good with laying it out.
Things you will need to consider is if your apps have the option to use certain feature only a phone would have, like making calls. Also if you want to make a iPad or universal app, it would be very handy to have an actual iPad, but the simulator does work very good.

How do you debug an iPhone app without having an iPhone?

Okay, I made a pretty simple iPhone app, I tested it with the iOS 5 and iOS 6 simulators for both iPhone and iPad, and everything worked fine! But when I submitted it for review, they rejected it because it crashes on the iPhone 4, and iPad 3, Is there a way I can figure out how to fix this without buying an iPhone 5 and an iPad 3? They did send me the crash files, but I have no clue how to read them. Any suggested is appreciated! :)
You don't. The Simulators are not accurate enough to debug certain problems (the Simulators only run x86 code, not the actual ARM code in the more constrained environment a device presents).
So, you may need to buy, beg or borrow a suitable iOS device or two for testing.
You have to learn how to read the crash files. Try dragging them into the organizer to start with.
There are some aspects that are different on the devices, you should have at least one iOS device of some kind to test on. An iPad is a good choice as you can test both iPhone and iPad apps. Running on any one device will shake out many errors that would happen on all of them - if for example, you didn't realize the iPhone file system was case sensitive but the Mac (and therefore the simulator) filesystem is not.
Even an older iPad 2 refurb would do for such testing...
It's very difficult to debug problems on a specific device without actually having that device. I know that there are some apple stores in my area that will rent devices for a short time for that purpose. You might check around and see if you can find one to borrow or rent for a day or two.

Using iOS Simulator to emulate iPad 1

One of my customers is experiencing terrible slow downs only on the iPad 1 device. Is there any way of getting the iOS Simulator to emulate the performance/hardware of the original iPad?
I would rather not have to go out and buy an iPad one as iOS 6 won't support it this autumn.
The app runs fine on the iPad 2 and 3. I have tested using the allocations and activity instruments.
Any ideas? Thanks again. The OS is 5.1.1 and the App was compiled with XCode 4.4.1.
As far as I know, the simulator is what it says, a simulator, and as such will run as fast as it can on whichever computer you're using. You'll probably need to put delaying code into your run loops to simulate the slowness of the iPad 1, but maybe your client needs to be aware that the iPad 1 is much slower than the newer models - there are certainly some (particularly graphic intensive) operations which will struggle - you may find you need to adapt the user experience according the the device being used (Apple don't encourage this - they prefer you to check the iOS version - but there are several questions on SO which could help you here).

Is owning hardware necessary for iOS development and testing?

I'm learning iOS development and I need to know what hardware I need to test my apps.
Is the iPhone/iPad simulator in Xcode sufficient? Or do I need the hardware? I have an iPad 2, and an iPhone 3G. The iPad 2 is one generation old, while my iPhone 3G is three generations old.
My first project is a basic card game with networking, based on a tutorial.
Opinion: Considering the number of questions I see of the form "this works great on the simulator but not on my device" I'd say that having hardware for testing is necessary. I don't think you need every possible device but certainly ones that cover the features that your app uses.
It depends on features you need.
Example of things you can't test in the simulator:
Push notifications
Performance of an OpenGL game (usually you need a wide set of device to test OpenGL)
The simulator can be used for development, but the simulator is not relevant for efficiency. It is very recommended to testing on a real device too. Some of the services can not be developed on the simulator:
the push notifications
in-app purchase
iCloud services
And you know, that the iPhone 3G is not able to be updated for the lastest iOS (your iPad is able).
The first answer is YES, you need hardware as there are differences between the behaviour of the simulator and the devices. They won't always act the same as the simulator is a bit more forgiving than the device.
For example the simulator will find files (images/sounds/models etc.) even if the case is different between the request and the file name, the device will not find them. And there are more.
An other point is whether to buy/have devices to hold different iOS versions. I don't have them all as this is too expensive for me but I can say that this is a problem. No matter how much you will try to consider the differences between the devices you will always miss something and your app might not work or crash on this device.
Still you can consider this question by looking at the apps that you are going to work on. I would say that if your apps don't use the device hardware (camera for example) and don't have features that might cause problems on different devices you will be able to start with out the devices.
Bottom line is that if you want to deploy good working apps, in most cases it will be better if you could test your apps on a variety of devices.
It's not a requirement to have a equipment to test, but certainly very important. You can test FPS of your app, even not containing hand-made OpenGL. All features that you use on your app, like view effects, are tested for sure on a device. Since simulator uses your mac memory, you won't see any side effects from memory shortage. I believe your best chance is to have a iPhone 4 and your iPad 2.

Resources