I need to run a beta test for an iOS 8 app through Apple TestFlight. Is it allowed to upload 2 versions of a beta app for an A/B test? (I would like to switch the app icon and some other things to check what works better for the users.)
Notes on bounty:
Especially interested if there are any experiences with submitting two
similar builds for external testing, as builds have to go through the
(albeit lighter) review process.
I would presume it would be OK, as one can submit freeform notes for
the reviewers and explain the situation—AND because, in any case,
externally tested apps have to go through the normal App Store review
process before going live; so acceptance in external testing wouldn’t
be a free ticket to App Store.
But these are just my assumptions and hence the bounty. Has someone
done this? With negative or positive results? Or both—review processes
are notoriously independent and, at times, arbitrary.
I’m not interested in interpretations of review
guidelines,
but how they are executed in practice.
Or should this all be handled inside the app with some A/B testing
framework? (Which is unfortunately more work than just creating two
bundle IDs… And this wouldn’t help in testing the app icon)
In order to have 2 active builds in test you need 2 apps (different bundle ids), the only choice you have is either through the same account or different accounts. It's easier to handle in the same account, less context switching for management.
We did both successfully without any issue raised from Apple. We did not put any special mentions in the review notes either.
Eventually you end up with more dead projects in your account but its only an issue for you, Apple wont complain.
Why tough
A/B testing can be done in the same build except for some things like you mentioned (app icon). Usually this is handled through surveys tough.
The only advantage of having 2 apps is that all users can test both versions in your A/B test mitigated by their willigness to spend time testing both.
Disadvantages that jumps to mind:
sooner or later you will kill one of those version, potentially loosing some of your beta testers that chose only that version. Mitigated by the fact that you may not need beta testers after you release your appstore version
you wont be able to control who has which version to insure proper distribution and move users from one to the other unless you are ready for lots of hands on communicating with your beta testers.
I have not been able to have multiple versions in TestFlight. If I were to do this I would use a separate provisioning file and iTunes Connect account. There are a few steps to change the provisioning account and the app name so it is unique, but I think that is the only way to get around it. However to use external testers (since internal testers are limited to 25) you will have to go through the review process. Apple might object to having to review the same app twice if they caught it. I would advise rereading the Terms and Conditions, remembering you are submitting for Beta App Review.
As of April 11, 2017, Testflight allows you to you can distribute and test multiple builds at the same time.
https://developer.apple.com/news/?id=04112017a
You can upload multiple builds but you can test only ONE at a time and there is no way to do what you want unless you make a new unique app identifier and upload the same app on the new app identifier APP.
It is possible to upload more than one builds of same version!!! Just change the build number. Example, 'version 1.0, build 1.0', 'version 1.0, build 1.1'. You can see different builds in Test Flight.
Related
I've just published an app on Apple's store and I'm wondering about having multiple versions of the same app for testing on TestFlight. Of course dev doesn't stop when publishing... from now on I'll have to update the app store version (v1.0.0) with bug fixes (v1.0.1, v1.0.2, ...) and before doing so I'd like to check them in test flight to ensure the fix was appropriate.
My problem is that I'm already starting to develop the next version with further functionalities of the app which will become v1.1
So ideally I'd like to have my app available both for my bug fixes, for instance v1.0.2 and also my next version v1.1.0 (this will include all bug fixes made to the store version and also many new features, refactors, redesign, etc)
I know that if I build and upload to the apple store connect a build with v1.1.0 (next version) I won't be able to upload one for a built with a bug fix on the current app store version (v1.0.2) since this version would be lower than the one I uploaded (next version)
Is there a way to accomplish this? I've read this article https://savvyapps.com/blog/using-testflight-to-distribute-multiple-versions-ios-app which solution is to create extra applications in iTunes with different app ids and bind them to different certificates. But what will happen when the next release is ready to be in the store? I would have to release it and then disable the previous one? How may this affect my users? Will they have to re-install a new app rather than updating it?
I really need to start testing and checking the next release of my app in TestFlight and also support the current one with updates if something pops up. Thanks in advance!
I am able to upload multiple versions of the app to TestFlight. Each upload requires a higher version/build number, but you can switch the TestFlight test version between them as need be for testing.
Once I submit a particular build for release, however, I seem to lose the TestFlight access to the old builds.
In short, you can have many builds available in TestFlight, but once you submit the app for release, you have to start over making builds for TestFlight.
You keep talking about numbers like v1.0.1. That looks like a public-facing version string, with a major, minor, and patch number.
But that is not what TestFlight cares about. Well, it cares to some extent. But all TestFlight really cares about is that every new build you upload has a new build number. This is just an integer which you simply increment every time you submit a new build.
So you could have v1.0.1(23) on the App Store, and then on TestFlight you could upload v1.0.2(24) which starts moving forward toward version 1.0.2, but also upload v1.0.2(25) which is actually an attempt at a prospective version 1.1. TestFlight doesn't know or care what these different builds signify. They can all exist simultaneously on TestFlight. Keeping them all straight and on their individual trajectories is up to you.
I currently have an app on iTunesConnect with a few hundred external beta testers using it. Important to note, we will have 2000 testers by the end of the month. I want to push new builds to this app- and this, I know how to do.
What I don't know is, what is the expected behavior for my beta users when I add a new build?
Our company cares a lot about user experience, and we don't want to have our current testers of our current build open the version that they've already installed, only to see it crash because I added a new build that i'm hoping will just update their current version automatically.
Apple does a good job of making a developer think this might happen. I've searched everywhere to find this answer in the docs- please help! After selecting a newly approved build to switch to in the External Testing portal, upon selecting Save, this alert appears:
(405 is the first build, 407 is the new build)
So, what happens when I save this- will users be notified that they need to update the app?
Will the "update" happen automatically for them if they've already installed the first build?
If they open the already installed version, will it simply crash?
If so, what can I do to prevent this from happening?
My team will likely want to send out 1-2 builds / week (of the same app, with fixes and improvements) to the same group of testers until we're ready to officially launch the app. I'd hate to think this would crash the app on them every time. As far as I know, there is no way for me to test this before performing this action- I'm already added as an Internal Tester, but that's a completely different UX in TestFlight (builds are made available to internal testers immediately after uploading)
If you think this has been asked already:
This is not a duplicate of this question- because I haven't attempted to send out the build yet. I want to make sure that linked issue does not happen to my testers!
My question is unlike this one where the user did not know how to properly increase his build number, unlike this one, referring to testing a new build of an app that already has a version in the app store, unlike this one which refers to a bug in the app store where a user couldn't initiate an external test after uploading a build, and unlike this one where the user just didn't know how the iTunesConnect portal works.
Is there a way to remvoe TestFlight apps from users that have installed them? Also is there a way that TestFlight can bake into the app some sort of password that the users all have to log in with (in case of a lost phone, we don't want our developement apps exposed).
If left untouched, the provisioning of your apps will eventually expire automatically. Even without the native ability to remove applications with TestFlight there is still something of an expiration date on the application.
That would still leave your question of a "baked in password prompt" and removing the application itself physically from the device.
The first part, the app checking for authentication could be solved by implementing a solution with a more robust SDK that happens to have that sort of security-minded approach. As far as I know, and based on TestFLight's feature grid, this exceeds the abilities of their tool.
The second part, removing the application itself from the device, would be accomplished by using a tool that has the ability to use MDM (Mobile Device Management) for device-level control. Specifically you'd want to look for something that can selectively control a single application, rather than having to apply a blanket MDM policy. Again based on knowledge of TestFlight and based on their web page this is also not something TestFlight is capable of.
There are solutions out there that will give you exactly what you are asking about - easy beta testing with the added ability to force the app to check in and re-authenticate as well as the ability to remove applications from the device when you're done testing. If you hit your search engine of choice you can find a few tools that will give you a "yes" to all of your questions here. The list is very short so they're easy to find. :)
If it is at all helpful to you, I am associated with one of those companies, AppBlade, and would be happy to answer questions about this sort of thing. We're at https://AppBlade.com and you're welcome to give us a call or even log into the tool to see how it works for yourself.
Unfortunately you can't delete apps that are already installed on the device via TestFlight, unless you do it on the device itself. As for the password, TestFlight doesn't exactly support that either. You could however put a passcode lock feature in all of the Beta versions of your apps through your code. Sorry thats probably not the answers that you wanted to hear, but TestFlight is still in its early stages.
You are not able to delete apps from a users device, however TestFlight is testing in their 'Area51' an option to force users to update to a new build if there is one available.
If you no longer want testers to access your app you probably could add a new build which justs shows some info screen.
There is a way to expire the builds in the app store connect when you click on build.
Another way if you want to get rid of it as a tested to open the app page and click on stop testing.
I'm about to make an individual iOS dev account, but I would like to share my work with a colleague or two for input that do not have an account (they have Xcode). They will look at the code maybe a bit, but mainly to test the app itself and provide feedback for me. Currently I don't have an account and what I have done is take a screencast of the app in the simulator and send the screencast. Obviously not ideal. So what are my options to share my progress on a daily basis? I think just to have them run in the simulator on their end is fine, until the app is almost complete then maybe on their phone would be good too. Thanks,
rc
A) look at testflight (testflightapp.com) - it's a site which allows you to email ad hoc builds of an app to testers.
B) put your code into version control and give them access. (ie GitHub.com) then they can build it themselves with Xcode onto their devices.
I'd go with A, it gives you more control and the potential for fewer support questions :)
I'm not sure why you would want non-developers looking at your code but I'd that's really needed, option B :)
The type of developer account you get and your ability to share your code are two unrelated issues. Unless you want your colleagues to be able to build your code and install it on test devices, they don't need developer program subscriptions. You can share your code with them in whatever way suits you (give them read access to your version control system), and you can build test versions that they can install on their devices. The only thing they won't be able to do is to build the app themselves for installation on a device.
If the main purpose is for testing, then get an enterprise certificate, sign the app with it and send the ipa file. They will install it and test the app.
My application is in the Apple App Store but when downloaded it crashes after the splash screen.
I thought the week long approval process was to ensure the quality of the app.
Version 1.0 of my app does run but I hear there is no way to roll it back. For now I have changed the availability date to the future so that people do not download it. When will it be taken out of the search results?
Thanks.
The approval process is not for QA testing. (Of course, they will reject an app if it crashes while testing they are other for things, such as violation of various SDK rules, HIG guidelines, etc.) A developer has to test and QA your apps themselves on the OS versions and the iOS device types for which they submit the app as appropriate for, and under stress conditions as well. A developer also needs to make very certain that the build they submit is identical (except for certificate signing) to the builds they have tested. (It is a common mistake to have different Build Settings or source files selected between the Release and Distribution builds.)
Check to see if a bad preference setting is the culprit.
Or if it worked only for you, then it may be the lack of a preference setting. You may have created a good preference before the bug was introduced.
Was taken out of search results by the end of the day.