We were exploring various test suites for mobile automated testing and ran into this company called Perfecto Mobile. One of the features that blew me away was they are able to (without jailbreaking) effectively perform a "Remote desktop" on a physical iPad.
So, the iPad's screen is mirrored within a web application, it can register touch / swipe events on the web app and perform them on the device. The only relevant technical detail I have is that all this is being performed using commands sent over the USB cable.
I'm really curious as to how this is implemented and details on relevant Private APIs if any.
Thanks,
Teja
I'm not familiar with PerfectoMobile, but I can give you a few pointers on how this can be accomplished:
For the mirroring, one way would be to look at using AirPlay, the APIs are pretty well documented, but not to do what we're talking about which would require some serious reverse engineering, but it's definitely possible, these guys have done it. A different approach would be to run a background app that would periodically take snapshots of the main screen, and send them over a socket connection to a client. You could do this as a VNC server, and to incorporate the remote view in a web app, you could use noVNC. As far using a USB connection, in the case of the background app talking to a client over TCP, you could to a port forward.
To actually perform on the device the touch events sent from your remote viewer, most people have been using the GSEvent group of functions from the GraphicsServices private framework without needing to jailbreak the device. Again, a background app would receive over a socket an instruction such as "Tap there", instantiate the GSEvent, and inject it so it gets processed in the run loop of the most front app.
These few possibilities, at least, have been implemented successfully in different iOS apps up to iOS 6.1 (iOS7 is a different animal). You won't find any such app in the App Store, since Apple clearly prohibits the use of private frameworks in 3rd party apps, instead people deploy them in-house using Enterprise and ad-hoc provisioning profile. On Android however, there's VMLite available in the Play Store.
If you looking to share screen from ios / android, check out skreen.me. They have sample apps you can try out, also they provide libs for mobile app integration.
Related
This question was asked a couple of years ago (iOS app Blocker possibilities) and I'm wondering if there is anything that has changed in IOS since this question was answered in 2017 that would allow a third party app to prevent an app from being opened or used (that doesnt use the Device Enrolment Program)?
As mentioned in the linked post, the freedom app prevents the use of other apps via a VPN. I am wondering if there is a more straightforward means of a third party app blocking other apps or whether sandboxing means that the use of a VPN is the only way of a third party app blocking other apps?
I am not an expert, but it seems like Freedom is routing an iPhone's network traffic through a "local" VPN. This means that, while you could open an app like Chrome and navigate to google.com, Freedom's VPN would not forward the request.
I haven't installed or tested the app, but their website is carefully worded: "Install App Blocker" to install the Freedom app blocker. This blocks content on other browsers like Google Chrome and for apps."
I read "blocks content" to mean you can open the app, but it will fail to load content without a network connection. I haven't tested, but I suspect that you could still use "offline" apps.
Here is a screen grab of their website.
As you noted, Apple's mobile device management (MDM) solution is capable of preventing apps and other activities. Here is the list for MDM restrictions, and a list of Supervised restrictions.
It's my belief that a third-party app cannot prevent another app from functioning. Freedom is playing a network trick. To stop an app from actually booting would require a jailbreak. But I haven't explored this space in depth.
Is there a way to avoid bloating of my app in appstore if my app wants to support more integrations. For example i have 10 barocde integartions done in my app either as a single framework or multiple frameworks. But the user needs only one integration out of 10. So is there a way to provide the only integration required by him without unnecessarily downloading all 10 integrations either though on-demand or achieving it through inter app communication by providing each integrations as a separate app and communicating between them without any hassle or anything like services in android can be achieved in iOS so that service will run at background and broadcasts the data to the app once it receives.
I'm going to develop few apps for iOS. Can they communicate to each other via sockets? Let's say one app runs as server (even in background mode) and another as client connects to server app and perform some communication. Does it violate any App Store rules? Any available IPC in iOS if my idea is not working for some reason?
Of course you can and it doesn't violate any rules.
Just just the CoreFoundation libraries.
There's no point in me describing it to you if it already has been all written in the official Apple documentation:
https://developer.apple.com/library/ios/documentation/NetworkingInternet/Conceptual/NetworkingTopics/Articles/UsingSocketsandSocketStreams.html
You can find here examples, all approaches and things you can and cannot do.
EDIT:
You didn't make that clear but it seems like you wanted the apps to run on the same device. You can do that but that would mean that one app has to run in background, and to do that you need some kind of hack (for example: background updating location mode ON) and that won't get past apple store. You need a very good reason to have your app ran in background (music in background, update location in background for fitness apps etc.)
Besides, it's a duplicate question:
iOS - Is it possible to communicate between apps via localhost?
As long as your apps are in the same app group, then yes, socket IPC is allowed. If not, then no. Sockets map to file descriptors (Berkeley sockets), and these files are sandboxed to the app or app group. More info: http://ddeville.me/2015/02/interprocess-communication-on-ios-with-berkeley-sockets
When an iOS Device goes into "sleep mode", even a configured VPN is, unfortunately, turned off.
A customer that wants an app for his iPad has to use VPN for security, so anything must be done through VPN. The app has to synchronize a lot, but the iOS device might turn off the VPN by itself. That's why I just need to trigger it again every time i need an internet connection (or keep it alive).
Do any of you guys have anything in mind?
Even using Private API is ok (the app will only be distributed to a customer, not through App Store).
Or using any app like Cisco AnyConnect would do the job.
Two ideas:
1) You can create a configuration profile with VPN on demand. This require client certificat authentication though. My guess that if some application will try to access domains or hosts defined in VPN on demand section in sleep mode, VPN will be turned on.
Update: At some moment Apple was sued regarding VPN on demand, so I believe they dropped this feature and after returned it back modified. It's good idea to read this article for iOS 7: http://support.apple.com/kb/ts4550
2) Idea which I offered for this question (iOS6 toggle WiFi on non-jailbroken device). Here is the copy of text from there:
I think it make sense to look at SystemConfiguration.framework.
It has set of API to work with different connection interfaces, which are defined here: http://developer.apple.com/library/mac/#documentation/Networking/Reference/SCNetworkConfiguration/Reference/reference.html
This one is private API.
Update: I believe there were some changes in SystemConfiguration and Apple tightened security around this framework.
3) Apple may grant to some applications additional entitlements, so they can build functionality which no other application can build. As example, VPN client companies (like Cisco) got special entitlements to build what in iOS called "VPN plugins", which is essentially VPN client.
Its now possible to do with NetworkExtension framework available in iOS 8 and up.
If you go to project settings->capabilities you will find "personal VPN". By turning it on you get the framework added to your project.
There is little to no official documentation, but you can check out the WWDC video here https://developer.apple.com/videos/wwdc/2015/?id=717
I wanted to make an application for the iPad that will be controlled from an iPhone-app. Is there a possibility to access the PhoneGap-App on the iPhone?
I don't want to use an external server because the app should run offline in an Ad-Hoc network.
First I thought of creating a local webserver on the iPad, but I couldn't find anything PhoneGap related. Any Idea?
You were probably on the right track with the web server idea.
I am not sure how well the various solutions are supported in PhoneGap, but I would think you could use some sort of WebSockets implementation for the bi-directional communication between the iPhone and iPad apps.