Is there any way to train PyTorch models directly on-device on an iPhone via the GPU? PyTorch Mobile docs seems to be completely focused on inference only, as do the iOS app examples (https://github.com/pytorch/ios-demo-app). I did find this article about using MPS backend on Macs (https://developer.apple.com/metal/pytorch/), but not sure if this is at all viable for iOS devices. There's also this prototype article about using iOS GPU for PyTorch mobile (https://pytorch.org/tutorials/prototype/ios_gpu_workflow.html), but it too seems to be focused on inference only.
We are attempting to train a large language model on the iPhone 14 and in order to make that possible given the memory constraints, we would like to a) discard intermediate activations and recompute them, and b) manage memory directly to write some intermediate activations to the filesystem and later read them back. We suspect that converting a PyTorch model to CoreML format and using CoreML for training would prevent us from making these low-level modifications, but PyTorch might have the APIs necessary for this. If there's any examples/pointers that anyone can link to that would be great.
Related
I am trying to deploy a pretrained model on a android application . Now, the need is to retrain the model with the data captured locally.
Specifically what is happening is, There is a pretrained dnn model which predicts the quality of a video seeing the bandwidth. The neural network was trained on some data which had the bandwidth and the corresponding video quality. Now that model has to be deployed on device for retraining on the new data.
This new data is already captured via a mobile application and is stored in required format(csv) .
First i thought of using tflite but it does not support on device retraining.
Now i am trying to use DL4J but could not understand how to do it.
If it possible to use DL4J in this case then how can i do it.
If not, then is there a another approach.
ps. I have tried my best to write the problem statement clearly. Pardon me, if u find it difficult to understand. Please comment what u have not understood, i will clear it.
The documentation for porting an already trained TensorFlow Model to iOS is well defined:
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/ios
However, nowhere is mentioned if the model:
can be further trained on the device, or
can be created from scratch and trained on the device
Is this possible with TensorFlow?
I am aware of other Swift/C++ libraries that offer on-device training, but I am more interested in this technology.
Starting with CoreML3 and the UpdatableTask, the training on device is now part of the API: https://developer.apple.com/documentation/coreml/mlupdatetask
I have a project which I should classify the data coming from several sensors(time series based data) like gyroscope to several classes. I have used several classifiers including SVM, decision tree, neural networks, KNN,... in a batch scenario. My ultimate goal is to find a real-time classifier which is accurate, light and also has the ability to improve itself to implement it on my device which has limited sources(CPU, RAM,..). I was thinking a semi-supervised classifier since I can save a few labeled data on my device and use the future data points to improve my classifier. Does anyone have any recommendation or experience in this regard?
Online learning is very challenging. I recommend you steer away from now and use batch learning. You can always update the model as you update the mobile app or just make the app look for a new updated model on your server every x days.
Now, how to run a machine learning algorithm efficiently on a phone with limited resources. First, you have to identify which platform you are using. I assume you want to get a platform agnostic answer. Most ML algorithms (except lazy learning ones) can run efficiently on smartphone, have a look at this benchmarking experiment.
You have several options here:
iOS: Here's a list of all machine learning libraries available publicly.
Android: Weka for Android, this lib has a huge number of ML algorithms.
Platform agnostic deep learning: Tensorflow, you can export your models to TensorFlow lite (tutorial) and deploy them on any mobile OS and Caffe2 to train deep learning models and export them to any smartphone OS.
Is it possible to ship an iOS app with a CoreML model and then have the app continue improving (training) the model on device based on user behaviour for example? So, then the model would keep growing and improving right on the device with no need of a server support...
It's now possible with Core ML 3.
https://developer.apple.com/videos/play/wwdc2019/704/
Skip to 9:00 to see it in action. If you just want the code, skip to 13:50.
The answer is YES.
Since CoreML 3 is greatly optimised – the answer is YES, you can train a CoreML model on device when your app is running.
However, using CoreML 2 it's not possible to train a model on device, because running CoreML 2 app, considerably much power is required to train a model in comparison with CoreML 3. That's why desktop and cloud computers with power GPUs are used for creating a pre-trained models. In CoreML 2 your MLmodel must be pre-configured and you have to include all pre-processing techniques like Edge Detection or Frame Differencing at that stage.
I'm trying to do the same thing. Apparently, when you convert your model to Core ML format with coremltools, you can pass the "respect_trainable" argument to the converter and it will automatically make the model updatable.
I'd like to make an app using iOS's new CoreML framework that does image recognition. To do so I'd probably have to train my own model, and I'm wondering exactly how much data and compute power it would require. Is it something I could feasibly accomplish on an dual core i5 Macbook Pro using Google Images for source data or would it be much more involved?
It depends on what sort of images you want to train your model to recognize.
What is often done is fine-tuning an existing model. You take a pretrained version of Inception-v3 (let's say) and then replace the final layer with your own. You train this last layer on your own images.
You still need a fair number of training images (a few 100 per category, but more is better) but you can do this on your MacBook Pro in anywhere between 30 minutes to a few hours.
TensorFlow comes with a script that makes it really easy to do this. Keras has a great blog post on how to do this. I used the TensorFlow script to re-train Inception-v3 to tell apart my two cats, from 50 or so images of each cat.
If you want to train from scratch you probably want to do this in the cloud using AWS, Google's Cloud ML Engine, or something easy like FloydHub.