I have already make some research but could not find any useful information. Is it possible to install something like mini pytorch just for load a pre-trained model and call prediction method?
Entire PyTorch library size is too big, so I would like to prevent on this issue.
Does anyone have idea? Thanks in advance.
For production you need to use libtorch and the whole package is around 160MB compressed I guess.
You ship the Dlls that your application requires and at the very minimum I guess it could be around 170-200 MBs if you only use the torch.dll
And there is not a pytorch mini or anything like that as far as I know.
Related
i am currently playing around with some generative models, such as stable-diffusion and i was wondering if it is technically possible and actually sensible to fine-tune the model on a Geforce RTX3070 with 8GB VRAM. Its just to play around a bit so small dataset and i dont expect good results out of it, but to my understanding if i turn down the batch size far enough and use lower resolution images it should be technically possible. Or am i missing something because on their repository they say that you need a GPU with at least 24GB.
I did not get to coding yet because i wanted to first check if its even possible before i end up setting everything up and then find out it does not work.
I've read about a person that was able to train using 12GB of ram, using the instructions in this video:
https://www.youtube.com/watch?v=7bVZDeGPv6I&ab_channel=NerdyRodent
It sounds a bit painful though. You would definitely want to try using the
--xformers
and
--lowvram
command line arguments when you startup SD. I would love to hear how this turns out and if you get it working.
I wanted to make a naruto hand sign detection model to later use it in a AR game and I tried implementing it in fastai using resnet50 after getting crowdsourcing the data( mostly me). But, I got a model predicting only one category for everything.
This is the kaggle link:
https://www.kaggle.com/vikranthkanumuru/naruto-hand-sign-detection-usin-fastai-diff-method
Not sure if this is a problem, but earlier I had around 28 images per group so I made a video of myself doing the various signs and used opencv to save frame by frame. I later removed the ones that did not confine to any group and this increased the size of the dataset from 220mb to 2GB. Was this proper or is it the reason the model is bad?
This is the link to the dataset
https://www.kaggle.com/vikranthkanumuru/naruto-hand-sign-dataset
I am not sure how to proceed further and would appreciate any help. Thank you very much.
Edit: If anyone is interested in the completed thing, here's the link: https://www.linkedin.com/feed/update/urn:li:activity:6640529067936440320/
I have fixed a bit of your code. Have a look. I did not run it for the whole stretch though but it should show you what you need.
The test set only contained those images but take a look at the validation.
Also switch to vgg16. resnet50 is not necessary here.
https://www.kaggle.com/subhaditya/naruto-hand-sign-detection-using-fastai?scriptVersionId=29471636
What is the efficient way to prepare ImageNet (or other big datasets) for Tensorflow federated simulations? Particularly with applying custom map function on tf.Dataset object? I looked into the tutorials and docs but did not find anything helpful for this usecase. This tutorial (https://www.tensorflow.org/federated/tutorials/custom_federated_algorithms_2) shows MNIST processing but this dataset is relatively small.
Could you please clarify what exactly you mean by "efficient" in this context. I presume you've tried something, and it wasn't working as expected. Could you please describe here how you went about setting it up, and what problems you ran into. Thanks!
One thing to note is that the runtime included in the first release will only work with datasets that fit in memory. Perhaps this is the limitation you are running into.
I want to train a new haar-cascade for glasses as I'm not satisfied with the results I'm getting from the cascade that is included in OpenCV.
My main problem is that I'm not sure where to get eyeglasses images. I can manually search and download, but that's not practical for the amount of images I really need. I'm specifically looking for images of people wearing eyeglasses.
As this forum contain many experienced computer vision experts, I hope someone here can guide as to how to obtain images for training.
I'll also be happy to hear other approaches for detecting eyeglasses (on people).
Thanks in advance,
Gil
If you simply want images, it looks like #herhuyongtao pointed you to a good place. Then you can follow opencv's tutorial on training.
Another option is to see what others have trained:
There's a trained data set found here that might be of use, which states simply that it is "better". I'm assuming that it's supposed to be better than opencv.
I didn't immediately see any other places for trained or labeled data.
I'm starting to work with crf++ and crfsuite (both use a very similar file format). I want to do things related to images (segmentation, activiy recognition, etc). My main problem is how to build the training file. Has anybody work with crf and images? Has anybody explain me or give some file to learn.
Thanks in advance.
CRFsuite is faster than CRF++ and it can deal with a huge training data. I tried both of them. They perfectly work on a reasonable amount of data, but when my dataset increased to be more than 100,000 sentences, CRF++ did not manage to deal with it and suddenly stopped working.
Look at the following link
CRFsuite - CRF Benchmark test
there is a comparison between many CRF software in some criteria
I used CRF++ before and it worked very well.
But my field is natural language processing, and I use CRF++ for named entity recognition or POS tagging. CRF++ is easy to install on Linux but has some minor issue when compiling on windows.
You can just follow its document for training data format: each row represents a data sample and each column represents a feature type.
Or, you can also consider Mallet which has a CRF component.
Probably you should start with the DGM library (https://github.com/Project-10/DGM), which is the best choice for those, who never worked with CRFs before. It includes a number of ready-to-go demo projects, which will classify/ segment your images just out-of-the-box. It is also well documented.
I have just came across this one for Windows:
http://crfsharp.codeplex.com/
maybe you also want to try CRF component in Mallet package.