Which ls-svm toolbox can use in matlab? Which implementation do you recommend?
I am not 100% sure what type of SVM you're referring to, but I'm assuming you're interested in an implementation of the least squares SVM of Suykens & Vanderwalle, NIPS 99. If that's the case, I believe neither libsvm nor liblinear do that; check out http://www.esat.kuleuven.be/sista/lssvmlab/ .
If you're interested in a normal quadratic-programming formulation of the svm with quadratic slack penalties, libsvm and liblinear should work. Also, the newer subgradient-based solvers, such as pegasos may be useful as well but I am not sure of whether there is a good matlab library for you to use.
Check out both libsvm and liblinear:
http://www.csie.ntu.edu.tw/~cjlin/libsvm/
http://www.csie.ntu.edu.tw/~cjlin/liblinear/
These the are the fastest SVM solvers that I know of.
Related
Many packages such as sklearn use RBF kernel as default for SVM classification. I wonder if there is a proof/explanation why this is the "best" kernel for a general use.
Is there some analysis for the richness of boundaries that can be written as a_1*K(x,x_1)+...+a_m K(x,x_m)=0 for different kernels?
I am looking for some references.
Thank you.
What does it mean to provide weights to each sample for
classification? How does a classification algorithm like Logistic regression or SVMs use weights to emphasize certain examples more than others? I would love going into details to unpack how these algorithms leverage sample weights.
If you look at the sklearn documentation for logistic regression, you can see that the fit function has an optional sample_weight parameter which is defined as an array of weights assigned to individual samples.
this option is meant for imbalance dataset. Let's take an example: i've got a lot of datas and some are just noise. But other are really important to me and i'd like my algorithm to consider them a lot more than the other points. So i assigne a weight to it in order to make sure that it will be dealt with properly.
It change the way the loss is calculate. The error (residues) will be multiplie by the weight of the point and thus, the minimum of the objective function will be shifted. I hope it's clear enough. i don't know if you're familiar with the math behind it so i provide here a small introduction to have everything under hand (apologize if this was not needed)
https://perso.telecom-paristech.fr/rgower/pdf/M2_statistique_optimisation/Intro-ML-expanded.pdf
See a good explanation here: https://www.kdnuggets.com/2019/11/machine-learning-what-why-how-weighting.html .
I have a datasets of MFCC that I know is good. I know how to put a row vector into a machine learning algorithm. My question is how to do it with MFCC, as it is a matrix? For example, how would I put this inside a machine learning algorithm:?
http://archive.ics.uci.edu/ml/machine-learning-databases/00195/Test_Arabic_Digit.txt
Any algorithm will work. I am looking at a binary classifier, but will be looking into it more. Scikit seems like a good resource. For now I would just like to know how to input MFCC into an algorithm. Step by step would help me a lot! I have looked in a lot of places but have not found an answer.
Thank you
In python, you can easily flatten a matrix so it becomes in a vector,for example you can use numpy and numpy's flatten function ,additionally an idea that comes to my mind(it's just an idea may or may not work) is to use convolutions, convolutions work very well with 2d structures.
I'm using both LibSVM and LIBLINEAR libraries in my program. LIBLINEAR gives pretty good results, but LibSVM with linear kernel performs much worse on the same problem with the same C parameter and bias = 1 for LIBLINEAR.
What could be the reason for that?
Also LinearSVC class from scikit-learn performs even better than LIBLINEAR, whch is also surprising considering that it's a wrapper of LIBLINEAR.
I have to train a Support Vector Machine model and I'd like to use a custom kernel matrix, instead of the preset ones (like RBF, Poly, ecc.).
How can I do that (if is it possible) with opencv's machine learning library?
Thank you!
AFAICT, custom kernels for SVM aren't supported directly in OpenCV. It looks like LIBSVM, which is the underlying library that OpenCV uses for this, doesn't provide a particularly easy means of defining custom kernels. So, many of the wrappers that use LIBSVM don't provide this either. There seem to be a few, e.g. scikit for python: scikit example of SVM with custom kernel
You could also take a look at a completely different library, like SVMlight. It supports custom kernels directly. Also take a look at this SO question. The answers there include a handful of SVM libraries, along with brief reviews.
If you have compelling reasons to stay within OpenCV, you might be able to accomplish it by using kernel type CvSVM::LINEAR and applying your custom kernel to the data before training the SVM. I'm a little fuzzy on whether this direction would be fruitful, so I hope someone with more experience with SVM can chime in and comment. If it is possible to use a "precomputed kernel" by choosing "linear" as your kernel, then take a look at this answer for more ideas on how to proceed.
You might also consider including LIBSVM and calling it directly, without using OpenCV. See FAQ #418 for LIBSVM, which briefly touches on how to do custom kernels:
Q: I would like to use my own kernel. Any example? In svm.cpp, there are two subroutines for kernel evaluations: k_function() and kernel_function(). Which one should I modify ?
An example is "LIBSVM for string data" in LIBSVM Tools.
The reason why we have two functions is as follows. For the RBF kernel exp(-g |xi - xj|^2), if we calculate xi - xj first and then the norm square, there are 3n operations. Thus we consider exp(-g (|xi|^2 - 2dot(xi,xj) +|xj|^2)) and by calculating all |xi|^2 in the beginning, the number of operations is reduced to 2n. This is for the training. For prediction we cannot do this so a regular subroutine using that 3n operations is needed. The easiest way to have your own kernel is to put the same code in these two subroutines by replacing any kernel.
That last option sounds like a bit of a pain, though. I'd recommend scikit or SVMlight. Best of luck to you!
If you're not married to OpenCV for the SVM stuff, have a look at the shogun toolbox ... lots of SVM voodoo in there.