I want to learn General SVM implementation which uses QP problem for training. Initially I do not want to learn Sequential minimal Optimization(SMO) kind of algorithm which over comes the QP matrix size issue. Can any one please give me some references to learn Pure General SVM implementation in any programming languages like C,C++ or Java. So that I can understand basic issues in SVM and it will help me in learning some other SVM optimized algorithms.
This blog post by Mathieu Blondel explains how to solve the SVM problem both with and without kernels using a generic QP solver in Python (in this case he is using CVXOPT).
The source code is published on this gist and is very simple to understand thanks to the numpy array notation for n-dimensional arrays (in this case, mostly 2D matrices and 1D vectors).
You could check some of the resources mentioned here. It is also advisable to have a look at the existing code. One of the most popular implementations, LIBSVM, is open-source, so you can study the implementation.
Related
I am referring Google's machine learning DataPrep course, in this lecture https://developers.google.com/machine-learning/data-prep/construct/sampling-splitting/imbalanced-data about solving class imbalanced problem, the technique mentioned is to first downsample and then upweight. This lecture talks about the theory but I couldn't find its practical implementation. Can someone guide?
Upweighting is done to calibrate the probablities provided by probabilistic classifiers so that the output of the predict_proba method can be directly interpreted as a confidence level.
Python implementation of the two calibration methods is provided here - https://scikit-learn.org/stable/auto_examples/calibration/plot_calibration.html#sphx-glr-auto-examples-calibration-plot-calibration-py
More details about probablity calibration is provided here - https://scikit-learn.org/stable/modules/calibration.html
Looking for scientific article references for the network architecture presented in Deep MNIST for Experts tutorial (https://www.tensorflow.org/versions/r0.9/tutorials/mnist/pros/index.html)
I have a similar image processing data and I'm looking for a good vanilla architecture, any recommendations?
Currently the best solution for this problem are wavelet transform based solutions
You probably don't want to look at Deep MNIST for Experts as an example of a good architecture for MNIST or as a scientific baseline. It's more an example of basic Tensorflow building blocks and a nice introduction to convolutional models.
I.e, you should be able to get equal or better results with a model with 5% of the free parameters and less layers.
As the question states. I am aiming to train a neural network where the weights are complex numbers. Using the default scikit learn netwokrs and building on this (editing the source code) the main problem I have encountered is that the optimizing functions used in scikit learn taken from scipy only support numerical optimization of functions whose input are real numbers.
Scikit learn is rather poor for neural networks it seems specially if you are wishing to fork and edit the structure is rather unflexible.
As I have noticed and read in a paper here I need to change things such as the error function to ensure that at the top level the error remains in the domain of real numbers or the problem becomes ill defined.
My question here is are there any standard libraries that may do this already ? or any easy tweaks that I could do the lasagne or tensorflow to save my life ?
P.S. :
Sorry for not posting any working code. It is a difficult question to format to the stackoverflow standards and I do admit it may be out of topic in which case I apologize if such.
The easiest way to do this is to divide your feature into the real and imaginary components. I've done similar work with vector input from a leap motion and it significantly simplifies things if you divide vectors into their component axis.
Tensorflow has elementary complex number support.
If you have to build the neural network nodes by yourself, you can take a glance at this blog.
For holomorphic functions, complex BP are fairly straight forward.
For non-holomorphic functions, they need careful treat.
I am trying to implement an SVM in Rapidminer. However I am presented with several SVM implementations, libsvm, mysvm,JMySVM, Particle Swarm Optimization based SVM and Evolutionary SVM. Know I know the basic differences between the implementations but what are the advantages and disadvantages of them to know which one to implement?
I am not finding much information about this online, I would like to avoid a try them all to see which one presents the best results. So I would like to know in which situation I should use them.
From the first, you seem to confuse different implementations and algorithms. As far as I know, libsvm, mysvm and JmySVM are standard implementation which solve the SVM optimization problem by algorithms such as sequential minimal optimization.
On the contrary, the other SVMs you mentioned (additionally) use less common approaches like particle swarm optimization or evolutionary algorithms for the optimization. Such methods usually give you good approximation with small effort, which might be advantageous for large-scale problems (--but I admit I don't know the exact motivation for their invention).
If you are looking for the SVM model which is common in machine learning and related fields, I would suggest you to try the library libsvm. Alternatively, you can have a look on the collection here.
I was reading the paper on Relational Fisher Kernel which involves Bayesian Logic Programs to calculate the Fisher score and then uses SVM to obtain the class labels for each data item.
I don't have strong background from Machine learning. Can someone please let me know about how to go about implementing an end-to-end Relational Fisher Kernel and what sort of input would it expect? I could not find any easy step-by-step flow showing this implementation. I am ok with using libraries for SVM etc. (e.g. libsvm), but I would like to know the end-to-end flow (in as easy language as possible). Any help will be highly appreciated.
libsvm does not implement the Relation Fisher Kernel, however, you can calculate the Fisher information matrix as described in the paper, and the use it as the precomputed kernel input to libsvm. See: using precomputed kernels with libsvm