I am wondering if there's support for RFE execution over GPUs
From what I checked in sklearn site, I couldn't find any documentation for that, therefore, I assume there's no official support. [https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.RFE.html]
If there are some pointers to either official support, or any academic/non-academic attempts, would appreciate the info.
SKLearn does not support GPU utilization for any of their functions. You can find more about that here: Will scikit-learn utilize GPU?
Related
I have started using Julia.I read that it is faster than C.
So far I have seen some libraries like KNET and Flux, but both are for Deep Learning.
also there is a command "Pycall" tu use Python inside Julia.
But I am interested in Machine Learning too. So I would like to use SVM, Random Forest, KNN, XGBoost, etc but in Julia.
Is there a native library written in Julia for Machine Learning?
Thank you
A lot of algorithms are just plain available using dedicated packages. Like BayesNets.jl
For "classical machine learning" MLJ.jl which is a pure Julia Machine Learning framework, it's written by the Alan Turing Institute with very active development.
For Neural Networks Flux.jl is the way to go in Julia. Also very active, GPU-ready and allow all the exotics combinations that exist in the Julia ecosystem like DiffEqFlux.jl a package that combines Flux.jl and DifferentialEquations.jl.
Just wait for Zygote.jl a source-to-source automatic differentiation package that will be some sort of backend for Flux.jl
Of course, if you're more confident with Python ML tools you still have TensorFlow.jl and ScikitLearn.jl, but OP asked for pure Julia packages and those are just Julia wrappers of Python packages.
Have a look at this kNN implementation and this for XGboost.
There are SVM implementations, but outdated an unmaintained (search for SVM .jl). But, really, think about other algorithms for much better prediction qualities and model construction performance. Have a look at the OLS (orthogonal least squares) and OFR (orthogonal forward regression) algorithm family. You will easily find detailed algorithm descriptions, easy to code in any suitable language. However, there is currently no Julia implementation I am aware of. I found only Matlab implementations and made my own java implementation, some years ago. I have plans to port it to julia, but that has currently no priority and may last some years. Meanwhile - why not coding by yourself? You won't find any other language making it easier to code a prototype and turn it into a highly efficient production algorithm running heavy load on a CUDA enabled GPGPU.
I recommend this quite new publication, to start with: Nonlinear identification using orthogonal forward regression with nested optimal regularization
There are two PyTorch repositories :
https://github.com/hughperkins/pytorch
https://github.com/pytorch/pytorch
The first clearly requires Torch and lua and is a wrapper, but the second doesn't make any reference to the Torch project except with its name.
How is it related to the Lua Torch?
Here a short comparison on pytorch and torch.
Torch:
A Tensor library like numpy, unlike numpy it has strong GPU support.
Lua is a wrapper for Torch (Yes! you need to have a good understanding of Lua), and for that you will need LuaRocks package manager.
PyTorch:
No need for the LuaRocks package manager, no need to write code in Lua. And because we are using Python, we can develop Deep Learning models with utmost flexibility. We can also exploit major Python packages likes scipy, numpy, matplotlib and Cython with PyTorch's own autograd.
There is a detailed discussion on this on pytorch forum. Adding to that both PyTorch and Torch use THNN. Torch provides lua wrappers to the THNN library while Pytorch provides Python wrappers for the same.
PyTorch's recurrent nets, weight sharing and memory usage with the flexibility of interfacing with C, and the current speed of Torch.
For more insights, have a look at this discussion session here.
Just to clarify the confusion between both pytorch repositories:
pytorch/pytorch is very similar to (Lua) Torch but in Python. So it's a wrapper over THNN. This was written by Facebook too.
hughperkins/pytorch: I have come across this repo when I was developing in Torch before pytorch existed, but I have never used it so I'm not quite sure if it is a wrapper written in Python over (Lua) Torch which is in turn a wrapper over THNN OR a wrapper over THNN and Lua. In both case, this is not the original version of Torch. It was written by Hugh Perkins when there was no Python alternative for Torch.
If you are wondering which one to go for, I would definitely recommend pytorch/pytorch as it communicates directly with THNN, is written by the people who made THNN and is continuously maintained. hughperkins/pytorch does not seem to be maintained anymore.
I am exploring some of the deep learning libraries including Chainer, Torch, TensorFlow, and Theano.
Formerly I was a user of Chainer, and I found that Theano or Tensorflow has a great flexibility and seems to have a nice future potential.
However, what keeps me from moving to Theano or Tensorflow is the memory issue. Is there an option to make Theano or Tensorflow do not keep computation history? In Chainer it can be done by setting volatile flag, so that I can evaluate large data with less memory because it does not keep unnecessary data (which are only necessary when calculating gradients).
I am primarily working with RNNs, and the typical approach to training RNNs is to use truncated BPTT. However I found that it is useful and has slightly more accurate to feed the full sequence to the network when I want just forward computations, not backpropagation.
I tried to find this option from documentions of both frameworks, but I couldn't find. Is there a reason that this feature cannot be implemented?
I'm interested in implementing a convolutional neural network in my C++ program where I'm tracking tagged insects (I'm also using OpenCV). I see people mention Caffe, Torch and Theano a lot but I haven't heard the CNN in the SHOGUN Toolbox discussed. Does this CNN work well and would anyone recommend it if you're working in C++? I've used Theano via scikit-neuralnetwork in Python to test out some images and that worked really well, except unfortunately Theano is Python-only.
Shogun also has GPU support of some of the operations used in the NN code. This is work in progress though. At this point in time, other libraries might be faster. We mostly built these networks in there in order to be able to easily compare them to the other algorithms in the toolbox.
The advantage, however, is that you can use it from a large number of languages (while internally, C++ code is executed) -- useful if you don't want to use python.
Here are some IPython notebooks that you could use as a basis to compare:
autoencoders for denoising and classification
(convolution) networks for digit classification
We appreciate any experience to be shared. Shogun is in constant development and especially the NNs attract a lot of people to work on them, so expect things to change. If you are interested in helping GPU-fying Shogun, please let us know.
The difference lies in the speed. cnn is computationally expensive, so a GPU implementation is at least 10 times faster than CPU. caffe and theano provide seamless integration of calling either CPU or GPU, which may not be easy for you to implement without much GPU programming experience.
Other factors may exist including a unified interface for multiplayer, stochastic gradient descent, and etc. but I think speed issue is most crucial among all these factors.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
I am looking for an open source neural network library. So far, I have looked at FANN, WEKA, and OpenNN. Are the others that I should look at? The criteria, of course, is documentation, examples, and ease of use.
Last update: 2020/03/24 (I will update this answer from time to time...)
Simple Implementations of Neural Networks
FANN is a very popular implementation in C/C++ and has bindings for many other languages.
In scikit-learn (Python) 0.18 (current developement version) there will be an implementation of feed-forward neural networks (API documentation).
And I must mention my own project, which is called OpenANN (Documentation). It is written in C++ and has Python bindings.
Deep Learning
Because neural networks are quite popular in research and industry at the moment ("deep learning") there are many research libraries available. Most of them are kind of easy to set up, integrate, and use. Although not as easy as the libraries mentioned above. They provide leading edge functionality and high performance (with GPUs etc.). Most of these libraries also have automatic differentiation. You can easily specify new architectures, loss functions etc. and don't have to specify the backpropagation manually.
TensorFlow from Google (C++/Python)
PyTorch from Facebook, in Python, can be extended with C/C++
mxnet (C++, Python, R, Scala, Julia, Matlab, Javascript)
Deeplearning4j (Java)
CNTK from Microsoft (training in Python / evaluation in C++/C#/Java/Python)
Chainer (Python)
PaddlePaddle from Baidu in CUDA/C++ with Python bindings
NNabla from Sony in Cuda/C++11 with Python bindings
A performance comparison for GPU-accelerated libraries can be found here (a bit outdated unfortunately). A comparison of GPUs and library versions can be found here.
Inactive:
Keras: It could use Tensorflow, Theano, and CNTK as a backend. (Now part of tensorflow as its high-level interface.)
Caffe from Berkeley Vision and Learning Center in C++ with Python bindings
Darknet: CNNs in C, known for the implementations of the YOLO object detector.
Neon from Intel Nervana provides very efficient implementations (Python)
MatConvNet (Matlab)
Theano (Python) and its high-level APIs:
Pylearn 2
Theanets
scikit-neuralnetwork
Lasagne
Blocks based on Theano (Python)
cuda-convnet2 in CUDA/C++ with Python bindings
Hebel (Python)
Caffe2 from Facebook in C++ with Python bindings; has been joined with PyTorch
Neural Networks for Torch 7 (Lua, Torch 7 is a "Matlab-like environment", overview of machine learning algorithms in Torch)
PyBrain (Python) contains different types of neural networks and training methods.
Encog (Java and C#)
If you want flexibility in defining network configurations, like sharing parameters or creating different types of convolutional architectures, then you should look at the family of Torch libraries: http://www.torch.ch/.
I haven't gone through the documentation for Torch 7 yet, but documentation for the other versions was pretty decent and the code is very readable (in Lua and C++).
You can use accord.net framework. http://accord-framework.net/
It contains Neural learning algorithms such as Levenberg-Marquardt, Parallel Resilient Backpropagation, the Nguyen-Widrow initialization algorithm, Deep Belief Networks and Restrictured Boltzmann Machines, and many other neural network related items.
Netlab is a commonly used Matlab library. (free and open source)
The Netlab toolbox is designed to provide the central tools necessary
for the simulation of theoretically well founded neural network
algorithms and related models for use in teaching, research and
applications development. It is extensively used in the MSc by
Research in the Mathematics of Complex Systems.
The Netlab library includes software implementations of a wide range
of data analysis techniques, many of which are not yet available in
standard neural network simulation packages. Netlab works with Matlab
version 5.0 and higher but only needs core Matlab (i.e. no other
toolboxes are required). It is not compatible with earlier versions of
Matlab.