i want to run fp-growth algorithm of Mahout. i run this algoritm on Hadoop or in Eclips environment? Which is the best and easiest way? and How do I run? Please give me step by step what to do. Thank you
Related
Im new to machine learning, and I want to develop an application that takes all the data from multiple user's bash history, and predict the next command of another user based on other's executed commands.
I searched for it a lot but didnt find any good answer. Appreciate the ML expert's help if know about sample of similar code, or have any comments that might be useful such as what algorithm.etc. should I look into.
You can check Language Modeling topic, which is able to predict the next word in the sequence given the words that precede it. You probably work with RNN or LSTM based networks for Language Modeling.
I am currently building a MINLP model which has around 200k decision variables and upto 100 constraints. I have access to only open source solvers which are BONMIN and COUENNE.
When I try to solve the problem, I see that the solver keeps on running for more than 2 hours.
I have been reading the BONMIN documentation and there I see various heuristic algorithms as options. I wanted to know is there any options list i can pass to this BONMIN solver which will trigger a heuristic algorithm that will give me a suboptimal solution in ~15 minutes?
I am working with the Pyomo package.
Thanks in Advance!
See this section of the Pyomo documentation on sending options to a solver: https://pyomo.readthedocs.io/en/latest/working_models.html#sending-options-to-the-solver
Recently, we planned to build a system for image processing to extract info from images. At present we are using AWS Rekognition to do that. But, in some cases, we are not getting accurate information from AWS. So, we've planned to build our own custom one.
We've 4/5 months to do that. At least a POC version. Also, we've planned to use Tensorflow for that. We all have no prior experience about Machine Learning & Deep Learning but already have 5/6yrs of experience on Computer Programming by using different languages.
Currently, I'm studying ML from a course of Udemy & my approach to solve this problem is...
Learn Machine Learning(ML)
Learn Deep Learning(DL)
Above ML & DL maybe I'll be ready to understand the whole thing & can able to build a system for Image Processing.
In abstract what I've understood is, I've to write one Deep Learning program in Python by using Tensorflow. By using that Program I've to build a Model. Then I've to train that Model by using some training data. Then, when my Model achieves a certain level of accuracy I'll use some test data.
Now, there some places at where I've bit confused & here are my questions regarding that confusion...
I know tensorflow is a library but at some places, it's also mentioned as a system. So, is it really a library(piece of code) only & something more than that?
I got some Image Processing Python code in Tensorflow tutorial section (https://www.tensorflow.org/tutorials/image_recognition). We've tested that code & it's working exactly the way AWS Recognition service work. So, here my doubt is... can I use this Python code as it is in our production work?
After train a model with some training data does those training data get part of the whole system or Machine Learning Model extract some META info from those training data & keep with itself rather whole raw training data(in my case it'll be raw images).
Can I do all these ML+DL programmings over my Linux System? It has Pentium 4 with 8GB RAM.
Also, want to know... the approach which I've mentioned to build a solution for my problem is sufficient or I need to do something else also.
Need some guidance to clear out all these confusion.
Thanks
1 : tensor-flow is like anything else we have been worked with (like Numpy ) but only difference is we have to first defined what we want to use the use it , every thing in tensor-flow are running into a computational graph and evaluating every thing in that graph require a Session , we could call it library because it just piece of code and have interface in python , and system because of all those mechanism it uses
2 :
can I use this Python code as it is in our production work? Why not !
3:
yes you could do that with your system , but the main advantage of tensor-flow and theano , .. the tool like those is that you could run your code on GPU it a more faster way than on CPU because the GPU could handle a lot more matrix multiplication and stuff like that
4:
you know you don't have to learn all the machine learning stuff to built a image recognition system , it may be take years for you to understand whats going on there , Udemy course is very good source but you I highly recommend you to see the machine learning courses of coursera , there is to courses there about machine learning : the great Andrew NG course and Emily fox course , the first one is more theoretical than practical , but second on is more practical ,
and about the Deep learning , there is nothing fancy about Deep learning and it's just a method in machine learning , after you gain some experience in machine learning and understood some basic or you could do it right know , go to fast.ai , it has a really good course about deep learning for coder and it's also free
I hope this will help you
I am trying to Code a genetic algorithm in Matlab but really dont know how it works in images and how to proceed? Is there any basic tutorial that can help me understand how to apply GA on images (starting from 2d to multidimentional images ).
That will be a great help for me.
Thanking everyone in anticipations.
Kind Regards.
For GA you need two things: a fitness function that can evaluate any solution and tell how good it is, and a representation of your solution so that you can do crossover and mutation. Once you have these, you are good to go. I'm not an expert on image processing so I can't help you with that exactly.
Look at the book Essentials of metaheuristics which is a very good resource for start with evolutionary computation (and not only that) in general. It's free.
There is a paper on this subject which you can find at the IEEE library. I believe it solves the problem you vaguely describe.
For a novice to machine learning, what are the learning prerequisites to using Apache Mahout in an efficient way?
I know that a committer to Mahout would need calculus, linear algebra, probability and machine learning before they can contribute anything useful. But does a "User" of Apache Mahout need all of this?
I'm asking this because learning/revising all of the above would take me ages..
Mahout In Action provides a good overview of what you need to know to use Mahout.
Typically, scalable machine learning does not require advanced mathematics for use. It may require serious math to develop, but not necessarily to use.
The primary requirement is that you really understand your data and its origins and what you want to do with it. That understanding doesn't have to come all at once and can be developed over time.
Try to Google the topics below:
Programming Collaborative Intelligence
Similarity calculation with vectors
What's the different between cluster and classification.