I am working on building a model to predict user actions from log data. I have data collected about user actions in multiple sessions for over a year.
I am bit confused about the way to represent logs in form of dataset so as to apply Machine Learning algorithms. I went through couple of research papers where scientists used Bayesian approaches and HMM to represent states of user actions in the dataset. But the issue is that the papers don't go into implementation details.
So my question is - given a set of user actions over time in multiple sessions, how can one create a state transition diagram (or any other representation of user actions) to predict next user action.
Any help is appreciated!
Related
Most tutorials and RL courses focuses on teaching how to apply a model (e.g. Q-Learning) to an environment (gym environments) one can input a state in order to get some output / reward
How it is possible to use RL for historical data, where you cannot get new data? (for example, from a massive auction dataset, how can I derive the best policy using RL)
If your dataset is formed, for example, of time series, you can set each instant of time as your state. Then, you can make your agent to explore the data series for learning a policy over it.
If your dataset is already labeled with actions, you can train the agent over it for learning the a police underlying those actions.
The trick is to feed your agent with each successive instant of time, as if it were exploring it on real time.
Of course, you need to model the different states from the information in each instant of time.
I'm currently in the process of building a recommendation system with implicit data (e.g. clicks, views, purchases), however much of the research I've looked at seems to skip the step of "aggregating implicit data". For example, how do you aggregate multiple clicks and purchases overtime into a single user rating (as is required for a standard matrix factorization model)?
I've been experimenting with several Matrix Factorization based methods, including Neural Collaborative Filtering, Deep Factorization Machines, LightFM, and Variational Autoencoders for Collaborative Filtering. None of these papers seem to address the issue of aggregating implicit data. They also do not discuss how to weight different types of user events (e.g. clicks vs purchase) when calculating a score.
For now I've been using a confidence score approach (the conference score corresponds to the count of events) as outlined in this paper: http://yifanhu.net/PUB/cf.pdf. However this approach doesn't address incorporating other types of user events (other than clicks), nor does it address negative implicit feedback (e.g. a ton of impressions with zero clicks).
Anyway, I'd love some insight on this topic! Any thoughts at all would be hugely appreciated!
There's the method for building a recommendation system - Bayesian personalized ranking from implicit feedback. I also wrote an article on how it can be implemented using TensorFlow.
There's no "right" answer for the question of how to transfer implicit feedback explicitly. The answer will depend on business requirements. If the task is to increase the click rate, you should try to use the clicks. If the task of increasing conversion, you need to work with purchases.
Please help me to get the right answer of the below question, which is asked in one of the interview.
There is a bank, no of users visit bank for different- different services, but most of the users give bad rating and goes unsatisfied. What should bank do identify the reasons for the bad ratings. Bank capture data like, user info, agent info who deals with users, services offered, and no if things.
How to identify rules or reasons which are playing an important role in bad rating using machine learning techniques only.
If we build a classification model, that a user will be unsatisfied/satisfied. Then let say we get list of users who will be unsatisfied. Now what should we do with this data of unsatisfied users to help bank improve rating and business.
Train a classifier to predict dissatisfaction.
Don't use a neutral network, but something interpretable.
Then interpret the model for feature importance.
A good choice would be a random forest.
I hope your unsatisfied customer dataset has information about reason of dissatisfaction along with other attributes like user details,age,region and services they avail from bank.In this case there might be the case that one more more ML models and algorithm might need to be trained and optimised to get accurate picture.However critical factors to look for would be Services they are associated with,Reason of dissatisfaction and might be agent they are mapped to.
Above mentioned 3 key attributes can be executed with regression models to get prediction and improvement plan.Please note selection of ML models also depend upon detail analysis of dataset and asking and deriving right question from stakeholders.
For this question they may be looking for feature importance. Hence, interpretable algorithms are preferred. You may use logistic regression to find the top coefficients or a tree-based ensemble method such as random forests or gradient boosting. The top features will denote the features driving the model's decisions. Shap values are also something to look at.
Sources: An Introduction to Statistical Learning With Applications in R, AceAI Prep, Andrew Ng courses.
I have developed a ML model for a classification (0/1) NLP task and deployed it in production environment. The prediction of the model is displayed to users, and the users have the option to give a feedback (if the prediction was right/wrong).
How can I continuously incorporate this feedback in my model ? From a UX stand point you dont want a user to correct/teach the system more than twice/thrice for a specific input, system shld learn fast i.e. so the feedback shld be incorporated "fast". (Google priority inbox does this in a seamless way)
How does one build this "feedback loop" using which my system can improve ? I have searched a lot on net but could not find relevant material. any pointers will be of great help.
Pls dont say retrain the model from scratch by including new data points. Thats surely not how google and facebook build their smart systems
To further explain my question - think of google's spam detector or their priority inbox or their recent feature of "smart replies". Its a well known fact that they have the ability to learn / incorporate (fast) user feed.
All the while when it incorporates the user feedback fast (i.e. user has to teach the system correct output atmost 2-3 times per data point and the system start to give correct output for that data point) AND it also ensure it maintains old learnings and does not start to give wrong outputs on older data points (where it was giving right output earlier) while incorporating the learning from new data point.
I have not found any blog/literature/discussion w.r.t how to build such systems - An intelligent system that explains in detaieedback loop" in ML systems
Hope my question is little more clear now.
Update: Some related questions I found are:
Does the SVM in sklearn support incremental (online) learning?
https://datascience.stackexchange.com/questions/1073/libraries-for-online-machine-learning
http://mlwave.com/predicting-click-through-rates-with-online-machine-learning/
https://en.wikipedia.org/wiki/Concept_drift
Update: I still dont have a concrete answer but such a recipe does exists. Read the section "Learning from the feedback" in the following blog Machine Learning != Learning Machine. In this Jean talks about "adding a feedback ingestion loop to machine". Same in here, here, here4.
There could be couple of ways to do this:
1) You can incorporate the feedback that you get from the user to only train the last layer of your model, keeping the weights of all other layers intact. Intuitively, for example, in case of CNN this means you are extracting the features using your model but slightly adjusting the classifier to account for the peculiarities of your specific user.
2) Another way could be to have a global model ( which was trained on your large training set) and a simple logistic regression which is user specific. For final predictions, you can combine the results of the two predictions. See this paper by google on how they do it for their priority inbox.
Build a simple, light model(s) that can be updated per feedback. Online Machine learning gives a number of candidates for this
Most good online classifiers are linear. In which case we can have a couple of them and achieve non-linearity by combining them via a small shallow neural net
https://stats.stackexchange.com/questions/126546/nonlinear-dynamic-online-classification-looking-for-an-algorithm
i'm working on a project and i have a subset of user's key-stroke time data.This means that the user makes n attempts and i will use these recorded attempt time data in various kinds of classification algorithms for future user attempts to verify that the login process is done by the user or some another person. (Simply i can say that this is biometrics)
I have 3 different times of the user login attempt process, ofcourse this is subset of the infinite data.
until now it is an easy classification problem, i decided to use WEKA but as far as i understand i have to create some fake data to feed the classification algorithm.The user's measured attempts will be 1 and fake data will be 0.
can i use some optimization algorithms ? or is there any way to create this fake data to get min false positives ?
Thanks
There are a couple of different ways you could go about approaching this.
Collect Negative Examples - One easy solution would be to just gather keystroke timing data from other people that could be used as negative examples. If you want to gather a large sample very cheaply, as in about 1000 samples for about $10, you could use a service like Amazon Mechanical Turk.
That is, you could put together a human intelligence task (HIT) that has people type in randomized password like sequences. To get the timing information you'll need to use an External Question, since the restricted HTML for regular questions doesn't support JavaScript.
Use a Generative Model - Alternatively, you could train a generative probability model to a user's keystroke behavior. For example, you could train a Gaussian mixture model (GMM) to the user's delay between keystrokes.
Such a model will give you a probability estimate of keystroke timing information being generated by a specific user. You would then just need to set a threshold of how likely the timing information should be in order for the user to be authenticated.
Use 1-class SVMs - Finally, 1-class SVMs allows you to train a SVM like classifier using only positive examples. To learn one-class SVMs in WEKA, use the LibSVM wrapper if you're using v3.6. If you're using the bleeding edge developer version, there's weka.classifiers.meta.OneClassClassifier.