i am working on mahout recommendation engine use case.I precomputed recommendations and stored in database. now i am planning to expose with taste rest services to .net.i had limited customers and products.it is distributor level recommendation use case.my question is if new distributor comes in ,how would i suggests recommendations to him.and also how would i suggest the Quantity of Recommended product to each distributor.could you people give me some guidance.am i going to face performance issues..?
One way is, when new user comes, to precompute the recommendations from scratch for all the users or only for this user. You should know that this user might change the recommendations for the others too. Its up to your needs frequently do you want to do the pre-computations.
However, if you have limited number of users and items, another way is to have online recommender that computes the recommendations in real time. If you use the FileDataModel, there is a way to get the data from the new user periodically (See the book Mahout in Action). If you use in memory data model, which is faster, you can override the methods: setPreference(long userID, long itemID, float value) and removePreference(long userID, long itemID), and whenever new user comes and likes or removes some items you should call these methods on your data model.
EDIT: Basically you can get the GenericDataModel, and add this to the methods setPreference and removePreference. This will be your lower level data model. You can wrap it afterwards with ReloadFromJDBCDataModel by setting your data model in the reload() method like this:
DataModel newDelegateInMemory =
delegate.hasPreferenceValues()
? new MutableDataModel(delegate.exportWithPrefs())
: new MutableBooleanPrefDataModel(delegate.exportWithIDsOnly());
The overridden methods:
#Override
public void setPreference(long userID, long itemID, float value) {
userIDs.add(userID);
itemIDs.add(itemID);
setMinPreference(Math.min(getMinPreference(), value));
setMaxPreference(Math.max(getMaxPreference(), value));
Preference p = new GenericPreference(userID, itemID, value);
// User preferences
GenericUserPreferenceArray newUPref;
int existingPosition = -1;
if (preferenceFromUsers.containsKey(userID)) {
PreferenceArray oldPref = preferenceFromUsers.get(userID);
newUPref = new GenericUserPreferenceArray(oldPref.length() + 1);
for (int i = 0; i < oldPref.length(); i++) {
//If the item does not exist in the liked user items, add it!
if(oldPref.get(i).getItemID()!=itemID){
newUPref.set(i, oldPref.get(i));
}else{
//Otherwise remember the position
existingPosition = i;
}
}
if(existingPosition>-1){
//And change the preference value
oldPref.set(existingPosition, p);
}else{
newUPref.set(oldPref.length(), p);
}
} else {
newUPref = new GenericUserPreferenceArray(1);
newUPref.set(0, p);
}
if(existingPosition == -1){
preferenceFromUsers.put(userID, newUPref);
}
// Item preferences
GenericItemPreferenceArray newIPref;
existingPosition = -1;
if (preferenceForItems.containsKey(itemID)) {
PreferenceArray oldPref = preferenceForItems.get(itemID);
newIPref = new GenericItemPreferenceArray(oldPref.length() + 1);
for (int i = 0; i < oldPref.length(); i++) {
if(oldPref.get(i).getUserID()!=userID){
newIPref.set(i, oldPref.get(i));
}else{
existingPosition = i;
}
}
if(existingPosition>-1){
oldPref.set(existingPosition, p);
}else{
newIPref.set(oldPref.length(), p);
}
} else {
newIPref = new GenericItemPreferenceArray(1);
newIPref.set(0, p);
}
if(existingPosition == -1){
preferenceForItems.put(itemID, newIPref);
}
}
#Override
public void removePreference(long userID, long itemID) {
// User preferences
if (preferenceFromUsers.containsKey(userID)) {
List<Preference> newPu = new ArrayList<Preference>();
for (Preference p : preferenceFromUsers.get(userID)) {
if(p.getItemID()!=itemID){
newPu.add(p);
}
}
preferenceFromUsers.remove(userID);
preferenceFromUsers.put(userID, new GenericUserPreferenceArray(newPu));
}
if(preferenceFromUsers.get(userID).length()==0){
preferenceFromUsers.remove(userID);
userIDs.remove(userID);
}
if (preferenceForItems.containsKey(itemID)) {
List<Preference> newPi = new ArrayList<Preference>();
for (Preference p : preferenceForItems.get(itemID)) {
if(p.getUserID() != userID){
newPi.add(p);
}
}
preferenceForItems.remove(itemID);
preferenceForItems.put(itemID, new GenericItemPreferenceArray(newPi));
}
if(preferenceForItems.get(itemID).length()==0){
//Not sure if this is needed, but it works without removing the item
//preferenceForItems.remove(itemID);
//itemIDs.remove(itemID);
}
}
If by "new distributor" you mean that you have no data for them, no historical data. Then you cannot make recommendations using Mahout's recommenders.
You can suggest other items once they chose one. Use Mahout's "itemsimilarity" driver to calculate similar items for everything in your catalog. Then if they choose something you can suggest similar items.
The items that come from the itemsimilarity driver can be stored in you DB as a column value containing ids for similar items for every item. Then you can index the column with a search engine and use the user's first order as the query. This will return realtime personalized recommendations and is the most up-to-date method suggested by the Mahout people.
See a description of how to do this in this book by Ted Dunning, one of the leading Mahout Data Scientists. http://www.mapr.com/practical-machine-learning
Related
I Want to clear one session Data but it's clearing both Sessions data. below is my 2 MVC action methods which are used to clear Session data.
1)
public int Clearsesson()
{
int i = 0;
List<Wish_Product> products = new List<Wish_Product>();
if (**Session["WishItem"]** != null)
{
Session.Clear();
i = 1;
}
else
{
i = 0;
}
return i;
}
2)
public int Clearsesson2()
{
int i = 0;
List<Cart_Product> products = new List<Cart_Product>();
if (**Session["CartItem"]** != null)
{
// Session.Remove("CartItem");
Session.Clear();
i = 1;
}
else
{
i = 0;
}
return i;
}
So like this I am storing Data in two Sessions but when I run this asp.net MVC Action method to clear One session data, it automatically clears second one Session Data too.
Why so like this? Please make it clear to me why it's happening.
Or what I've to do to solve this problem?
The short answer is yes, on the current collection.
According to ASP.NET documentation, HttpSessionState.Clear method removes all keys and values from currently active session state collection (it doesn't destroy user's session state base), so that both Session["WishItem"] & Session["CartItem"] values are cleared together from collection when the method is executed, but the session state base kept alive.
There are 2 different session collection keys in this case & actually both of them using same session state. To clear single session data, there are 2 different ways:
1) Remove session key using Session.Remove.
Session.Remove("WishItem");
2) Assign null value to the session state based with its key.
Session["WishItem"] = null;
The difference between them is that the former deletes both keys and values so that it won't appear as key-value pair in session collection anymore, and the latter overwrites current collection value with specified key but the key name still unchanged.
Therefore, you can remove each session data separately without clearing another session key like this:
public int ClearSession()
{
int i = 0;
// other code logic
if (Session["WishItem"] != null)
{
// only WishItem key has cleared
Session.Remove("WishItem");
i = 1;
}
// other code logic
}
public int ClearSession2()
{
int i = 0;
// other code logic
if (Session["CartItem"] != null)
{
// only CartItem key has cleared
Session.Remove("CartItem");
i = 1;
}
// other code logic
}
Note that each authenticated user always be assigned with one session state base, hence using Session.Clear removes all existing key-value pairs stored in session collection for current user (doesn't affect other user's session state).
The project I am working is 'University Management System' and it's a big one. Right now, I am implementing the student registration section that works fine (A small portion of the project). I've used 'Three-Tier Architecture' and 'ORM - EF' in ASP.NET MVC template. In the project, I need to do some validations for registering students depending upon their year, department etc. So there are sections like DAL, BLL, finally controller and view. I've done the validations in the controller and getting the data from BLL that again retrieves data from DAL (This is the simple condition of 'Three-Tier Architecture'). So my questions are:
1) Is it OK to do the validations in the controller?
2) If not and need to do it in the BLL, will it be just fine and why or I can
continue doing it in the controller?
Note: To me, doing the validations in the controller or BLL seems OK and the same. Does it have any effect?
Right now, I've done the following:
DAL:
public List<Student> Add(int studentID, string studentName, string email, DateTime regDate)
{
List<Student> lst = null;
Student aStudent = new Student();
aStudent.StudentID = studentID;
aStudent.StudentName = studentName;
aStudent.Email = email;
aStudent.RegDate = regDate;
try
{
db.Students.Add(aStudent);
db.SaveChanges();
}
catch (Exception ex)
{
ex.ToString();
}
return lst;
}
BLL:
public List<Student> Add(int studentID, string studentName, string email, DateTime regDate)
{
return aStudentGateway.Add(studentID, studentName, email, regDate);
}
Controller:
/**Student Registration - Starts**/
[HttpPost]
public ActionResult AddStudent(Student aStudent)
{
List<Department> departments = aDepartmentManager.GetAllDepartments();
List<DepartmentViewModel> departmentsViewModel = aDepartmentManager.GetAllDepartmentViewModel();
DateTime yearInDateTime = Convert.ToDateTime(Request.Form["RegDate"]);
string extractYear = yearInDateTime.ToString();
var year = DateTime.Parse(extractYear).Year;
int department = Convert.ToInt32(Request.Form["Department"]);
List<Student> studentList = aStudentManager.GetAllStudents();
int count = 1;
var query = (from c in studentList
where c.Department == department && c.Year == year
select c).ToList();
foreach (var c in query)
{
if (query.Count() > 0)
{
int m = Convert.ToInt32(c.StudentID);
count = m + 1; //Incrementing the numbers by one with the table column
}
else
{
int m = 1;
count = m + 1; //Incrementing the numbers by one with the variable assigned one
}
}
Student student = new Student();
student.StudentName = Request.Form["StudentName"];
student.Email = Request.Form["Email"];
student.RegDate = Convert.ToDateTime(Request.Form["RegDate"]);
student.StudentID = count;
if (aStudentManager.ExistEmailAny(student.Email))
{
ViewBag.ErrorMessage = "Email already exists";
}
else
{
aStudentManager.Add(aStudent.StudentID, aStudent.StudentName, aStudent.Email, aStudent.RegDate);
ViewBag.Message = "Registration successful. See below to verify.";
/**This section used to show student details after registration**/
var result = (from c in departments
join d in departmentsViewModel on c.DepartmentID equals d.DepartmentId
where d.DepartmentId == department
select c);
foreach (var items in result)
{
if (count.ToString().Length > 1)
{
ViewBag.StudentID = items.Code + "-" + year + "-" + "0" + count;
}
else
{
ViewBag.StudentID = items.Code + "-" + year + "-" + "00" + count;
}
StudentViewModel.StudentID = student.StudentID;
StudentViewModel.StudentName = student.StudentName;
StudentViewModel.Email = student.Email;
StudentViewModel.RegDate = student.RegDate;
}
/**This section used to show student details after registration**/
}
return View();
}
/**Student Registration - Ends**/
I would provide multiple steps of validation in the different layers, depending on the context and the meaning of the layer.
First, it's a best practice to provide validation both on client and server side.
For the client side you should provide field checks for required fields and other simple validations. If you are using MVC you can use data annotations.
The same validation should be replicated in the controller. Here you should fail fast applying some kind of contract to the parameters that have been passed. One good practice is using Code Contracts that provide preconditions that need to be satisfied to go on in your pipeline of execution.
In the business layer provide the check that needs to be done in the business logic.
Finally in the data access layer provide all the checks that are needed to persist your data. If you are using EF a good practice is implementing the IValidatableObject for your entity classes. Here in Scott Gu's blog you can find a post that explains this technique.
Even though this approach look like it will introduce repetitions, it will provide consistency in your data and separate concerns between your layers.
1) Is it OK to do the validations in the controller?
No at all, it would be more better to use Data Annotation Validator Attributes, and to do validation in your model class.
Second thing, you're doing some stuff of DAL in your controller, like
List<Department> departments = aDepartmentManager.GetAllDepartments();
List<DepartmentViewModel> departmentsViewModel = aDepartmentManager.GetAllDepartmentViewModel();
var query = (from c in studentList
where c.Department == department && c.Year == year
select c).ToList();
These all queries should be in DAL, which is exact use of DAL to interact with the database, and keep your controller clean.
Third thing,
If you pass Student to the controller, then not need to get each attribute using Request.Form.
Hope this make sense!
I've got:
my-app
community-list
On attached, my-app gets the user and loads the app.user. In the meantime, community-list is attached (even before app.user is loaded) and so I haven't been able to get the user's starred communities yet. Therefore, the solution I'm working on is as follows.
In community-list.attached():
app.changes.listen((List<ChangeRecord> records) {
if (app.user != null) {
getUserStarredCommunities();
}
});
Elsewhere in community-list is said metho:
// This is triggered by an app.changes.listen.
void getUserStarredCommunities() {
// Determine if this user has starred the community.
communities.forEach((community) {
var starredCommunityRef = new db.Firebase(firebaseLocation + '/users/' + app.user.username + '/communities/' + community['id']);
starredCommunityRef.onValue.listen((e) {
if (e.snapshot.val() == null) {
community['userStarred'] = false;
} else {
community['userStarred'] = true;
}
});
});
}
Note that communities is an observable list in community-list:
#observable List communities = toObservable([]);
Which is initially populated in community-list.attached():
getCommunities() {
var f = new db.Firebase(firebaseLocation + '/communities');
var communityRef = f.limit(20);
communityRef.onChildAdded.listen((e) {
var community = e.snapshot.val();
// If no updated date, use the created date.
if (community['updatedDate'] == null) {
community['updatedDate'] = DateTime.parse(community['createdDate']);
}
// snapshot.name is Firebase's ID, i.e. "the name of the Firebase location"
// So we'll add that to our local item list.
community['id'] = e.snapshot.name();
// Insert each new community into the list.
communities.add(community);
// Sort the list by the item's updatedDate, then reverse it.
communities.sort((m1, m2) => m1["updatedDate"].compareTo(m2["updatedDate"]));
communities = communities.reversed.toList();
});
}
In summary, I load the list of communities even before I have a user, but once I have a user I want to update each community (Map) in the list of communities with the userStarred = true/false, which I then use in my community-list template.
Alas, it doesn't seem like the List updates. How do I achieve this?
This whole app.changes.listen business is expensive. What's the proper practice in a case like this, where an element is loaded before I load objects (like app.user) that will modify it in some way.
1)
toList() creates a copy of the list. You need to apply toObservable again to get an observable list.
communities = toObservable(communities.reversed.toList());
This also assigns a new list to communities which is covered by #observable.
I think it should trigger anyway
2) You update your communities explicitly. It shouldn't be necessary to listen for changes. You can call a method containing
if (app.user != null) {
getUserStarredCommunities();
}
explicitly each time you change the list.
You also call Firebase for each community when a change in communities occurs. I don't know Firebase but it seems you send a request to a server each time which is of course expensive.
You should remember for what user+community combination you already made the call and use the remembered result instead.
With app.changes.listen you listen to any updated of any #observable field in your component. If you have other observable fields beside communities this method might be called too often.
If you are only interested in changes to communities you should put this code into a method like
communitiesChanged(oldVal, newVal) {
if (app.user != null) {
getUserStarredCommunities();
}
}
but the better option is to not listen to changes and another method name and call it explicitly as state above anyways if possible.
I am getting rss feed from a another website and in case that website shut down, I don't want to get an error. So in order to avoid from error I am trying to save rss feeds to database and if the rss server shuts down I will be able to get feed from my database.
I also want to keep only 6 feed in database. If new feed comes I want to delete the last feed by PublishDate
public static List<Rss.News> GetRssFeed(ApplicationDbContext db)
{
try
{
XDocument feedXml = XDocument.Load("http://www.gib.gov.tr/
rss/haberguncel.php");
var feeds = from feed in feedXml.Descendants("item")
select new Rss.News
{
Title = feed.Element("title").Value,
Link = feed.Element("link").Value,
Description = feed.Element("description").Value,
PublishDate=feed.Element("pubdate").Value
};
int counter = 0;
var itemE = db.News.FirstOrDefault();
if (itemE != feeds.First())
{
foreach (var itemC in feeds)
{
if (!db.News.Contains(itemC))
{
db.News.Add(itemC);
db.SaveChanges();
counter += 1;
}
else
{
break;
}
if (counter == 6) { break; }
}
}
return feeds.ToList();
}
catch (Exception)
{
// i will get data from database here.
}
}
I am getting this error when I run this code:
Unable to create a constant value of type
'...Models.Rss+News'. Only primitive types or enumeration
types are supported in this context.
The error is resulting from this line:
if (!db.News.Contains(itemC))
In order to do this type of evaluation at the database level, Entity Framework must be able to convert item being compared (itemC) into a constant value, which it cannot do with this type, hence your error. You can try casting db.News to a list, first, which would switch the evaluation over to in-memory instead at the the database, i.e.:
var news = db.News.ToList();
if (news.Contains(itemC))
However, you'll have to evaluate how that might affect your application's performance. Alternatively, you simply query on a particular value that you determine as the "key" for lookup. For example, you might say that Link will only ever match if it's the same item, so based on that:
if (!db.News.Any(m => m.Link == itemC.Link))
How can DB4o users get back the data for only one object?
(This is akin to getting the data for only one row of a traditional relational database table.)
With DB4o, I only know how to get the data back for a class of objects but not simply one unique object instance.
just query objects and get first item out of the result (the same like in relational database)
to get it by Guid ID:
using (IObjectContainer session = this.GetNewSession())
{
Dummy result = (from Dummy item in session
where item.Id == Guid.Parse("....")
select item).FirstOrDefault()
}
the result will be either null if item doesn't exist or the object found
other option is to get it directly by internal ID such as (or even UUID):
long id = ....;
using (IObjectContainer session = this.GetNewSession())
{
Dummy result = (Dummy)session.Ext().GetByID(id);
}
I have answered my own question (I believe):
Solution #1:
public List<Object> getListOfObjects(final Object o){
List<Object> result = db.query(new Predicate<Object>(){
#Override
public boolean match (Object arg0){
if(arg0.equals(o)){
return true;
}
else{
return false;
}
});
return result;
}
Solution #2:
public ObjectSet<Class<?>> getListOfObjects(Object o){
Query q = db.query();
q.constrain(o);
ObjectSet<Class<?>> set = q.execute();
return set;
}
Maybe someone knows if one of these solutions is better than the other, or whatever.