Speeding up a linq query - asp.net-mvc

I am trying to replicate the following SQL query with linq. On SQL Server it takes a fraction of a second to run:
select g.reference, count(*)
from isis.dbo.[group] as g inner join
isis.dbo.enrolment as e on g.groupid = e.groupid inner join
isis.dbo.student as s on e.studentid = s.studentid inner join
isis.dbo.progression as p on s.studentid = p.studentid
where p.academicyear = '12/13' and g.istutorgroup = 1
group by reference
In my MVC application I am passing a listing of "TutorGroups" to the view. For each tutor group in the view I need to display various information about them, one item being the number of "Progression" interviews they have had.
I have tried a couple of methods but they both take upwards of 30 secs to run in my MVC application:
<%TTMrequired = tg.Enrolments
.SelectMany(e => e.Student.Progressions
.Where(p => p.TTMChecked == false &&
p.TTMInterview == true &&
p.AcademicYear == year))
.Count(); %>
and
<%TTMrequired = tg.Enrolments
.Where(e => e.Student.Progressions
.Any(p => p.TTMChecked == false &&
p.TTMInterview == true &&
p.AcademicYear == year))
.Count(); %>
Anyone got any suggestions on how I can speed this up? I suspect the problem is me trying to do it a stupid way - it usually is!

You could try doing a Sum of counts instead of a SelectMany:
tg.Enrolments.Sum(e => e.Student.Progressions
.Count(p => p.TTMChecked == false &&
p.TTMInterview == true &&
p.AcademicYear == year)
);

Should be written using this syntax:
var TTMRequired = (from g in tg.Groups
join e in tg.Enrolment on g.groupid equals e.groupid
join s in tg.Students on e.studentid equals s.studentid
join p in tg.Progressions on s.studentid = p.studentid
where p.academicyear.Equals("12/13") && g.istutorgroup.Equals(1)
group g by g.reference into grp
select new {
grpRef = grp.Key,
grpCount = grp.Count()
});
Note: if g.istutorgroup is of type BIT instead of INT, consider using .Equals(true).

Related

Converting Sql to Linq (to Entities)

I have a query like that and it works normally. If a student gets more than one course, it list all of them:
SELECT ks.KullaniciKodu as username, ks.Sifre as password, k.adi as firstname, k.soyadi as lastname, k.Email as email,
MAX(CASE WHEN c.DERSKODU = 'ENF100' THEN 'ENF100' ELSE '' END)As course1,
MAX(CASE WHEN c.DERSKODU = 'ATA101' THEN 'ATA101' ELSE '' END) As course2,
MAX(CASE WHEN c.DERSKODU = 'TDB101' THEN 'TDB101' ELSE '' END) As course3,
MAX(CASE WHEN c.DERSKODU = 'İNG101' THEN 'İNG101' ELSE '' END) As course4
FROM Kayit k
JOIN DersNotu dn ON dn.KayitNo = k.KayitNo
JOIN Ders c ON c.DersKayitNo = dn.DersKayitNo AND c.DERSKODU IN ('ENF100','ATA101','TDB101', 'İNG101') AND c.DONEM = '201512'
JOIN KullaniciSifre ks ON ks.KullaniciKodu = k.KullaniciKodu
GROUP BY ks.KullaniciKodu, ks.Sifre, k.adi, k.soyadi, k.Email
But when i convert it to Linq (to Entity), it only gets one course or all courses are null. Where is the difference and problem in this code?
from k in db.Kayit
join dn in db.DersNotu on k.KayitNo equals dn.KayitNo
join c in db.Ders on dn.DersKayitNo equals c.DersKayitNo
join ks in db.KullaniciSifre on k.KullaniciKodu equals ks.KullaniciKodu
where
(new string[] { "ENF100", "ATA101", "TDB101", "İNG101" }).Contains(c.DersKodu) &&
c.Donem == 201512
group new { ks, k, c } by new
{
ks.KullaniciKodu,
ks.Sifre,
k.Adi,
k.Soyadi,
k.Email
} into g
select new
{
KullaniciKodu = g.Key.KullaniciKodu,
Sifre = g.Key.Sifre,
Adi = g.Key.Adi,
Soyadi = g.Key.Soyadi,
Email = g.Max(p => (
p.k.Email == string.Empty ? "NULL" : p.k.Email)),
Course1 = g.Max(p => (
p.c.DersKodu == "ENF100" ? "ENF100,1" : "NULL,1")),
Course2 = g.Max(p => (
p.c.DersKodu == "ATA101" ? "ATA101,1" : "NULL,1")),
Course3 = g.Max(p => (
p.c.DersKodu == "TDB101" ? "TDB101,1" : "NULL,1")),
Course4 = g.Max(p => (
p.c.DersKodu == "İNG101" ? "İNG101,1" : "NULL,1"))
}
Using MAX in SQL is just a trick to work-around the problem. You should not convert that explicitly to LINQToEntity. The problem here is you use Max against strings. In your case the NULL,1 seems be always selected as max except for the TDB101,1.
The actual logic here is such as for ENF100, if any one found, project for "ENF100", otherwise project for "NULL,1". The same logic for others. So it should be like this:
Course1 = g.Any(p => p.c.DersKodu == "ENF100") ? "ENF100,1" : "NULL,1",
Course2 = g.Any(p => p.c.DersKodu == "ATA101") ? "ATA101,1" : "NULL,1",
Course3 = g.Any(p => p.c.DersKodu == "TDB101") ? "TDB101,1" : "NULL,1",
Course4 = g.Any(p => p.c.DersKodu == "İNG101") ? "İNG101,1" : "NULL,1"
Also the Email is projected wrong, it is already included in the group key, so you can just project it via the group key:
Email = g.Key.Email

Lambda not in like SQL

I’m working with MVC 5 and I’m quite new on it. How can convert the below sql code to lambda expression. Basically what I’m trying to do is to show all records that ProductID not exist in Scrap Table
Select * from Product
where ProductID not in (Select ProductID from Scrap where ref = '123')
and active = 1
I believe your query is equivalent to:
select p.*
from Product p
join Scrap s on p.ProductID = s.ProductID
where
s.ref <> '123'
and p.active = 1
If so, try this:
from p in db.Products
join s in db.Scraps on p.ProductID equals s.ProductID
where s.Ref != "123" && p.Active == 1
select p
Try
var query =
from p in db.Products
where !(from s in db.Scrap
where s.ref == '123'
select s.ProductId)
.Contains(p.ProductId)
&& p.Active = 1
Try something like:
var products = (from p in db.Products //db is an instance of my datacontext
where !db.Scrap.Any(s => s.ProductId == p.ProductId && s.ref == "123")
&& p.active == 1 // p.active == true if active is of type bit in sql
select p);

Rails: Search for person with language skills - e,g, speaks "German AND English" on one-to-many table

This must be a basic thing in rails, but I don't know how to do it.
I would like to filter participants based on the languages they speak. People can speak multiple languages, and languages are stored in their own table with a one-to-many relationship.
Now my search looks really clunky and doesn't seem to work:
if #cvsearch.language.present? == true and #cvsearch.language != 0
#p = #p.joins(:languages).where('languages.name = ?', #cvsearch.language)
else
#cvsearch.language = 0
end
if #cvsearch.language1.present? == true and #cvsearch.language1 != 0
#p = #p.joins(:languages).where('languages.name = ?', #cvsearch.language1)
end
if #cvsearch.language2.present? == true and #cvsearch.language2 != 0
#p = #p.joins(:languages).where('languages.name = ?', #cvsearch.language2)
end
if #cvsearch.language3.present? == true and #cvsearch.language3 != 0
#p = #p.joins(:languages).where('languages.name = ?', #cvsearch.language3)
end
The resulting SQL, slightly shortened:
SELECT COUNT(*) FROM "participants" INNER JOIN "languages" ON "languages"."participant_id" = "participants"."id" WHERE (participants.id >= 2) AND (languages.name = 11) AND (languages.name = 10)[0m
It would be great to get a specific solution, but even better is a pointer as to where I can read up on this - what's the key word I am missing to describe this problem?
So this is the solution I am using for now:
if #cvsearch.language1.present? == true and #cvsearch.language1 != 0
safe_lang = ActiveRecord::Base::sanitize(#cvsearch.language1)
qry = "INNER JOIN languages l1 ON l1.participant_id = participants.id AND l1.name = " + safe_lang.to_s
#p = #p.joins(qry)
end
Works wonderfully, just need to get some feedback regarding the safety of this approach.
I'm not sure of a general reference to refer you to, but this is basic SQL stuff. Basically, the JOIN is performed first resulting in a number of rows and then the WHERE is applied, filtering the rows. The conceptual mistake here is thinking that the WHERE clause will somehow apply to the full set of matched languages, but it doesn't work that way, each row of the result is considered in isolation, therefore a clause like (languages.name = 11) AND (languages.name = 10) will never return anything, because languages.name only has a single value in each row. The query as constructed could only work for an OR clause, so you could say something like WHERE (languages.name = 11) OR (languages.name = 12).
In order to filter down the participants you need one join for each language, so you want something like this:
SELECT COUNT(*) FROM participants
INNER JOIN languages l1 ON l1.participant_id = participants.id AND (languages.name = 10)
INNER JOIN languages l2 ON l2.participant_id = participants.id AND (languages.name = 11)
WHERE participants.id >= 2
Offhand I'm not sure of the easiest way to do this in ActiveRecord, it's not a super common query. Your general structure should work, but with something like:
if #cvsearch.language1.present? == true and #cvsearch.language1 != 0
safe_language = ActiveRecord::Base.sanitize(#cvssearch.language1)
join_clause = "INNER JOIN languages l1 ON l1.participant_id = participants.id AND language.name = #{safe_language}"
#p = #p.joins(join_clause)
end

sql distinct statement and use of more than one table on rails

How would you write this line in the "rails way"?
unique_attendees = CourseSessionsUser.count_by_sql(["SELECT count(DISTINCT csu.user_id) AS count_user_id FROM course_sessions_users csu, course_sessions cs WHERE cs.course_id = ? AND csu.course_session_id = cs.id"], #course.id)
The query itself is:
SELECT count(DISTINCT csu.user_id) AS count_user_id
FROM course_sessions_users csu,
course_sessions cs
WHERE cs.course_id = ?
AND csu.course_session_id = cs.id
Use count method of the rails
count = CourseSessionsUser.count('csu.user_id',
:distinct => true, :conditions => ["cs.course_id = ?", #course.id]
:joins => "cs LEFT JOIN course_sessions_users csu ON cs.id = csu.course_session_id")
This will return directly non zero integer if condition matches otherwise return zero

Return Original datatype after LINQ group by

I need my LINQ Query to return the Product Datatype, after being grouped. It seems to be encased into an anonymous psuedo family.
I have some properties in Product that I don't care about, just needing the p.ID and p.Name etc.
The error I'm getting at the moment with this is:
The entity or complex type 'DatabaseModel.Product' cannot be constructed in a LINQ to Entities query.
This is my Method:
public static List<Product> GetSellableSpecialOffers(DatabaseEntities db)
{
var query = from p in db.Products
where (p.Visible == true && p.Active == true)
&& p.ShowInSpecialOffers == true
group p by p.FamilyID == null ? 0 : p.FamilyID into f
select new Product {
ID = f.First().ID,
Name = f.First().Name,
Cost = f.First().Cost,
RRP = f.First().RRP
};
return query.ToList();
}
What is the problem? Is there a better way around this? SQL would always return 1 record instead of encasing the object in a secondary datatype, I don't get it.
Many thanks,
EDIT 1:
My apologies for extending the specification, but I need the returned product to be programatically generated e.g.
select new Product {
ID = f.First().ID,
Name = f.First().Name,
Cost = f.OrderBy(p => p.NowCost).FirstOrDefault(),
RRP = f.First().RRP
}
or if I could strongly type the family class:
public partial class ProductFamily
{
public Product GroupedProduct
{
get
{
return this.Products.OrderBy(p => p.NowCost).FirstOrDefault();
}
}
}
Then I would do:
var query = (from p in db.Products
where (p.Visible == true && p.Active == true)
&& p.ShowInSpecialOffers == true
group p by p.ProductFamily == null ? null : p.ProductFamily into f
select f.GroupedProduct).ToList<Product>();
But I can't get either solution to work with what I have.
You can try (boring, but not sure you have the choice)
var query = from p in db.Products
where (p.Visible == true && p.Active == true)
&& p.ShowInSpecialOffers == true
group p by p.FamilyID == null ? 0 : p.FamilyID into f
select new {
ID = f.First().ID,
Name = f.First().Name,
Cost = f.OrderBy(m => m.NowCost).First().Cost,
RRP = f.First().RRP
};
return query.ToList().Select(m =>
new Product {
ID = m.ID,
Name = m.Name,
Cost = m.Cost,
RRP = m.RRP
};
EDIT
Or as pointed by Master Skeet (not exactly the same as what you tried, but much easier)
var query = from p in db.Products
where (p.Visible == true && p.Active == true)
&& p.ShowInSpecialOffers == true
group p by p.FamilyID == null ? 0 : p.FamilyID into f
select f.OrderBy(m => m.NowCost).First();
You could simply select f.First() and return the resulting data, like this:
var query = from p in db.Products
where (p.Visible == true && p.Active == true)
&& p.ShowInSpecialOffers == true
group p by p.FamilyID == null ? 0 : p.FamilyID into f
select f.First();
Then outside the method use whatever properties you need, ignoring what you don't care about (you are handling Product objects, properties should be there even if you don't need them).
Remove the word 'Product' from
select new Product{}
'select new' itself will create an anonymous type whereas you are specifying it as a Product.

Resources