linq: SQL performance on high loaded web applications - asp.net-mvc

I started working with linq to SQL several weeks ago. I got really tired of working with SQL server directly through the SQL queries (sqldatareader, sqlcommand and all this good stuff). 
After hearing about linq to SQL and mvc I quickly moved all my projects to these technologies. I expected linq to SQL work slower but it suprisongly turned out to be pretty fast, primarily because I always forgot to close my connections when using datareaders. Now I don't have to worry about it.
But there's one problem that really bothers me. There's one page that's requested thousands of times a day. The system gets data in the beginning, works with it and updates it. Primarily the updates are ++ # -- (increase and decrease values). I used to do it like this
UPDATE table SET value=value+1 WHERE ID=#I'd
It worked with no problems obviously. But with linq to SQL the data is taken in the beginning, moved to the class, changed and then saved.
Stats.registeredusers++;
Db.submitchanges();
Let's say there were 100 000 users. Linq will say "let it be 100 001" instead of "let it be increased by 1".
But if there value of users has already been increased (that happens in my site all the time) then linq will be like oops, this value is already 100 001. Whatever I'll throw an exception"
You can change this behavior so that it won't throw an exception but it still will not set the value to 100 002.
Like I said, it happened with me all the time. The stas value was increased twice a second on average. I simply had to rewrite this chunk of code with classic ado net.
So my question is how can you solve the problem with linq

For these types of "write-only queries" I usually use a Stored Procedure. You can drag the stored procedure into the designer and execute it through the Linq to SQL DataContext class (it will be added as a method).
Sorry for the trite answer but it really is that simple; no need to to finagle with raw ADO.NET SqlCommand objects and the like, just import the SP and you're done. Or, if you want to go really ad-hoc, use the ExecuteCommand method, as in:
context.ExecuteCommand("UPDATE table SET value = value + 1 WHERE ID = {0}", id);
(But don't overuse this, it can get difficult to maintain since the logic is no longer contained in your DataContext instance. And before anybody jumps on this claiming it to be a SQL injection vulnerability, please note that ExecuteCommand/ExecuteQuery are smart methods that turn this into a parameterized statement/query.)

Linq to Sql supports "optimistic" concurrency out of the box. If you need tighter control, you can add a Timestamp column to your table, and Linq to Sql will use that timestamp to tighten the concurrency.
http://mtaulty.com/CommunityServer/blogs/mike_taultys_blog/archive/2008/07/01/10557.aspx
However, as Morten points out in the comments below, this solution is not going to perform well. Of course, you can always use ADO.NET to update the value, just like you were doing before; that won't adversely affect the operation of your Linq queries at all.

You could turn off concurrency on that property by changing the UpdateCheck value:
http://msdn.microsoft.com/en-us/library/bb399394(v=VS.90).aspx
Messy if your using generated code and the designer but I think this is the only way to do this.

Related

MVC 4 Strongly Typed Data to View without ORM

I've seen other questions regarding using straight SQL for MVC data access, for example here. Most responses don't answer the question but ask
"Why would you not want to use ORM, EF, Linq, etc?"
My group does custom reporting out of a data warehouse that requires a lot of complex, highly tuned Oracle queries that are manipulated based on user GUI parameter selections.
My newest project is to develop a SQL plugin reporting tool for SQL report developers. They would create a pre-tuned SQL for the report with pseudo parameters and enter (and store) via the GUI. Then the GUI would prompt them for the parameter definitions (name and type) that need to be displayed/requested at run time to ultimately replace the pseudo variables.
So a SQL statement may look like:
SELECT * FROM orders WHERE order date BETWEEN '<Date1>' AND '<Date2>'
And the report developer would then, via the GUI, add two parameters named Date1 and Date2, and flag them as date fields.
End users would then select the report, get prompted for Date1 and Date2, and the GUI would do the substitution and run the SQL.
As you can see, I have no choice but to use straight SQL (especially in the 2nd example and understand I would have to forgo strongly typed in the 2nd also).
So my questions are:
When is it necessary to bypass EF/Linq (and there are definitely reasons to), what is best practice in MVC 4?
And how best to strongly type when I do know the output columns ahead of time?
And CRUD processing?
Can anyone point me to any examples of non-EF/Linq based coding in this regard?
I think this is a bit open ended question, so here's my 2c. (If tl dr, go to last section)
To me, it's not so much "by passing EF/Linq", but rather, the need to choose the appropriate data persistence library. I have used PetaPoco, Ado.Net, NHibernate/ActiveRecord, Linq2Sql, EF (My main choice) with MVC.
Best practice actually comes from realising that Controllers are STILL a part of presentation layer, and that it should not deal with anything other than HttpContext related operations + calling business logic service classes.
I arrange my classes as:
Presentation (MVC) -> Logic Services (Simple classes) -> Data Access (Context wrapped in "repositories").
So I can't quite imagine whether to use EF or not would have any implication on asp.net MVC.
For me, Data Access returns data in DTO, e.g.
public List GetAllFoos()
Whether that method string concatenate from a xml, etc or do a simple Context.Foos.ToList() is irrelevant to the rest of the application. All I care is Data Access do NOT return me a DataSet with string matching for columns. Those stay in DAL.
See point 1 and 2. My repositories have CRUD methods on it. How it's done is irrelevant to the rest of application. Take one of the most basic interface to my repositories classes:
public interface IFooRepository
{
void Save(Foo foo)
Foo Get(int id)
void Create(Foo foo)
void Delete(int id)
}
One point not mentioned yet, DI is also crucial. The concrete implimentation "FooRepository" may choose to request dependencies such as Web services, context classes, etc. Those are, however, again, completely irrelevant to the caller who depends on the interface.
If you still require an example after the 3 points above, drop a comment and I'll whip up something extremely simple using Ado.net.
===========================================================================
To EF or not to EF.
For me, if starting a new project with new schema, I use EF code first.
Fitting new code to old database + old project has no ORM mapping I can reuse = PetaPoco.
===========================================================================
In the context of your project:
The "SQL plugin reporting tool for SQL report developers". "The" sql reporting service? I'm not sure why you need to do anything? Doesn't SSRS already do that? (Enter sql statement/data source, generate form for parameter, etc).
If not I'd question the design decision. IMVHO, the need for users of an application (I don't care if it's "report developer" or w/e) to enter SQL statements is usually stemmed from "architectural astronauts". How do you debug the SQL statement when you enter via GUI as a string? How do you know the tables and the relationships? You either dig into SSMS and come back to gui, or you build complex UI (aka rebuild SSMS).
At the end of day, if you want bazillion reports for gazillion different users, you have to pay for it. I see too many "architectural astronauts" who exposes application to accept SQL statements only to make everyone waste time guessing what should be put into it. No cost saving at all.
Ok, if you must do that, well eh... Good luck. Best bet is to return as a DataTable and dump the rows/columns/data on to the view with nested foreach looping through rows then columns.

ASP.NET MVC: Best Way To Call Stored Procedure

I'm trying to decide which is the best way to call a stored procedure.
I'm new to ASP.NET MVC and I've been reading a lot about Linq to SQL and Entity Framework, as well as the Repository Pattern. To be honest, I'm having a hard time understanding the real differences between L2S and EF... but I want to make sure that what I'm building within my application is right.
For right now, I need to properly call stored procedures to: a) save some user information and get a response and, b) grab some inforation for a catalog of products.
So far, I've created a Linq to SQL .dbml file, selected the sotred procedure from the Server Explorer and dragged that instance into the .dbml. I'm currently calling the Stored Procedure like so:
MyLinqModel _db = new MyLinqModel();
_db.MyStoredProcedure(args);
I know there's got to be more involved... plus I'm doing this within my controller, which I understand to be not a good practice.
Can someone recognize what my issues are here?
LINQ and EF are probably overkill if all you're trying to do is call a stored proc.
I use Enterprise Library, but ADO.NET will also work fine.
See this tutorial.
Briefly (shamelessly copied-and-pasted from the referenced article):
SqlConnection conn = null;
SqlDataReader rdr = null;
// typically obtained from user
// input, but we take a short cut
string custId = "FURIB";
Console.WriteLine("\nCustomer Order History:\n");
// create and open a connection object
conn = new SqlConnection("Server=(local);DataBase=Northwind; Integrated Security=SSPI");
conn.Open();
// 1. create a command object identifying
// the stored procedure
SqlCommand cmd = new SqlCommand(
"CustOrderHist", conn);
// 2. set the command object so it knows
// to execute a stored procedure
cmd.CommandType = CommandType.StoredProcedure;
// 3. add parameter to command, which
// will be passed to the stored procedure
cmd.Parameters.Add(
new SqlParameter("#CustomerID", custId));
// execute the command
rdr = cmd.ExecuteReader();
// iterate through results, printing each to console
while (rdr.Read())
{
Console.WriteLine(
"Product: {0,-35} Total: {1,2}",
rdr["ProductName"],
rdr["Total"]);
}
}
Update
I missed the part where you said that you were doing this in your controller.
No, that's not the right way to do this.
Your controller should really only be involved with orchestrating view construction. Create a separate class library, called "Data Access Layer" or something less generic, and create a class that handles calling your stored procs, creating objects from the results, etc. There are many opinions on how this should be handled, but perhaps the most common is:
View
|
Controller
|
Business Logic
|
Data Access Layer
|--- SQL (Stored procs)
-Tables
-Views
-etc.
|--- Alternate data sources
-Web services
-Text/XML files
-blah blah blah.
MSDN has a decent tutorial on the topic.
Try this:
Read:
var authors = context.Database.SqlQuery<Author>("usp_GetAuthorByName #AuthorName",
new SqlParameter("#AuthorName", "author"));
Update:
var affectedRows = context.Database.ExecuteSqlCommand
("usp_CreateAuthor #AuthorName = {0}, #Email= {1}",
"author", "email");
From this link: http://www.dotnetthoughts.net/how-to-execute-a-stored-procedure-with-entity-framework-code-first/
And I would go with the framework David Lively mentioned, instead of having the routines in the controller. Simply pass the results back as IEnumerable<blah> from a function in a separate repository class for an edit, pass a boolean back for if the update succeeded for an update.
LINQ to SQL and ADO.NET EF attach read stored procs to the data/object context class that you use to go against its various entities. For create, update, and delete, you can create a proc that maps the properties of an entity that the model generates, and using the entity mapping window (forget the exact name right now), you can map an entities fields with the proc parameters. So, say you have a Customers table, EF generates a Customers Entity, and you can map the proc parameters to the properties of the Customer entity when attempting to update/insert/delete.
Now, you can map a CUD proc to a function, but I don't know all the repercussions; I like the way I just mentioned the best.
HTH.
I common pattern is to pass a repository interface into your controller by dependency injection. The choice of what persistence/orm technology you use is really another issue and unrelated to the fact that you are using MVC. Using the repository pattern and coding to abstractions (interfaces) makes your application easy to test by mocking out your repositories.
I think you should also try to use as few stored procedures as possible. This means you can more easily test your logic in isolation (unit tests) without needing to be connected to a database. I would highly recommend looking at NHibernate. The learning curve is fairly steep but you are in full control of your mappings and configuration. There are obviously occasions where you will need stored procs for performance reasons, but using an ORM predominantly is very beneficial.
I can't imagine that your goal is to be able to call a stored procedure. To me it sounds as if you need to forget stored procedures and use Linq to Sql. I say L2S because EF is far more to learn, and not needed in this case.

Entity Framework 4: Math.Sin()-function

is there an possibility to call the Math.Sin()-function in a Linq To Entites (Entity Framework 4) -Query?
I've read, that the current Entity Framework 4 doesn't implement this function.
Maybe there's a workaround to this solve problem?
(I don't want to invite all entries in the memory.)
Thanks and best regards
Several functions that (usually) have obvious SQL counterparts, like Math.Sin can't be used directly in Entity Framework queries. Presumably this is because they can't be reliably translated to different SQL implementations. A ton of MSSQL-specific functions are, however, exposed as static methods in the class System.Data.Objects.SqlClient.SqlFunctions. They throw exceptions if you call them directly, but are translated into the proper SQL if used in a LINQ query.
See this blog post about the magic that's happening under the covers (namely the EdmFunction attribute).
It is certainly possible to use such function starting with EF4. In EF4, EF team introduced SqlServer functions that can be consumed in linq. You should alway consider using canonical functions cuz they are database agnostic and every vendor should convert those functions to store specific equivalent. However when such functions are not available, you can resort to SqlServer namespace (ESQL) or SqlFunctions for linq
from l in db.Locations
select SqlServer.Sin(l.Latitude) + SqlServer.power(l.Longitutde)
I cover several of these options in my functions chapter in my book. Specifically you can look at 11-10 recipe Calling database function in esql
11-11 Calling Database Function in LINQ
Unfortunately it's impossible to call Math.Sin in a LinqToEntities query (or Entity SQL query).
The only way to accomplish this without resorting to retrieving all objects first, is to write a SQL query that does what you want and call it via ObjectContext.ExecuteStoreQuery. This isn't as bad as it sounds because you can still get back typed results.
EDIT: After reading the other answers, it appears that it is possible to call these types of functions (SqlFunctions contains 44 functions with various overloads). I leave my original answer as is because it's another way of achieving the same result.

How to update a single field using EF4

I want to update a single field in my table for a particular row. I am using Entity Framework 4 and Visual Studio 2010.
Options I can think of are:
Using a Stored Procedure
Direct connection to the database and using
sql statement
I am not aware of any more efficient method to perform this task.
[EDIT]
I would like to do the update in the same operation as the Get for that row, so that it is done in one DB call.
No need to complicate things. Just change the one property and SaveChanges. Unless you're doing something odd, that should only change the one column. Look at the SQL to verify.

How to make UPDATE queries in LINQ to SQL?

I like using LINQ to SQL. The only problem is that I don't like the default way of updating tables.
Let's say I have the following table with the following columns:
ID (primary key), value1, value2, value3, value4, value5
When I need to update something I call
UPDATE ... WHERE ID=#id
LINQ to SQL calls
UPDATE ... WHERE ID=#id and value1=#value1 and value2=#value2 and value3=#value3 and value4=#value4 and value5=#value5
I can override this behavior by adding
UpdateCheck=UpdateCheck.Never
to every column, but with every update of the DataContext class with the GUI, this will be erased. Is there any way to tell LINQ to use this way of updating data?
I'm confused by this statement:
but with every update of the DataContext class with the GUI, this will be erased. Is there any way to tell LINQ to use this way of updating data?
By "the GUI", do you mean the Linq to SQL designer? Because the property sheet for each member has an "Update Check" property that you can set to "Never". If you are manually editing the .designer.cs file, don't do that, instead change the Update Check setting in the actual designer.
Designer Screen http://img29.imageshack.us/img29/7912/updatecheckdesigner.png
Please note: The "default way" of updating used by Linq to SQL is called optimistic concurrency, and is a way of preventing conflicting updates from multiple users. If you turn this off by using the method above, you have to be prepared to deal with the fact that if two users have the same record open at the same time, the second user's changes will overwrite the first user's changes without any warning or confirmation. Be sure that this is the behaviour you really want.
Unfortunately, no, there's not. You have to edit the DBML manually after it is generated (or updated) - which is a pain (or use the Designer as already mentioned in the other answer).
When I last used L2S on a project, I wrote a quick utility which ran post-generation and fixed it up, but it's an unnecessary pain which (c)shouldn't be required IMHO.
Ran into this one myself. The trick is to change the way one generates the DBML--such as using l2st4. Then you can set that pesky UpdateCheck property to always be never by modifying the template.
That is how Linq works. Why don't you like this update behavior?
Read about optimistic concurrency
http://msdn.microsoft.com/en-us/library/bb399373.aspx

Resources