I used an ado entity data model and added it to my web application. When I want to debug
with breakpoints, I can't see the data of the query, only a reference to the model.
If you haven't executed the query by iterating over the results (e.g. using foreach or ToList) then this is expected.
All that
var query = from e in db.SomeTable
select e;
does is set up code that can eventually be executed and produce the results of the query but it doesn't produce the results until you iterate over query as above.
Beyond that, you haven't given us much to go on.
Related
I am trying to connect to my organisation's SQL database using Power Query to create some reports. I need to delete/edit some tables and join multiple tables to come up with the desired report output...
I don't want the change or edit I will do on the excel-power query to reflect on the live database but just in excel .
The short answer is no, any button you press in the Power Query Editor interface does not modify the source database. I must admit that I have not found any page in the Microsoft Docs on Power Query that states this clearly. The page What is Power Query? states that:
Power Query is a data transformation and data preparation engine. Power Query comes with a graphical interface for getting data from sources and a Power Query Editor for applying transformations.
Other pages contain similarly general and vague descriptions but let me reassure you that any data transformation you carry out by using the Power Query Editor interface will not modify your SQL database. All you see in Power Query is a view of the source database.
Seeing as you are connecting to a SQL database, it is likely that query folding is activated. This means that when you remove a column (or row), this will update the SQL query used to extract the data from the database. That query is written as a single SELECT statement that can contain multiple clauses like GROUP BY and WHERE. Transformations that add data (e.g. Add Custom Column, Fill Down) are not included in the query, they are carried out only within the Power Query engine. You can read more about this in the docs.
How to edit a database with Power Query when native SQL queries are supported
That being said, you can actually edit a database from within Power Query if the database supports the use of native SQL queries, if you have write permission for the database, and if you edit and run one of the two M functions that let you write native SQL queries. Here is an example using the Sql.Database function:
Sql.Database("servername", "dbname", [Query = "DROP TABLE tablename"])
And here is an example using the Value.NativeQuery function:
Source = Sql.Databases("servername"){[Name="dbname"]}[Data],
#"Native Query" = Value.NativeQuery(Source, "DROP TABLE tablename")
Unless you have changed the default Query Options, these functions should raise a warning message requiring you to permit running the query:
This prevents you from modifying the database without confirmation, so any database modification cannot happen just by accident.
I verified this using Excel Microsoft 365 (Version 2108) on Windows 10 64-bit connected to a local SQL Server 2019 (15.x) database.
I was trying to figure out how the DataContext.ObjectTrackingEnabled property works in F# SQL Type Provider. Unfortunatelly I don't find any useful documentation except this msdn entry which is unhelpful too.
Shouldn't the object tracking feature, track all the objects loaded inside the same "session/context"? Let's say I query for an object, it gets loaded to the "context" and if I query for the same object once again in the same "context" it should load it from the context without executing SQL statement in RDBMs?
This code results in two SQL queries. I was expected to see only one in SQL Server Profiler:
[<Literal>]
let connectionString = #"server=localhost;database=..."
type dbSchema = SqlDataConnection<ConnectionString=connectionString>
let db = dbSchema.GetDataContext(connectionString)
let result = (query
{
for t in db.Table1 do
where (t.Id = 1)
select t
}).SingleOrDefault()
let result2 = (query
{
for t in db.Table1 do
where (t.Id = 1)
select t
}).SingleOrDefault()
What do I miss?
This is really a property of the LINQ to SQL framework, so it shouldn't be specific to F# (and so finding more about object tracking in LINQ to SQL should answer the question).
I'm not really a LINQ to SQL expert, but my understanding of this feature is that having ObjectTrackingEnabled = true does not avoid running multiple queries to retrieve the data. What it does is that it ensures that the queries return the same .NET objects representing the entities.
This means that if you have two queries that return the same entitiy and you modify a property of one of the entity (before you commit the change to the database), this will also change the property of the entity returned by the other query (because they're the same object).
In contrast, if you disable this and run two queries, they both return new .NET objects with values exactly as in the database. If your program modifies one of the objects, it will not affect the other (and so if you try to commit changes to the DB using both entities, you'll get a conflict).
In summary - disabling this gives you better performance if you're just reading data (because there is less tracking overhead), but it might be inappropriate if you want to edit the objects in your application and save data back to the database.
As Tomas said this is not directly related to the Type Provider but to the underlying LINQ to SQL implementation. But many F# developers use those providers, so a little history on Linq to SQL would be useful.
In general ObjectTrackingEnabled = true avoid running multiple queries to retrieve the data under the following conditions :
In general, for a LINQ to SQL query to return an object from the
identity cache, the query must be based on the primary key of an
object and must return a single object.
so in where(predicate) predicate must be an expression in which the object's primary key property is set to a constant value. If an object has a primary key defined by more than one property, each primary key property must be set to a constant value. The following are examples of the form predicate must take:
c => c.PK1 == constant_value1 && c=> c.PK2 == constant_value2
This information is taken from MSDN
I have a requirement to fetch Price List Item records which adhere to the following requirements:
Filter by a specific PriceList
Filter by a specific currency
Filter by the Name of the related Product containing a given string
I got the first two points working no problem, but it feels like expanding doesn't cope well with filtering. I started from a "straight" query on Product entity:
.../ProductSet?$filter=substringof('sometext', Name)
Equivalent SQL (targeting the corresponding CRM filtered views for clarity):
SELECT * FROM FilteredProduct WHERE ProductNumber LIKE '%sometext%'
The above query works, I can tweak it and have no issues. However, if I attempt to move on to ProductPriceLevel (thus expanding the relationship with Product, which is product_price_levels) I end up with this:
.../ProductPriceLevelSet?$expand=product_price_levels&$filter=substringof('sometext', product_price_levels/Name)
Equivalent SQL (again, targeting the relevant filtered views):
SELECT * FROM FilteredProductPriceLevel PPL JOIN FilteredProduct P
ON PPL.ProductId = P.ProductId WHERE P.ProductNumber LIKE '%sometext%'
Which has two different outcomes I see:
If the $filter has no matches, it works fine and returns an empty result set
If the $filter matches something, I get an error
code: -2147220970
message: The result selector of the 'Join' operation must return an anonymous type of two properties.
AFAIK that's what happens when you hit a limitation of LINQ-to-CRM regarding using .Where() on multiple entities at once... doesn't seem relevant!
What's wrong with my query ?
NOTE: The CRM 2013 I'm using is On-Premise, without any update rollup / service pack.
ALSO NOTE: The equivalent SQL, as can be expected, works perfectly
I don't think CRM OData supports adding a filter on a joined entity. Try reversing the entity you're actually starting with, and add a path to the referencing entity:
ProductSet()?$filter=substringof('sometext',ProductNumber)&$expand=product_price_levels&$select=product_price_levels/*
P.S. If you have linqPad, this is the query I used to generate this:
from p in ProductSet
where p.ProductNumber.Contains("sometext")
select new { p.product_price_levels }
I've created a WebApi project in VS 2012, using NHibernate as my ORM and I intend to enable Odata support on it. So I've created a test controller with a single Get method that returns a list of entities from a table on my database.
Everything works fine, I can use OData to filter and order my results, etc. The problem is I couldn't find a way to limit the amount of data that's being returned from the database to the controller, and this table has millions of records in it.
Using the PageSize property of the Queryable attribute only seems to be limiting the amount of data returned to the client, but no the amount of Data returned from the DB.
I've tried applying a Take(n) on the IQueryable inside the get method before returning it, and it limits the results brought back from the DB, but it breaks the OData filtering, since if you try to query an entity that's not in the first n results, it just returns an empty collection.
I know you can use the $Top parameter on OData to accomplish this, but I would like not to depend on the client/consumer providing it in order to ensure that I'm not unnecessarily bringing thousands or even million of records that I'm not going to use.
I've also tried to manually check if the client provided a Top parameter on the query string, apply the OData transformation to my Queryable and then applying the Take(n) method over the transformed query. This approach enabled me to filter for any entity through OData, but it breaks pagination, because if I use the $Skip=n parameter, it again returns an empty collection.
So, is there any way to reliably limit the results fetched from the DB while not breaking the OData support?
We recently found that too. We are not applying a Take(pageSize) when server driven paging is enabled as we have to figure out if a next page link should be generated or not. We just enumerate the result set for pageSize number of entities and check if there are more entities or not. We thought that most providers generally bring a partial set of results as IQueryable is generally a lazy implementation. Turns out that is not true. Also, the database can optimize the query if it knows only pageSize number of results are required.
This is the issue that was opened for it. Good news is Youssef fixed it already :). This is the commit that fixed it. So, if you grab the nightly builds you should be good.
This is probably a very simple question that I am working through in an MVC project. Here's an example of what I am talking about.
I have an rdml file linked to a database with a table called Users that has 500,000 rows. But I only want to find the Users who were entered on 5/7/2010. So let's say I do this in my UserRepository:
from u in db.GetUsers() where u.CreatedDate = "5/7/2010" select u
(doing this from memory so don't kill me if my syntax is a little off, it's the concept I am looking for)
Does this statement first return all 500,000 rows and then filter it or does it only bring back the filtered list?
It filters in the database since your building your expression atop of an ITable returning a IQueryable<T> data source.
Linq to SQL translates your query into SQL before sending it to the database, so only the filtered list is returned.
When the query is executed it will create SQL to return the filtered set only.
One thing to be aware of is that if you do nothing with the results of that query nothing will be queried at all.
The query will be deferred until you enumerate the result set.
These folks are right and one recommendation I would have is to monitor the queries that LinqToSql is creating. LinqToSql is a great tool but it's not perfect. I've noticed a number of little inefficiencies by monitoring the queries that it creates and tweaking it a bit where needed.
The DataContext has a "Log" property that you can work with to view the queries created. I created a simple HttpModule that outputs the DataContext's Log (formatted for sweetness) to my output window. That way I can see the SQL it used and adjust if need be. It's been worth its weight in gold.
Side note - I don't mean to be negative about the SQL that LinqToSql creates as it's very good and efficient almost every time. Another good side effect of monitoring the queries is you can show your friends that are die-hard ADO.NET - Stored Proc people how efficient LinqToSql really is.