I have written a stored procedure that includes a SELECT on a number of tables that applies logic to calculate values and transforms some of the data.
I have been asked if I can exclude records from the resultset in the stored procedure and write the record to a separate log table. I was looking to loop through the result set from the SELECT statement and delete the record I want to exclude once I have written it to a table. At the moment I am struggling to find the syntax to delete from the result set of a SELECT statement in a stored procedure and can only find how to use the cursor reference to delete from the original database table.
I need to remove the records in the same stored procedure and I am looking to avoid duplicating the logic by using some of the logic to find the records to include and repeat some of the logic again to be able to find the records to exclude. The only other alternative I can think of is using a temporary table, but I think what I am trying to do should be possible.
Any help appreciated.
When you have an open cursor in a stored procedure (or in an application), you can perform positioned deletes. You can execute the statement,
DELETE WHERE CURRENT OF cursorname;
Please be aware that by default issuing a COMMIT statement will close any open cursors, so if you plan to have this delete operation spread over multiple transactions you will need to declare your cursors using WITH HOLD.
Related
Quick question (hopefully)
I have a large dataset (>100,000 records) that I would like to use as a lookup to determine existence or non-existence of multiple keys. The purpose of this is to find FK violations before trying to commit them to the database to try and avoid the resultant EDatabaseError messing up my transaction.
I had been using TClientDataSet/TDatasetProvider with the FindKey method, as this allowed a client-side index to be set up and was faster (2s to scan each key rather than 10s for ADO). However, moving to large datasets the population of the CDS is starting to take far more time than the local index is saving.
I see that I have a few options for alternatives:
client cursor with TADOQuery.locate method
ADO SELECT statements for each check (no client cache)
ADO SEEK method
Extend TADOQuery to mimic FindKey
The Locate method seems easiest and doesn't spam the server with the SELECT/SEEK methods. I like the idea of extending the TADOQuery, but was wondering whether anyone knew of any ready-made solutions for this rather than having to create my own?
I would create a temporary table in the database server. Insert all 100,000 records into this temp table. Do bulk inserts of say 3000 records at a time, to minimise round trips to the server. Then run select statements on this temp table to check for foreign key violations etc. If all okay, do an insert SQL from the temp table to the main table.
I'm using EF 4.1 (Code First). I need to add/update products in a database based on data from an Excel file. Discussing here, one way to achieve this is to use dbContext.Products.ToList() to force loading all products from the database then use db.Products.Local.FirstOrDefault(...) to check if product from Excel exists in database and proceed accordingly with an insert or add. This is only one round-trip.
Now, my problem is there are two many products in the database so it's not possible to load all products in memory. What's the way to achieve this without multiplying round-trips to the database. My understanding is that if I just do a search with db.Products.FirstOrDefault(...) for each excel product to process, this will perform a round-trip each time even if I issue the statement for the exact same product several times ! What's the purpose of the EF caching objects and returning the cached value if it goes to the database anyway !
There is actually no way to make this better. EF is not a good solution for this kind of tasks. You must know if product already exists in database to use correct operation so you always need to do additional query - you can group multiple products to single query using .Contains (like SQL IN) but that will solve only check problem. The worse problem is that each INSERT or UPDATE is executed in separate roundtrip as well and there is no way to solve this because EF doesn't support command batching.
Create stored procedure and pass information about product to that stored procedure. The stored procedure will perform insert or update based on the existence of the record in the database.
You can even use some more advanced features like table valued parameters to pass multiple records from excel into procedure with single call or import Excel to temporary table (for example with SSIS) and process them all directly on SQL server. As last you can use bulk insert to get all records to special import table and again process them with single stored procedures call.
I thought I would use a stored routine to clean up some of my more complex SQL statements. From what I've read, it seems impossible to use a stored procedure within an sql statement, and a stored function only returns a single value when what I need is a result set. I am using mySQL v5.0
SELECT p.`id`, gi.`id`
FROM `sport`.`players` AS p JOIN `sport`.`gameinstances` AS gi
ON p.`id` = gi.`playerid`
WHERE (p.`playerid` IN (CALL findPlayers`("Canada", "2002")))
AND (gi.`instanceid` NOT IN (CALL findGameInstances`("Canada", "2002")));
For example, the procedures 'findPlayers' and 'findGameInstances' are are stored routines that execute some SQL and return a result set. I would prefer not to include their code directly within the statement above.
I don't know if mysql can use any of these techniques, but in SQl server I would try one of two different things (at least it might give you something to look for in th emysql documentation):
First a table values used defined function then join to that
Second, insert the results set of the sp into a temp table then join to the tem table
You could also consider putting the complicated logic in a view and then just adding the where clause after joining to the view. This won't work if your stored proc does dynamic things a view can't do, but it is a possibilty.
I am attemting to create a storedproc that reads like:
Select
ep.EmployeeID, GetEmployeeFirstName(ep.EmployeeID),
GetEmployeeLastName(ep.EmployeeID), ed.EmployeeDateOfBirth,
ed.EmployeeAddress, ed.EmployeeAddress2, ed.City, ed.State, ed.ZipCode
From
EmployeeProfile ep, EmployeeDetail ed
Where
ep.EmployeeID = ed.EmployeeID
This block of code will be a stored procedure.
My issue is that GetEmployeeFirstName is a stored proc that has to be passed an EmployeeID to get the employees first and last name.
How can I call a storedproc within a stored proc.
Thanks
Mike
These would probably be better suited as a function.
GetEmployeeFirstName(ep.EmployeeID), GetEmployeeLastName(ep.EmployeeID)
Better yet, just join the table that has the names.
I don't understand what these stored procedures do. Even if the first and last name are not in the EmployeeProfile table, and even if you have to do some manipulation of the strings before they are returned, a join would be a much better solution than a stored procedure or function. Especially when you take performance into account.
If you have the GetEmployeexName sprocs because you use them elsewhere, that's fine. Whatever they do, I would not consider it code duplication if they don't get called from your query.
You need to understand that for every row in your result set, two other procedures or functions get called. This is extremly costly and can render an application unacceptably slow, even for relatively small result sets of a few thousand employees. I know what I am talking about - I removed a lot of function calls from queries during a recent database tuning initiative.
In SQL Server, in order to call the GetEmployeeLastName within the Select statement list I would convert it to a database Function.
You can use EXEC or sp_executesql to execute a stored procedure from another stored procedure. (BTW, you have not specified your RDBMS).
Doesn't your table EmployeeDetail contain the employee's first and last name?
Select
ep.EmployeeID, ed.FirstName
ed.LastName, ed.EmployeeDateOfBirth,
ed.EmployeeAddress, ed.EmployeeAddress2,
ed.City, ed.State, ed.ZipCode
From
EmployeeProfile ep
inner join EmployeeDetail ed ON ep.EmployeeID = ed.EmployeeID
Let's say I have 'myStoredProcedure' that takes in an Id as a parameter, and returns a table of information.
Is it possible to write a SQL statement similar to this?
SELECT
MyColumn
FROM
Table-ify('myStoredProcedure ' + #MyId) AS [MyTable]
I get the feeling that it's not, but it would be very beneficial in a scenario I have with legacy code & linked server tables
Thanks!
You can use a table value function in this way.
Here is a few tricks...
No it is not - at least not in any official or documented way - unless you change your stored procedure to a TVF.
But however there are ways (read) hacks to do it. All of them basically involved a linked server and using OpenQuery - for example seehere. Do however note that it is quite fragile as you need to hardcode the name of the server - so it can be problematic if you have multiple sql server instances with different name.
Here is a pretty good summary of the ways of sharing data between stored procedures http://www.sommarskog.se/share_data.html.
Basically it depends what you want to do. The most common ways are creating the temporary table prior to calling the stored procedure and having it fill it, or having one permanent table that the stored procedure dumps the data into which also contains the process id.
Table Valued functions have been mentioned, but there are a number of restrictions when you create a function as opposed to a stored procedure, so they may or may not be right for you. The link provides a good guide to what is available.
SQL Server 2005 and SQL Server 2008 change the options a bit. SQL Server 2005+ make working with XML much easier. So XML can be passed as an output variable and pretty easily "shredded" into a table using the XML functions nodes and value. I believe SQL 2008 allows table variables to be passed into stored procedures (although read only). Since you cited SQL 2000 the 2005+ enhancements don't apply to you, but I mentioned them for completeness.
Most likely you'll go with a table valued function, or creating the temporary table prior to calling the stored procedure and then having it populate that.
While working on the project, I used the following to insert the results of xp_readerrorlog (afaik, returns a table) into a temporary table created ahead of time.
INSERT INTO [tempdb].[dbo].[ErrorLogsTMP]
EXEC master.dbo.xp_readerrorlog
From the temporary table, select the columns you want.