SQL-Server Transaction Blocking Mystery - stored-procedures

I have a .Net app used for form processing which deletes/updates/inserts data across three different SQL Server 2012 databases. When the application runs, it opens a data context and then a transaction within that context for each form that needs to be processed (this runs every minute, so it's usually no more than one form at a time). A bunch of stuff happens within this transaction -- including multiple stored proc calls.
So here's the problem:
We have servers set up with what I'm told are the exact same specs (although I'm dubious :)). One is used for development work; the other for client testing. In our development environment, the processing runs without problems; but on the client testing site, it hangs every time. And I'm pulling my hair out trying to determine why.
In the following TSQL code, it is the insert into the Param table that is failing. The Param table is essentially the same as the Method table, except for the column names. Both inserts have similar foreign key relationships to the Form table, and both insert int values into the ID column.
When I run SQL Server Profiler, I'm told there's a lock on the FormDB which is not allowing the insert. However, I can alter the select statement for the Param insert and it works. I've altered in the following ways, all of which "work" in the sense that they do not cause the blocking issue:
Replaced the Param select with the Method select while keeping the insert to Param.(exact same column defs as param select)
Replaced #newKey with a valid integer for an existing form.
Removed the "from" portion of the Param select and hardcoded a single int value for the paramID (ie select #newKey, 1, #modifyDate, #modifyUser)
I feel like I'm losing my mind, because I just can't see why it's not working. The insert only seems to fail when three things are all in the select statement in combination -- #newKey, ParamID, and the from statement.
I've ensured each sproc has SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED and have used with nolock where necessary.
Why can I successfully insert into the Param table via the three scenarios above, but fail for the Param insert in the code that follows? Why would I not receive the same lock message in the profiler? There are about 5 other inserts in this procedure which follow the same pattern. All of them work with no problem.
Any ideas? Thanks.
USE [PROD]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROCEDURE [dbo].[HELPSPROC]
#oldKey int
AS
BEGIN
SET NOCOUNT ON;
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
declare #newKey int
, #spKey int
, #modifyDate datetime = getdate()
, #modifyUser varchar(30) = 'User'
/*
a bunch of stuff happens here, including setting the #spKey value.
this all happening correctly -- we have a valid integer value when we go into the next part
*/
---------------FORM---------------
INSERT INTO FormDB.dbo.Form
(formtype, formstatus, modifydate, modifyuser)
Select
'TestFormType', 'DRAFT', #modifyDate, #modifyUser
From FormDB2.dbo.Form f
Where f.Pkey = #oldKey
--grab the new int identifier -- works
set #newKey = (select scope_identity())
/*
stuff happens here. all is good in this part
*/
---------------Param---------------
INSERT INTO FormDB.dbo.Param
(FormKey, ParamID, ModifyDate, ModifyUser)
select #newKey, p.ParamID, #modifyDate, #modifyUser
from PROD.dbo.Table1 apd
inner join PROD.dbo.ParameterTable p
on apd.TableTwoKey = p.TableTwoKey
where apd.PKey = #spKey
---------------Method---------------
INSERT INTO FormDB.dbo.Method
(FormKey, MethodID, ModifyDate, ModifyUser)
select #newKey, r.MethodID, #modifyDate, #modifyUser
from PROD.dbo.Table1 apd
inner join PROD.dbo.MethodTable r
on apd.TableTwoKey = r.TableTwoKey
where apd.PKey = #spKey
/*
one more insert ...
*/
RETURN 1
END
GO

I still don't understand the why of this problem, but I found a solution.
There are a lot of things happening in the vb.net code for this process: multiple linq-to-sql inserts/deletes/updates across three separate databases, as well as two separate stored procedures calls. To add to that confusion, there are separate contexts declared for each db, each with its own transaction. In short, a bunch of moving parts.
The second stored proc call was conditional, based on certain values for the processing form. I just took that call out of the vb.net code, and placed the conditional logic and stored proc call within the first proc. That solved it. The second stored proc was called directly after the first anyway -- pending conditions -- so all is good.
Problem solved -- but if anybody can explain why this was occurring, I'd be grateful. Thanks!

Related

Can Firedac handle conditional SQL statements

Is there a way to use Firedac to handle conditional scenarios.
Master Table has a column called INVOICESCOUNT.
When an invoice is deleted successfully, then the INVOICESCOUNT is decreased.
For example, a SQL psuedo-code statement like this:
Delete From Invoices where INVOICE=500;
Update Customers SET INVOICECOUNT=INVOICECOUNT-1 WHERE Customer=1 (if prior statement returns 1 affected row);
I need it to be embedded within the same SQL statement instead of having the Delphi source code handling executing the 2 statements separately, after the first FDQuery returns a successful execution.
Thanks for any advice.
That's fine ?
I understood, lack of attention from me, in this case I could use the information from the DBMS itself, an example, if the DBMS is SQL server.
Delete From Invoices where INVOICE=500
IF ##ROWCOUNT=1
begin
Update Customers SET INVOICECOUNT=INVOICECOUNT-1
WHERE Customer=1
end.
If your DBMS doesn't allow it, I recommend you to use a stored procedure.

FireDAC - Show SQL after Macro Expantion

I am trying to use Macros in FireDAC to Preprocess my SQL Queries. I have a TADQuery object on a Data Module with the SQL set to something like:
Select * from MyTable
join OtherTable on MyTable.Key = OtherTable.Key
&Where
Then in my code I do this:
WhereClause = 'stuff based on my form';
Query.MacroByName('Where').AsRaw := WhereClause;
Query.Open;
This has worked great for complicated queries because it lets me make sure my fields and join conditions are correct using the SQL Property editor.
My problem is when the SQL statements ends up invalid because of my where clause. Is there any way to see the SQL after pre-processing that is going to be executed? Right now I am catching the FireDac errors and showing the SQL that is on EADDBEngineException object. However that is still showing my original SQL with the macros. If I can't get to it after the error happens is there anyway to force the Macro replacement to take place so I can look at the SQL in the debugger to help me see what is wrong.
If it matters I am connecting to a MS Access database with the goal of moving to SQL Server in the near future.
Apart from using Text property, to monitor what SQL is actually going to the database engine, consider using the "FDMonitor" FireDAC utility. According to the DokWiki pages (below):
drop a TFDMoniRemoteClientLink component on your form,
Set its Tracing property to True,
Add the MonitorBy=Xxx connection definition parameter to your existing FDConnection component. You can do this in the IDE object inspector, by selecting your FDConnection component, expanding the Params property, and setting MonitorBy to mbRemote.
Note that the TFDMoniXxxxClientLink should come before TFDConnection in the data module or form creation order, so adjust this by right clicking on the form or data module, then Creation Order, and moving the TFDMoni.. component above the FDConnection.
Also, it's helpful in the options of the TFDMoniXxxxClientLink, to disable most of the events being recorded, otherwise all the data returned is also shown in the FireDAC monitor. Expand the EventKinds property, and turn all the event kinds off, except for perhaps ekConnConnect, ekConnPrepare, and ekCmdExecute.
Then open the FireDAC Monitor from the IDE, (Tools > FireDAC Monitor). Start your app only once the monitor is running. Double click on a trace event (in the Trace Output tab), and you will see the actual SQL sent to the database in the bottom pane.
It also seems likely that adding the EventType of ekConnPrepare as mentioned above, would show you when the query's Prepare is called, but I haven't played enough with it say for sure.
Please see the following pages on the DocWiki for more information:
Overview: FDMonitor
How to: Tracing and Monitoring (FireDAC)
Other FireDAC utilities: Utilities (FireDAC)
(Just to remove this question from list of unanswered questions)
From comments:
Well, I've roughly checked what's happening there and I'm still not
sure if calling Prepare (which is useless for you as I get) is the
minimal requirement to trigger that preprocessing. Though, the
preprocessed SQL, the one which is sent to the DBMS you can access
through the Text property (quite uncommon name for such property). – TLama Feb
21 '14 at 8:18

TClientDataSet and processing records with StatusFilter

I'm using a TClientDataSet as a local dataset without the provider concept. After working with it a method is called that should generate the corresponding SQL statements using the StatusFilter to resolve the changes (generate SQL basically).
This looked easy initially after reading documentation (set StatusFilter to [dsInsert], process all inserts SQL, set StatusFilter to [dsModified] process all updates, the same with deletes) but after a few tests now looks far from trivial, for example:
If I add a record, then edit it: setting the StatusFilter to [dsInserted] displays it, but with the original data.
If I add a record, then edit, then delete it: the record appears with StatusFilter set to [dsInserted] and [dsModified] also.
And other similar situations..
1) I know that if first I process all inserts, then all updates then all deletes the database will be updated in the correct state but it looks far from right this approach (generating useless sql statements).
2) I've tried to access the PRecInfo(ClientDataSet.ActiveBuffer + ClientDataSet.RecordSize).Attribute information (dsRecNew, dsRecOrg, etc.) but still not manage to resolve the logic.
3) I can program the logic to resolve it, for example before processing and insert, set StatusFilter to [dsDeleted], and locating by the primary key if the record to see if its deleted thereafter.. the same with edits, before inserting, checking if the record was updated after so the insert sql in the updated version and so on.. but it should be more easy..
¿Did someone tried to solve this in an elegant and straightforward way? ¿I'm missing something? Thanks

BDE says "Field not found" but field exists

I have the following query to one of my database tables:
select count(*) as mycount
from mytable
where fieldone = :fieldone
and fieldtwo = :fieldtwo
Parameters are correctly loaded into the query (both of type String).
When I run this query outside the app (for instance, through the dbexplore) and replace the parameters with the actual values, I get the correct result. But when running it in the app, I get a Field 'fieldtwo' not found error, right on the Query.Open call.
Why would the BDE not find this field, when it actually exist?
Update: The following query, executed right after the first one (the one that fails), works fine in the app:
select *
from mytable
where fieldone = :fieldone
order by fieldone, fieldtwo
The best guess is that you have populated the field list in the query, this overrides any concept of the underlying fields that are in the query and is a cause of countless confusion.
Right click on the query, pick the fields editor clear all the values that are there and then choose 'add all fields' that should cause the missing field to appear once the query is executed.
I think it should auto-populate the fields if there are no defined fields when the query is executed, so you may not need to choose 'add all fields' after clearing the fields.
Whenever we come across a problem like this we tend to remove the query from the form and create it dynamically at run time... It depends how ingrained into the form it is...
E.g. If you have a data aware control looking at "fieldtwo" which tries to fetch some data when the underlying data set gets updated then it'll trigger an error like this, but it's more obvious when you've written code such
SomeEdit.Text = Query.FieldByName("fieldtwo").AsString;
That way it falls over on the relevant line instead of the open (triggering a related event)
Clear the query content using Query1.SQL.Clear; statement before opening it.
Other reason can be you are opening other database which may not have the specified field. Be sure that both the DatabaseName's in your app and dbexplore are same
I used to face porblems with BDE when i have SQLExplorer open and the app accesses the DB at the same time (but i had errors like ), try closing the Explorer it may help, if not i would build the SQL as text without the Parameters and try if it works then (if its possible in your situation).
I don't use parameters, so I'm just grabbing at straws here. I still use the BDE regularly, but am no expert. I find I shy away from more complex expressions (which yours is not!) because of the little "surprises" like this that the BDE throws at you.
Perhaps adding parentheses:
where (fieldone = :fieldone)
and (fieldtwo = :fieldtwo)
Or, single or double quote signs (this probably will make it worse?)
where (fieldon = ":fieldone")
and (fieldtwo = ":fieldtwo")
Or, to explore the problem, remove the "and fieldtwo = :fieldtwo" line and see if it runs.
Would it be possible for you to do your own parameter substitution with a StringReplace as in
Query1.SQL.Text := StringReplace(Query1.SQL.Text, ":fieldone", "MyVarName",[rfReplaceAll ]);
If you are creating a ClienDataSet in memory by the Create DataSet method, you should check the TFieldDefs property, which must have a different field name or not created
I was having a weird but small problem, I'll post in case it will help someone in some day.
uRegPeople.pas
with frmEditPerson do
begin
PersonID := qryPerson.FieldByName(ID).AsInteger;
...
end;
I had qryPerson both in frmRegPeople and in frmEditPerson, by using with I was referencing to frmEditPerson.qryPerson, however I wanted to reference to frmRegPeople.qryPerson. Then I need to change to the following code.
with frmEditPerson do
begin
PersonID := Self.qryPerson.FieldByName(ID).AsInteger;
...
end;
// Explanation
// qryPerson --> frmEditPerson.qryPerson;
// Self.qryPerson --> frmRegPeople.qryPerson;

How to capture input parameters from within stored procedure (SQL Server 2005)?

I would like to create a generic logging solution for my stored procedures, allowing me to log the values of input parameters. Currently I am doing this more or less by hand and I am very unhappy with this approach. Ideally, I would like to say something like the following:
"given my spid, what are my input parameters and their values?"
This is the same information exposed to me when I run SQL Profiler -- the stored procedure's name, all input params and all input VALUES are listed for me. How can I get my hands on these values from within a stored procedure?
Thanks;
Duncan
That is going to be difficult to do within a stored procedure. SQL profiler runs under a different SPID and runs a statement like this to capture the other users statements:
DECLARE #handle VARBINARY(64)
SELECT #handle = sql_handle from sys.sysprocesses where spid = #SPID
SELECT text FROM sys.dm_exec_sql_text(#handle)
The problem is if you run this in a stored proc for the current SPID all your going to get back is the statement above. I don't believe SQL server provides a T-SQL construct to execute a batch under a different SPID. I suppose you could write a .Net dll stored procedure that executes a batch on a different connection. to do that sort of thing but it may be more trouble than it's worth.

Resources