My application allows the administrator to define SQL templates scripts, that users can apply to data structures. The templates are stored within a database textfield, and will be merged with the data structure metadata.
A template statement looks like the following:
CREATE TABLE #{#target.code} (
#{#target.getColList('ORACLE', 'typed')}
)
and I plan to evaluate it to get the desired SQL query:
CREATE TABLE DSD_TEST (
AHVN13_1 number,
ALTER_MT_ERFUELLT_1 number,
CANTONS_2_1 varchar2(32),
POPULATION_TYPE_1 varchar2(32)
)
but actually the template statement (of Postgres text field) returns the following:
"CREATE TABLE \#{#target.code} ( \r\n\#{#target.getColList('ORACLE', 'typed')} \r\n)"
I found out that I can get rid of the new line using task.statement.gsub("\r\n", ''), but I cannot get rid of the special character '\ '.
How can I build this template so it can be evaluated?
Related
I have a .Net app used for form processing which deletes/updates/inserts data across three different SQL Server 2012 databases. When the application runs, it opens a data context and then a transaction within that context for each form that needs to be processed (this runs every minute, so it's usually no more than one form at a time). A bunch of stuff happens within this transaction -- including multiple stored proc calls.
So here's the problem:
We have servers set up with what I'm told are the exact same specs (although I'm dubious :)). One is used for development work; the other for client testing. In our development environment, the processing runs without problems; but on the client testing site, it hangs every time. And I'm pulling my hair out trying to determine why.
In the following TSQL code, it is the insert into the Param table that is failing. The Param table is essentially the same as the Method table, except for the column names. Both inserts have similar foreign key relationships to the Form table, and both insert int values into the ID column.
When I run SQL Server Profiler, I'm told there's a lock on the FormDB which is not allowing the insert. However, I can alter the select statement for the Param insert and it works. I've altered in the following ways, all of which "work" in the sense that they do not cause the blocking issue:
Replaced the Param select with the Method select while keeping the insert to Param.(exact same column defs as param select)
Replaced #newKey with a valid integer for an existing form.
Removed the "from" portion of the Param select and hardcoded a single int value for the paramID (ie select #newKey, 1, #modifyDate, #modifyUser)
I feel like I'm losing my mind, because I just can't see why it's not working. The insert only seems to fail when three things are all in the select statement in combination -- #newKey, ParamID, and the from statement.
I've ensured each sproc has SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED and have used with nolock where necessary.
Why can I successfully insert into the Param table via the three scenarios above, but fail for the Param insert in the code that follows? Why would I not receive the same lock message in the profiler? There are about 5 other inserts in this procedure which follow the same pattern. All of them work with no problem.
Any ideas? Thanks.
USE [PROD]
GO
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROCEDURE [dbo].[HELPSPROC]
#oldKey int
AS
BEGIN
SET NOCOUNT ON;
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
declare #newKey int
, #spKey int
, #modifyDate datetime = getdate()
, #modifyUser varchar(30) = 'User'
/*
a bunch of stuff happens here, including setting the #spKey value.
this all happening correctly -- we have a valid integer value when we go into the next part
*/
---------------FORM---------------
INSERT INTO FormDB.dbo.Form
(formtype, formstatus, modifydate, modifyuser)
Select
'TestFormType', 'DRAFT', #modifyDate, #modifyUser
From FormDB2.dbo.Form f
Where f.Pkey = #oldKey
--grab the new int identifier -- works
set #newKey = (select scope_identity())
/*
stuff happens here. all is good in this part
*/
---------------Param---------------
INSERT INTO FormDB.dbo.Param
(FormKey, ParamID, ModifyDate, ModifyUser)
select #newKey, p.ParamID, #modifyDate, #modifyUser
from PROD.dbo.Table1 apd
inner join PROD.dbo.ParameterTable p
on apd.TableTwoKey = p.TableTwoKey
where apd.PKey = #spKey
---------------Method---------------
INSERT INTO FormDB.dbo.Method
(FormKey, MethodID, ModifyDate, ModifyUser)
select #newKey, r.MethodID, #modifyDate, #modifyUser
from PROD.dbo.Table1 apd
inner join PROD.dbo.MethodTable r
on apd.TableTwoKey = r.TableTwoKey
where apd.PKey = #spKey
/*
one more insert ...
*/
RETURN 1
END
GO
I still don't understand the why of this problem, but I found a solution.
There are a lot of things happening in the vb.net code for this process: multiple linq-to-sql inserts/deletes/updates across three separate databases, as well as two separate stored procedures calls. To add to that confusion, there are separate contexts declared for each db, each with its own transaction. In short, a bunch of moving parts.
The second stored proc call was conditional, based on certain values for the processing form. I just took that call out of the vb.net code, and placed the conditional logic and stored proc call within the first proc. That solved it. The second stored proc was called directly after the first anyway -- pending conditions -- so all is good.
Problem solved -- but if anybody can explain why this was occurring, I'd be grateful. Thanks!
I'm using pg_search for some text searching within my model. Among other attributes, I have an url field.
Unfortuantelly Postgres doesn't seem to identify / and . as word separators, therefore I cannot search within the url.
Example: searching for test in http://test.com yields no results.
Is there a way to fix this problem, perhaps using another gem or some inline SQL ?
As stated in the documentation (and noticed by AJcodez), there is a solution in creating a dedicated column for tsvector index. Then define a trigger that catches insertions to index urls properly:
CREATE test_url (url varchar NOT NULL, url_tsvector tsvector NOT NULL);
This method will transorm any non alpha characters into single space and turn the string into a tsvector:
CREATE OR REPLACE FUNCTION generate_url_tsvector(varchar)
RETURNS tsvector
LANGUAGE sql
AS $_$
SELECT to_tsvector(regexp_replace($1, '[^\w]+', ' ', 'gi'));
$_$;
Now create a trigger that calls this function:
CREATE OR REPLACE FUNCTION before_insert_test_url()
RETURNS TRIGGER
LANGUAGE plpgsql AS $_$
BEGIN;
NEW.url_tsvector := generate_url_tsvector(NEW.url);
RETURN NEW;
END;
$_$
;
CREATE TRIGGER before_insert_test_url_trig
BEFORE INSERT ON test_url
FOR EACH ROW EXECUTE PROCEDURE before_insert_test_url();
Now, when url are inserted, the `url_tsvectorè field will be automatically populated.
INSERT INTO test_url (url) VALUES ('http://www.google.fr');
TABLE test_url;
id url url_tsvector
2 http://www.google.fr 'fr':4 'googl':3 'http':1 'www':2
(1 row)
To FT search on URLs you only need to query against this field.
SELECT * FROM test_url WHERE url_tsvector ## 'google'::tsquery;
I ended up modifying the pg_search gem to support arbitrary ts_vector expressions instead of just column names.
The changes are here
Now I can write:
pg_search_scope :search,
against: [[:title , 'B'], ["to_tsvector(regexp_replace(url, '[^\\w]+', ' ', 'gi'))", 'A']],
using: {tsearch: {dictionary: "simple"}}
Slightly simpler approach, add the protocol token type to the simple dictionary:
ALTER TEXT SEARCH CONFIGURATION simple
ADD MAPPING FOR protocol
WITH simple;
you can also add it to the english dictionary if you need stemming
https://www.postgresql.org/docs/13/textsearch-parsers.html
https://www.postgresql.org/docs/13/sql-altertsconfig.html
I'm using SQL Server 2012, is there any possible way to get all the parameters of a stored procedure along with the values passed to it?
I need these things to build a xml. I mean this should happen in the procedure which being executed and it should be common for all the procedures.
For example, let us suppose we have to procedures,
uspSave, #name='test' #age=20
uspDelete #id=2
now in uspSave procedure, I need to get #name, #age and the values 'test', 20
and in uspDelete, I should get #id with value 2.
For getting the column names, I tried this,
select parameter_name
from information_schema.PARAMETERS
where specific_name = OBJECT_NAME(##procid)
now is it possible to loop through the result of above query and can we get the values.
I think your best bet would be to use some code generation to generate the code block you require.
i.e.
Create your sproc without the code to XML-ify the parameters
Knock up a quick script (could be done in TSQL) to then construct the sproc-specific block of TSQL to convert the parameters into XML, using INFORMATION_SCHEMA.PARAMETERS
Copy that bit of auto-generated script into your sproc
The way you were thinking with dynamic sql wouldn't work because of the scope - the parameters would not be accessible within that dynamically generated SQL, you'd need to pass them in as args via sp_executesql, which puts you back in square 1.
e.g.
DECLARE #someval int = 7
EXECUTE('SELECT #someval') -- #someval is not in scope
So, if it will help save time, then code gen looks like your best bet.
I have a rails application and I am trying to load data into it via PostgreSQL's COPY command. The file is CSV. The table maps to a Rails model and I need to retain the primary-key ID column so I can use model.for_each to play with the data--the table will have 5M+ rows.
My question is, how can I structure my CSV so that I can load data into the table and still allow the ID column to be there? It's a bummer because the only reason I need it is for the for_each method.
Regardless, I tried sending NULLs, as in:
NULL,col1,col2,col3,col4, etc.
But it doesn't work.
Thanks.
Passing in null for the primary key will never work no matter what options you have set for null string. You would be telling the backend to set the primary key to null which it will never allow no matter what the command to insert might be.
I really have no idea what you mean by retaining the primary key. That is something that is going be retained no matter what do. If you mean letting the DB pick the value for you and the primary key is a serial (auto-increment) then explicitly name all the columns but the primary key:
COPY country (colA, colB, colC) FROM '/usr1/proj/bray/sql/country_data'; -- leave out pkey
It also might be quicker to read the documentation on what null string options you would like to use instead of guessing possible values:
http://www.postgresql.org/docs/9.0/static/sql-copy.html
The default when using WITH CSV is an unquoted empty string, such as:
,col1,col2,col3,,col5
Which would create a record that looks like:
NULL,'col1','col2','col3',NULL,'col5'
I have a table in my rails project, which contains a column of type string. One of the entries for this column is "one\ntwo". When I try to display the table, that entry is displayed as "one two". I serialized the entry, so I tried displaying an inspected version as well, which displayed "one\ntwo". Is there any way to display that entry as following?
one
two
s.gsub(/\n/, '<br>') try this, and play with sanitize or maybe html_safe