I use below code to get selected columns. But in the column item, why the table.getName() is alias name t1 or t2 and table.getAlias() is null?
Is any sample code to get the table name(Spark_Test_1, Spark_Test_2) and the alias table name(t1,t2) in the same time?
String sql = "SELECT t1.AsOfD,t1.ValidD,t1.urn,t1.Money FROM Spark_Test_1 as t1 join Spark_Test_2 as t2 on ( t1.AsOfD = t2.AsOfD)";
Statement statement = CCJSqlParserUtil.parse(sqlStr);
Select selectStatement = (Select) statement;
for (int i = 0; i < size; i++) {
Expression expression = ((SelectExpressionItem) selectitems.get(i))
.getExpression();
//System.out.println("Expression:" + expression);
if(expression instanceof Column){
Column col = (Column) expression;
Table table = col.getTable();
logger.info(table.getFullyQualifiedName());
logger.info(table.getAlias());
logger.info(table.getName());
}
}
This is not an issue but normal JSqlParser behaviour. JSqlParser gives you a structured way to look at your SQL but does no semantic processing. It is a parser.
Therefore for a column the tablename is in your example indeed the alias. JSqlParser does not resolve this alias to the real table name. You have to process the from items to get the tablenames its aliases and map it to your columns.
IMHO you should follow the path of TableNamesFinder to build a visitor that extracts your columns and additional gets your tables including name and alias. You have to be careful to use only the tables that are valid within your columns context, e.g.
select data.a from (select a from mydata) as data
Here data is an alias for a subsql and not for a table.
Related
Would you please provide an an example for a Redshift procedure where you have used a cursor and an UPDATE statement in conjunction? Is that even feasible, I couldn't find an example. I'm looking for a simple template code to learn how to have these 2 together in a single procedure on Redshift.
Here is an example use case:
I have a table like this:
CREATE TABLE test_tbl
(
Contactid VARCHAR(500),
sfdc_OppId_01 VARCHAR(500),
sfdc_OppId_02 VARCHAR(500),
sfdc_OppId_03 VARCHAR(500),
sfdc_OppId_04 VARCHAR(500),
sfdc_OppId_05 VARCHAR(500),
sfdc_OppId_06 VARCHAR(500)
)
I want to update each sfdc_OppId_xx with the relative value from another table; sfdc_tbl. Here is what sfdc_tbl looks like:
sfdc_contactId
sfdc_Opp_Id
AA123hgt
999999
AA123hgt
888888
AA123hgt
777777
AA123hgt
432567
AA123hgt
098765
AA123hgt
112789
So as you see, there are duplicate sfdc_contactid in the sfdc_tbl. My final goal is to list all the sfdc_Opp_Id for given contactid horizontally in the test_tbl. I shall not have duplicate contactid in the test_tbl.
INSERT INTO test_tbl (Contactid)
SELECT sfdc_contactId
FROM sfdc_tbl
GROUP BY sfdc_contactId
And here is what I'm trying to do:
CREATE OR REPLACE PROCEDURE testing_procedure (results INOUT refcursor)
AS
$$
BEGIN
OPEN cursor_testing FOR
SELECT
Ops.sfdc_Opp.id,
ROW_NUMBER () OVER(PARTITION BY Ops.sfdc_contactId ORDER BY sfdc_Opp_Id ) RWN
FROM sfdc_tbl Ops
INNER JOIN test_tbl tbl
ON Ops.sfdc_contactId = tbl.contactid;
UPDATE test_tbl
SET sfdc_Opp_01 = CASE WHEN cursor_testing.RWN = 1 THEN cursor_testing.sfdc_Ops_id ELSE NULL END,
sfdc_Opp_02 = CASE WHEN cursor_testing.RWN = 2 THEN cursor_testing.sfdc_Ops_id ELSE NULL END,
sfdc_Opp_03 = CASE WHEN cursor_testing.RWN = 3 THEN cursor_testing.sfdc_Ops_id ELSE NULL END,
sfdc_Opp_04 = CASE WHEN cursor_testing.RWN = 4 THEN cursor_testing.sfdc_Ops_id ELSE NULL END,
sfdc_Opp_05 = CASE WHEN cursor_testing.RWN = 5 THEN cursor_testing.sfdc_Ops_id ELSE NULL END,
sfdc_Opp_06 = CASE WHEN cursor_testing.RWN = 6 THEN cursor_testing.sfdc_Ops_id ELSE NULL END
;
END;
$$
LANGUAGE plpgsql;
I keep getting an error
incorrect syntax at or near "cursor_testing"
I've answered a question with a similar solution. The SQL uses a cursor's data to INSERT into a table and this same path should work for UPDATE - How to join System tables or Information Schema tables with User defined tables in Redshift
That being said and looking at your code I really think you would be better off using a temp table rather than a cursor. The first thing to note is that a cursor is not a table. Your use pattern won't work AFAIK. You read a cursor row by row (or bunches) which is contrary to Redshift's columnar table storage. So you will need to loop on the rows from the cursor and perform N updates. This will be extremely slow! You would be querying columnar data, storing the results in a cursor as rows, reading these row one by one, and then performing a new query (UPDATE) for each row. Ick! Stay in "columnar land" and use a temp table.
I have a table that already has values in it. The value I want to update is g_fuel_prft.billed_qty. I need to multiple a number from this table times a number from another table to get the value.
So table names are inv_header which contains inv_header.rpt_factor and g_fuel_prft which contains g_fuel_prft.billed_qty. The criteria is where inv_header.link=g_fuel_prft.lnk AND inv_header.rpt_factor = 0.
once I have those rows selected I want to update them to billed_qty * rpt_factor
update g_fuel_prft
set billed_qty = (inv_header.rpt_factor * g_fuel_prft.billed_qty)
where exists
(select billed_qty,ivh_rpt_factor from g_fuel_prft,inv_header
where g_fuel_prft.prodlnk = inv_header.ivh_link
and inv_header.ivh_rpt_factor = 0)
I am getting an 201 syntax error
You can't refer to inv_header.rpt_factor like that. You'll need to place it into a sub-query.
UPDATE g_fuel_prft
SET billed_qty = ((SELECT inv_header.rpt_factor
FROM inv_header
WHERE g_fuel_prft.prodlnk = inv_header.ivh_link) *
g_fuel_prft.billed_qty)
WHERE EXISTS (SELECT *
FROM g_fuel_prft
JOIN inv_header ON g_fuel_prft.prodlnk = inv_header.ivh_link
WHERE inv_header.ivh_rpt_factor = 0)
With an EXISTS query, the select-list doesn't matter and * is conventional. You might need more restrictions in the sub-select within the SET clause. It depends on what sort of relationship there is between the joining columns (1:1, 1:N, N:1, M:N).
Given the minimal schema below, the query above is syntactically valid. It's hard to test it without valid sample data.
DROP TABLE IF EXISTS g_fuel_prft;
CREATE TABLE g_fuel_prft
(
prodlnk INTEGER NOT NULL,
billed_qty DECIMAL(8,2) NOT NULL
);
DROP TABLE IF EXISTS inv_header;
CREATE TABLE inv_header
(
rpt_factor DECIMAL(8,4) NOT NULL,
ivh_link INTEGER NOT NULL,
ivh_rpt_factor DECIMAL(8,4) NOT NULL
);
I have two tables that I want to join.
table1 has id and value columns.
table2 has id and color columns.
final = join table1 by id, table2 by id;
dump final;
I received table whose columns are id, value, id, color. But I want to obtain a table which has columns like id, value and color. How can I remove this duplicate id column from this table?
If you do DESCRIBE final; you will see that the schema looks something like this:
final: {table1::id: chararray,table1::value: chararray,table2::id: chararray,table2::color: chararray}
To distinguish between the two ID columns, you can use table1::id or table2::id. So to remove one of the duplicate columns, you can do:
A = FOREACH final GENERATE
table1::id AS id,
table1::value AS value,
table2::color AS color;
(I've also renamed the fields to get rid of the table1:: and table2:: prefixes since they're no longer necessary.)
I could have also done:
A = FOREACH final GENERATE
table1::id AS id,
value AS value,
color AS color;
This would not have given me an error since value and color are unambiguous names.
Execute your final PIG script:
grunt> table1 = LOAD 'table1_input_path' USING PigStorage(',') as (id:int, value:int);
grunt> table2= LOAD 'table2_input_path' USING PigStorage(',') as (id:int, color:chararray);
grunt> joinlevel = JOIN table1 BY id, table2 BY id;
grunt> final = FOREACH joinlevel generate table1::id as id, table1::color as color, table2::value as value;
grunt> dump final;
I'm trying to convert the following SQL statement to Core Data:
delete from SomeTable
where someID not in (
select someID
from SomeTable
group by property1, property2, property3
)
Basically, I want to retrieve and delete possible duplicates in a table where a record is deemed a duplicate if property1, property2 and property3 are equal to another record.
How can I do that?
PS: As the title says, I'm trying to convert the above SQL statement into iOS Core Data methods, not trying to improve, correct or comment on the above SQL, that is beyond the point.
Thank you.
It sounds like you are asking for SQL to accomplish your objective. Your starting query won't do what you describe, and most databases wouldn't accept it at all on account of the aggregate subquery attempting to select a column that is not a function of the groups.
UPDATE
I had initially thought the request was to delete all members of each group containing dupes, and wrote code accordingly. Having reinterpreted the original SQL as MySQL would do, it seems the objective is to retain exactly one element for each combination of (property1, property2, property3). I guess that makes more sense anyway. Here is a standard way to do that:
delete from SomeTable st1
where someID not in (
select min(st2.someId)
from SomeTable st2
group by property1, property2, property3
)
That's distinguished from the original by use of the min() aggregate function to choose a specific one of the someId values to retain from each group. This should work, too:
delete from SomeTable st1
where someID in (
select st3.someId
from SomeTable st2
join SomeTable st3
on st2.property1 = st3.property1
and st2.property2 = st3.property2
and st2.property3 = st3.property3
where st2.someId < st3.someId
)
These two queries will retain the same rows. I like the second better, even though it's longer, because the NOT IN operator is kinda nasty for choosing a small number of elements from a large set. If you anticipate having enough rows to be concerned about scaling, though, then you should try both, and perhaps look into optimizations (for example, an index on (property1, property2, property3)) and other alternatives.
As for writing it in terms of Core Data calls, however, I don't think you exactly can. Core Data does support grouping, so you could write Core Data calls that perform the subquery in the first alternative and return you the entity objects or their IDs, grouped as described. You could then iterate over the groups, skip the first element of each, and call Core Data deletion methods for all the rest. The details are out of scope for the SO format.
I have to say, though, that doing such a job in Core Data is going to be far more costly than doing it directly in the database, both in time and in required memory. Doing it directly in the database is not friendly to an ORM framework such as Core Data, however. This sort of thing is one of the tradeoffs you've chosen by going with an ORM framework.
I'd recommend that you try to avoid the need to do this at all. Define a unique index on SomeTable(property1, property2, property3) and do whatever you need to do to avoid trying to creating duplicates or to gracefully recover from a (failed) attempt to do so.
DELETE SomeTable
FROM SomeTable
LEFT OUTER JOIN (
SELECT MIN(RowId) as RowId, property1, property2, property3
FROM SomeTable
GROUP BY property1, property2, property3
) as KeepRows ON
SomeTable.RowId = KeepRows.RowId
WHERE
KeepRows.RowId IS NULL
A few pointers for doing this in iOS: Before iOS 9 the only way to delete objects is individually, ie you will need to iterate through an array of duplicates and delete each one. (If you are targeting iOS9, there is a new NSBatchDeleteRequest which will help delete them all in one go - it does act directly on the store but also does some cleanup to eg. ensure relationships are updated where necessary).
The other problem is identifying the duplicates. You can configure a fetch to group its results (see the propertiesToGroupBy of NSFetchRequest), but you will have to specify NSDictionaryResultType (so the results are NOT the objects themselves, just the values from the relevant properties.) Furthermore, CoreData will not let you fetch properties (other than aggregates) that are not specified in the GROUP BY. So the suggestion (in the other answer) to use min(someId) will be necessary. (To fetch an expression such as this, you will need to use an NSExpression, embed it in an NSExpressionDescription and pass the latter in propertiesToFetch of the fetch request).
The end result will be an array of dictionaries, each holding the someId value of your prime records (ie the ones you don't want to delete), from which you have then got to work out the duplicates. There are various ways, but none will be very efficient.
So as the other answer says, duplicates are better avoided in the first place. On that front, note that iOS 9 allows you to specify attributes that you would like to be unique (individually or collectively).
Let me know if you would like me to elaborate on any of the above.
Group-wise Maximum:
select t1.someId
from SomeTable t1
left outer join SomeTable t2
on t1.property1 = t2.property1
and t1.property2 = t2.property2
and t1.property3 = t2.property3
and t1.someId < t2.someId
where t2.someId is null;
So, this could be the answer
delete SomeTable
where someId not in
(select t1.someId
from SomeTable t1
left outer join SomeTable t2
on t1.property1 = t2.property1
and t1.property2 = t2.property2
and t1.property3 = t2.property3
and t1.someId < t2.someId
where t2.someId is null);
Sqlfiddle demo
You can use exists function to check for each row if there is another row that exists whose id is not equal to the current row and all other properties that define the duplicate criteria of each row are equal to all the properties of the current row.
delete from something
where
id in (SELECT
sm.id
FROM
sometable sm
where
exists( select
1
from
sometable sm2
where
sm.prop1 = sm2.prop1
and sm.prop2 = sm2.prop2
and sm.prop3 = sm2.prop3
and sm.id != sm2.id)
);
I think you could easily handle this by creating a derived duplicate_flg column and set it to 1 when all three property values are equal. Once that is done, you could just delete those records where duplicate_flg = 1. Here is a sample query on how to do this:
--retrieve all records that has same property values (property1,property2 and property3)
SELECT *
FROM (
SELECT someid
,property1
,property2
,property3
,CASE
WHEN property1 = property2
AND property1 = property3
THEN 1
ELSE 0
END AS duplicate_flg
FROM SomeTable
) q1
WHERE q1.duplicate_flg = 1;
Here is a sample delete statement:
DELETE
FROM something
WHERE someid IN (
SELECT someid
FROM (
SELECT someid
,property1
,property2
,property3
,CASE
WHEN property1 = property2
AND property1 = property3
THEN 1
ELSE 0
END AS duplicate_flg
FROM SomeTable
) q1
WHERE q1.duplicate_flg = 1
);
Simply, if you want to remove duplicate from table you can execute below Query :
delete from SomeTable
where rowid not in (
select max(rowid)
from SomeTable
group by property1, property2, property3
)
if you want to delete all duplicate records try the below code
WITH tblTemp as
(
SELECT ROW_NUMBER() Over(PARTITION BY Property1,Property2,Property3 ORDER BY Property1) As RowNumber,* FROM Table_1
)
DELETE FROM tblTemp where RowNumber >1
Hope it helps
Use the below query to delete the duplicate data from that table
delete from SomeTable where someID not in
(select Min(someID) from SomeTable
group by property1+property2+property3)
I am writing a PL/SQL stored procedure which will be called from within a .NET application.
My stored procedure must return
the count of values in a table of part revisions, based on an input part number,
the name of the lowest revision level currently captured in this table for the input part number
the name of the revision level for a particular unit in the database associated with this part number and an input unit ID.
The unit's revision level name is captured within a separate table with no direct relationship to the part revision table.
Relevant data structure:
Table Part has columns:
Part_ID int PK
Part_Number varchar2(30)
Table Part_Revisions:
Revision_ID int PK
Revision_Name varchar2(100)
Revision_Level int
Part_ID int FK
Table Unit:
Unit_ID int PK
Part_ID int FK
Table Unit_Revision:
Unit_ID int PK
Revision_Name varchar2(100)
With that said, what is the most efficient way for me to query these three data elements into a ref cursor for output? I am considering the following option 1:
OPEN cursor o_Return_Cursor FOR
SELECT (SELECT COUNT (*)
FROM Part_Revisions pr
inner join PART pa on pa.part_id = pr.part_id
WHERE PA.PART_NO = :1 )
AS "Cnt_PN_Revisions",
(select pr1.Revision_Name from Part_Revisions pr1
inner join PART pa1 on pa1.part_id = pr1.part_id
WHERE PA.PART_NO = :1 and pr1.Revision_Level = 0)
AS "Input_Revison_Level",
(select ur.Revision_Name from Unit_Revision ur
WHERE ur.Unit_ID = :2) as "Unit_Revision"
FROM DUAL;
However, Toad's Explain Plan returns Cost:2 Cardinality: 1, which I suspect is due to me using DUAL in my main query. Comparing that to option 2:
select pr.Revision_Name, (select count(*)
from Part_Revisions pr1
where pr1.part_id = pr.part_id) as "Count",
(select ur.Revision_Name
from Unit_Revision ur
where ur.Unit_ID = :2) as "Unit_Revision"
from Part_Revisions pr
inner join PART pa on pa.part_id = pr.part_id
WHERE PA.PART_NO = :1 and pr.Revision_Level = 0
Essentially I don't really know how to compare the results from my execution plans, to chose the best design. I have also considered a version of option 1, where instead of joining twice to the Part table, I select the Part_ID into a local variable, and simply query the Part_Revisions table based on that value. However, this is not something I can use the Explain Plan to analyze.
Your description and select statements look different... I based the procedure on the SQL statements.
PROCEDURE the_proc
(
part_no_in IN NUMBER
, revision_level_in IN NUMBER
, unit_id_in IN NUMBER
, part_rev_count_out OUT NUMBER
, part_rev_name_out OUT VARCHAR2
, unit_rev_name_out OUT VARCHAR2
)
AS
BEGIN
SELECT COUNT(*)
INTO part_rev_count_out
FROM part pa
WHERE pa.part_no = part_no_in
AND EXISTS
(
SELECT 1
FROM part_revisions pr
WHERE pa.part_id = pr.part_id
);
SELECT pr1.revision_name
INTO part_rev_name_out
FROM part_revisions pr1
WHERE pr1.revision_level = revision_level_in
AND EXISTS
(
SELECT 1
FROM part pa1
WHERE pa1.part_id = pr1.part_id
AND pa.part_no = part_no_in
);
SELECT ur.revision_name
INTO unit_rev_name_out
FROM unit_revision ur
WHERE ur.unit_id = unit_id_in;
END the_proc;
It looks like you are obtaining scalar values. Rather than return a cursor, just return the values using clean sql statements. I have done this numerous times from .net, it works fine.
Procedure get_part_info(p_partnum in part.part_number%type
, ret_count out integer
, ret_revision_level out part_revisions.revision_level%type
, ret_revision_name out part_revisions.revision_name%type) as
begin
select count(*) into ret_count from ....;
select min(revision_level) into ret_revision_level from ...;
select revision_name in ret_revision_name...;
return;
end;