I am new in ODI and facing different issue while I am trying to delete data using procedure.
Steps I followed :
Create a procedure where in source command we kept select query. SELECT DOC_ID FROM SOURCE_TAB
In target command we kept delete query which suppose to delete only unmatched data from TARGET_TAB.
DELETE FROM TARGET_TAB WHERE TRAN_ID NOT IN (:DOC_ID)
But our concern is here, procedure executes successfully and its delete all data from target table where source and target table data has same.
Source table: SOURCE_TAB
DOC_ID NUMBER(10);
====
10
20
30
40
50
Target table : TARGET_TAB
TRAN_ID NUMBER(10);
====
10
20
30
40
50
NOTE: Source and target table data is sale. It should not delete any data.
For this method ODI uses row by row method. For every DOC_ID from the source it executes command on target
DELETE FROM TARGET_TAB WHERE TRAN_ID NOT IN (10)
DELETE FROM TARGET_TAB WHERE TRAN_ID NOT IN (20)
.....
Therefore it deletes all data
If source and target tables are in the same database, use single query to delete data.
DELETE FROM TARGET_TAB WHERE TRAN_ID NOT IN (SELECT DOC_ID FROM SOURCE_TAB)
Related
I have following fields in table 1-
db,schema,jobnm,status,runtime, ins_tstmp, upd_tstmp.
A stream has been created on table 1.
A stored procedure was written to loop through another table's dataset (4 records) and write all 4 records to table 1 if they don't already exist else update (using merge sql here; ins_tstmp gets populated via insert part of merge while upd_tstmp gets updated via update part ).
As expected, table1 has all 4 records and Stream also has 4 records with metadata$action as INSERT . UPD_TSTMP is null here.
Now on 2nd run, same 4 records were retrieved. Since they were a match, upd_tstmp got populated in both table 1 and stream but why metadata$action is INSERT only? Not seeing 2 entries for an update. Could someone please explain what I am missing here?
Thanks
Since they were a match, upd_tstmp got populated in both table 1 and
stream but why metadata$action is INSERT only?
The METADATA$ACTION column can have 2 possible values: INSERT and DELETE. So you can't see "UPDATE" in this column.
METADATA$ISUPDATE: This is an extra column indicating whether the operation was part of an UPDATE statement. In your case, you should also see it "false" because Streams record the differences between two offsets. If a row is added and then updated in the current offset, the delta change is a new row. The METADATA$ISUPDATE row records a FALSE value.
https://docs.snowflake.com/en/user-guide/streams-intro.html#stream-columns
I have an old application I am supporting that uses a Microsoft Access database. The original table design did not add primary keys to every table. I am working on a migration program that among other things is adding and filling in a new primary key field (GUID) when needed.
This is happening in three steps:
Add a new guid field with no constraints
Fill the field with new unique guids
Add the primary key constraints
My problem is setting the unique guids when the table has duplicate rows. Here is my code to set the guids.
Query.SQL.Add('SELECT * FROM ' + TableName);
Query.Open;
while Query.Eof = false do
begin
Query.Edit;
Query.FieldByName(NewPrimaryKeyFieldName).AsGuid := TGuid.NewGuid;
Query.Post;
Query.Next;
end;
FireDac generates an update statement that contains a where clause with all the original fields/values in the row (since there is no unique field for it to use). However, because the rows are complete duplicates the statement still updates two rows.
FireDac correctly errors with this message
Update command updated [2] instead of [1] record.
I can open up the database in Access and delete the duplicate records or assign them a unique guid by editing the table. I would like my conversion tool to automatically do this.
Is there some way to work with these duplicate rows in FireDac? Either to update just one at a time, or to delete just one of them?
In my opinion there is no way to do it with just one SQL Statement.
I would do this:
1. Copy the whole table without duplicates by using a new temp table
SELECT DISTINCT * FROM <TABLENAME>
Add the Keys
Delete old table content and copy new content from new table
Notes:
The DB Should be unavailable for everyone else for that Operation
2. Make BACKUP before
I'm working on data cleansing project where I'm executing 1st stored procedure to get all the data which has issue and store it into a staging table1 with ID, IND_REF, CODE.
Table structure is:
ID | IND_REF|CODE
12 | 2333 |ABC
13 | 1222 |EFG
Now each code associated with IND_ref is primary key of the table2 and email table where data will be updated.
Next I wrote another stored procedure with an IF statement stating,
If code = ABC then update school email as main email where emailtable_ID = staging table IND_REF
Once it update all the row of email table by reference of staging table IND_REF I used another if statement,
IF code = 'EFG' do that.... where table2_ID = staging table IND_REF...
and so on..
Basically I want to update the row of live table by referencing CODE associated with each IND_REF...
Can I achieve this with a SSIS package? Can I loop through the staging table to update the live table? Any help would be much appreciated. I am new to the SQL world so I find it difficult to loop through each record by setting counter to update live table. any help with script would be very helpful.
I don't understand your issue but let me show you an example:
If we have a table like this:
TABLE1
ID ind_ref code
1 1 ABC
2 15 DEF
3 17 GHI
and a table like this:
TABLE2
ind_ref2 code
1 ZZZ
2 XXX
3 DDD
4 ZZZ
5 XXX
15 FFF
17 GGG
Then if we run this query:
UPDATE TABLE2
SET Code = TABLE1.Code
FROM TABLE1
WHERE TABLE1.ind_ref = TABLE2.ind_ref2;
Table 2 will end up like this:
TABLE2
ind_ref2 code
1 ABC <= I got updated
2 XXX
3 DDD
4 ZZZ
5 XXX
15 DEF <= me too
17 GHI <= and me
If this is not your data or your requirement, please take the time to lay out examples as I have: explain the data that you have and what you want it to look like.
Note: SSIS is not required here and neither is looping.
i have three table
1) first table->illnessarea
ID Area
1 Heart
2 Ear
2) Second table ->Specialisation
ID Specialisation
12 Cardiovascular
3) Temp Table
Areaname Specialisationname
Heart Cardiovascular
Want new table using stored procedure
AreaId SpecializationID
1 12
I am not much exp in Stored procedure.
Help Me
I've got two csv files imported into neo4j named : uniq_names and all_names. I've got one column and about 5000 rows in uniq_names file , and i've got three columns : name , id1 and id2 it's about 300000 rows in all_names file.
Now i'm trying to create relationship with code below:
MATCH (a:uniq_names),(b:all_names)
WHERE a.name=b.name AND b.id1<>b.id2
CREATE (a)-[:child]->(b);
as i execute the code it thinks about 20 minutes but returns: " 0 rows returned " as result , and it down't create any relationship.it works perfectly when i've got 1000 rows in all_names file and 50 rows in uniq_names file
I've got windows 7 64bit, jdk 1.7.0_71, neo4j 2.1.6 enterprise. Any ideas?
That query basically creates a cross product of you 5k uniq_names and 300k all_names, so 1.5bn operations - is not very efficient.
To optimize:
Create an index: CREATE INDEX ON :all_names(name). Then first go over all uniq_names and find corresponding all_names via an index lookup, check the id condition and create the relationships:
MATCH (a:uniq_names)
WITH a
MATCH (b:all_names {name: a.name})
WHERE b.id1<>b.id2
CREATE (a)-[:child]->(b);