How to insert CDC Data from a stream to another table with dynamic column names - stream

I have a Snowflake stored procedure and I want to use "insert into" without hard coding column names.
INSERT INTO MEMBERS_TARGET (ID, NAME)
SELECT ID, NAME
FROM MEMBERS_STREAM;
This is what I have and column names are hardcoded. The query should copy data from MEMBERS_STREAM to MEMBERS_TARGET. The stream has more columns such as
METADATA$ACTION | METADATA$ISUPDATE | METADATA$ROW_ID
which I am not intending to copy.

I don't know of a way to not copy the METADATA columns if not hardcoding. However if you don't want the data maybe the easiest thing to do is to add them to your target, INSERT using a SELECT * and later in the sp set them to NULL.
Alternatively, earlier in your sp, run an ALTER TABLE ADD COLUMN to add the columns, INSERT using SELECT * and then after that run an ALTER TABLE DROP COLUMN to remove the columns? That way your table structure stays the same, albeit briefly it will have some extra columns.
A SELECT * is not usually recommended but it's the easiest alternative I can think of

Related

Using FireDac to update only 1 of a duplicate row (no primary key or unique field)

I have an old application I am supporting that uses a Microsoft Access database. The original table design did not add primary keys to every table. I am working on a migration program that among other things is adding and filling in a new primary key field (GUID) when needed.
This is happening in three steps:
Add a new guid field with no constraints
Fill the field with new unique guids
Add the primary key constraints
My problem is setting the unique guids when the table has duplicate rows. Here is my code to set the guids.
Query.SQL.Add('SELECT * FROM ' + TableName);
Query.Open;
while Query.Eof = false do
begin
Query.Edit;
Query.FieldByName(NewPrimaryKeyFieldName).AsGuid := TGuid.NewGuid;
Query.Post;
Query.Next;
end;
FireDac generates an update statement that contains a where clause with all the original fields/values in the row (since there is no unique field for it to use). However, because the rows are complete duplicates the statement still updates two rows.
FireDac correctly errors with this message
Update command updated [2] instead of [1] record.
I can open up the database in Access and delete the duplicate records or assign them a unique guid by editing the table. I would like my conversion tool to automatically do this.
Is there some way to work with these duplicate rows in FireDac? Either to update just one at a time, or to delete just one of them?
In my opinion there is no way to do it with just one SQL Statement.
I would do this:
1. Copy the whole table without duplicates by using a new temp table
SELECT DISTINCT * FROM <TABLENAME>
Add the Keys
Delete old table content and copy new content from new table
Notes:
The DB Should be unavailable for everyone else for that Operation
2. Make BACKUP before

How to change the sql column data type in iOS?

In my iOS app,I want to change the column data type in database.
ALTER TABLE XXX ALTER COLUMN myColumn INT.
I always get 'near ALTER Syntax error'
How to resolve the problem?
Thanks for your help.
You cannot change the column type. You can create a new table, using the correct data type for the column this time, and then select data from the old table and insert it into the new table. The full procedure is outlined in the ALTER TABLE documentation:
Remember the format of all indexes and triggers associated with table X. This information will be needed in step 7 below. One way to do this is to run a query like the following: SELECT type, sql FROM sqlite_master WHERE tbl_name='X'.
Use CREATE TABLE to construct a new table "new_X" that is in the desired revised format of table X. Make sure that the name "new_X" does not collide with any existing table name, of course.
Transfer content from X into new_X using a statement like: INSERT INTO new_X SELECT ... FROM X.
If foreign key constraints are enabled, disable them using PRAGMA foreign_keys=OFF.
Drop the old table X: DROP TABLE X.
Change the name of new_X to X using: ALTER TABLE new_X RENAME TO X.
Use CREATE INDEX and CREATE TRIGGER to reconstruct indexes and triggers associated with table X. Perhaps use the old format of the triggers and indexes saved from step 1 above as a guide, making changes as appropriate for the alteration.
If foreign key constraints were originally enabled (prior to step 4) then run PRAGMA foreign_key_check to verify that the schema change did not break any foreign key constraints, and run PRAGMA foreign_keys=ON to re-enable foreign key constraints.
If any views refer to table X in a way that is affected by the schema change, then drop those views using DROP VIEW and recreate them with whatever changes are necessary to accommodate the schema change using CREATE VIEW.
Note, SQLite uses type affinity (the column definition doesn't alter what type of data you insert into the table). So if you change the data type, you'll want to change the data, too.
ALTER TABLE table ADD newColumn INTEGER;
UPDATE table SET newColumn = oldColumn;

DROP Syntax error in SQLiteManager

I have downloaded sqlite manager app in Firefox browser and open sqlite manager app.
1) Created database named DBSQLTEST.
2) Created table named SQLTEST contain has threes fields are SLNO, NAME and AGE
3) Inserted new records
But i want remove a ‘AGE’ column in sqltest table
i using sql command like below
ALTER TABLE SQLTEST DROP COLUMN AGE
SQLiteManager message says
SQLiteManager: Likely SQL syntax error: ALTER TABLE SQLTEST DROP COLUMN AGE [ near "DROP": syntax error ]
Exception Name: NS_ERROR_FAILURE
Exception Message: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [mozIStorageConnection.createStatement]
What is error that?
sqlite does not support DROP COLUMN in ALTER TABLE. You can only rename tables and add columns.
Reference: http://www.sqlite.org/lang_altertable.html
If you need to remove columns, create a new table, copy the data there, drop the old table and rename the table to its intented name.
SQLite does not fully support ALTER TABLE statements. You can only rename table, or add columns.
If you want to drop a column, your best option is to create a new table without the column, and to drop the old table in order to rename the new one.
For example, suppose you have a table named "t1" with columns names "a", "b", and "c" and that you want to delete column "c" from this table. The following steps illustrate how this could be done:
BEGIN TRANSACTION;
CREATE TEMPORARY TABLE t1_backup(a,b);
INSERT INTO t1_backup SELECT a,b FROM t1;
DROP TABLE t1;
CREATE TABLE t1(a,b);
INSERT INTO t1 SELECT a,b FROM t1_backup;
DROP TABLE t1_backup;
COMMIT;
Credits to Benoit.
Create a table
CREATE TABLE SQLTEST ("SLNO" INTEGER PRIMARY KEY ,"NAME" TEXT,AGE INTEGER)
If want to remove a column
ALTER TABLE SQLTEST RENAME TO SQLTESTCOPY
CREATE TABLE SQLTEST ("SLNO" INTEGER PRIMARY KEY ,"NAME" TEXT)
INSERT INTO SQLTEST"SELECT "SLNO","NAME" FROM SQLTESTCOPY
DROP TABLE SQLTESTCOPY
easy.

Firebird: simulating create table as?

I'm searching a way to simulate "create table as select" in Firebird from SP.
We are using this statement frequently in another product, because it is very easy for make lesser, indexable sets, and provide very fast results in server side.
create temp table a select * from xxx where ...
create indexes on a ...
create temp table b select * from xxx where ...
create indexes on b ...
select * from a
union
select * from b
Or to avoid the three or more levels in subqueries.
select *
from a where id in (select id
from b
where ... and id in (select id from c where))
The "create table as select" is very good cos it's provide correct field types and names so I don't need to predefine them.
I can simulate "create table as" in Firebird with Delphi as:
Make select with no rows, get the table field types, convert them to create table SQL, run it, and make "insert into temp table " + selectsql with rows (without order by).
It's ok.
But can I create same thing in a common stored procedure which gets a select sql, and creates a new temp table with the result?
So: can I get query result's field types to I can create field creator SQL from them?
I'm just asking if is there a way or not (then I MUST specify the columns).
Executing DDL inside stored procedure is not supported by Firebird. You could do it using EXECUTE STATEMENT but it is not recommended (see the warning in the end of "No data returned" topic).
One way to do have your "temporary sets" would be to use (transaction-level) Global Temporary Table. Create the GTT as part of the database, with correct datatypes but without constraints (those would probably get into way when you fill only some columns, not all) - then each transaction only sees it's own version of the table and data...

INSERT INTO with unknown number of column in scriptella

I have to back up a table, which can change number of column. When my etl script starts it doesn't know the number of column. How can I create INSERT INTO table VALUES (?1, ?2, ...) script on the fly?
Regards,
Jacek
Depending on the database, one can use CREATE TABLE FROM SELECT (or similar) to back up the table. Example:
CREATE TABLE new_table AS (SELECT * FROM old_table);

Resources