I am trying to use SQLWorkBenchJ to add partition to my table in Amazon Athena. The query works fine when run in the Athena Query Editor. It does not work when I run it using SQLWorkbench
ALTER TABLE "AwsDataCatalog".mydb.mytable ADD IF NOT EXISTS
PARTITION (folder = '10036', full = 'fullvalue') LOCATION 's3://my-s3-folder/10036/fullvalue/';
Note: I have also tried using ALTER EXTERNAL_TABLE
sqlworkbench just returns:
An error occurred when executing the SQL command:
ALTER TABLE "AwsDataCatalog".mydb.mytable ADD PARTITION (folder = '10036', full >= 'fullvalue') LOCATION 's3://my-s3-folder...
Failed to run query
1 statement failed.
running in query editor
ALTER TABLE mydb.mytable ADD PARTITION (folder = '10039', full = 'fullvalue') >LOCATION 's3://my-s3-folder/10039/fullvalue/';
Query successful.
any body know how to make this work so I can batch some add partition queries together?
If you want to batch add partitions you can repeat the part including and after PARTITION as many times as you need:
ALTER TABLE mydb.mytable ADD IF NOT EXISTS
PARTITION (folder = '10036', full = 'fullvalue') LOCATION 's3://my-s3-folder/10036/fullvalue/'
PARTITION (folder = '10037', full = 'fullvalue') LOCATION 's3://my-s3-folder/10037/fullvalue/'
PARTITION (folder = '10038', full = 'fullvalue') LOCATION 's3://my-s3-folder/10038/fullvalue/';
Skip the "AwsDataCatalog" part of the table name, it should not be needed, and is probably the reason for your problem in SQLWorkbench.
I have not had success qualifying the database and table names with "AwsDataCatalog" in ALTER or SHOW PARTITION statements (but it worked in a SELECT). How about just:
ALTER TABLE "AwsDataCatalog".mydb.mytable ADD IF NOT EXISTS
PARTITION (folder = '10036', full = 'fullvalue')
LOCATION 's3://my-s3-folder/10036/fullvalue/';
Would that work for you?
Related
With the setting SHOW_INITIAL_ROWS = TRUE, we created a stream on top of a view (which has many joins).
We created a Stored procedure with a single merge statement that ingests all of the data from the stream into a target table. Following is the merge statement used by the stored procedure.
merge into target tgt
using
(
select id,fname,metadata$action,metadata$isupdate
from emp_stream where not(metadata$action = 'DELETE' and metadata$isupdate = 'TRUE')
) src
on src.id = tgt.id
when matched and metadata$action = 'DELETE' and metadata$isupdate = 'FALSE' then delete
when matched and metadata$action = 'INSERT' and metadata$isupdate = 'TRUE' then update
set tgt.id = src.id
,tgt.fname = src.fname
when not matched and metadata$action = 'INSERT' and metadata$isupdate = 'FALSE' then
insert (id,fname) values (src.id,src.fname);
A task was created to run the stored procedure for every 8 hours. It ran successfully for the first time, i.e. the full load, which inserts all of the records from the view into the target table. However, the second load failed due to a duplicate row error. When we queried the stream, we found two records with the same PK(i.e., id) but different metadata$rowids and metadata$actions of insert and delete, with metadata$isupdate set to false for each.
If it is an update, the metadata$isupdate parameter should be set to true which is not the case here.
Could someone please assist us with this?
Trying to do Incremental load using Streams in snowflake but facing duplicate row error.
SELECT
cntdpts."PROJECT_SID",
cntdpts."USER_SID",
"CNTDPTS",
"CNTQUERIES"
FROM (
SELECT
"PROJECT_SID",
"USER_SID",
COUNT("DATA_POINT_SID") AS "CNTDPTS"
FROM
CNTDPTS
GROUP BY
"PROJECT_SID",
"USER_SID" WITH HINT(RESULT_CACHE) ) cntdpts
INNER JOIN (
SELECT
"PROJECT_SID",
"USER_SID",
COUNT("QUERY_SID") AS "CNTQUERIES"
FROM
CNTQUERIES
GROUP BY
"PROJECT_SID",
"USER_SID" WITH HINT(RESULT_CACHE) ) cntqueries ON
cntdpts."PROJECT_SID" = cntqueries."PROJECT_SID"
AND cntdpts."USER_SID" = cntqueries."USER_SID" WITH HINT(RESULT_CACHE)
I am having troubles with using cached table functions. If I run the two subqueries "cntdpts" and "cntqueries" individually they return the result within <100ms (because they use the cache of the table function CNTDPTS and CNTQUERIES. However if I run the full query with joining the two subqueries it takes >5s and HANA does not seem to take advantage of the cached results from the subqueries. Is there any HINT I still need to add maybe?
You will need to add WITH HINT(RESULT_CACHE_NON_TRANSACTIONAL) to your outermost query.
See also https://help.sap.com/viewer/9de0171a6027400bb3b9bee385222eff/2.0.05/en-US/3ad0e93de0aa408e9238fa862e4780df.html
I have a sqlite database that I'm trying to build a query. The table column I need to retrieve is iEDLID from the table below :
Right now all I have to go on is a known iEventID from the table below :
And the the nClientLocationID from the table below.
So the requirements are I need to get current iEDLID to write, lookup from tblEventDateLocations for dEventDate and the tblLocation.nClientLocationID based on the tblLocations.iLocationID I already have and event selected on this screen.
So I would need a query that does a "SELECT DISTINCT table EventDateLocations.iEDLID FROM tblEventDateLocations ...."
So basically from another query I have the iEventID I need, and I have the event ID i need but where the dEventDate=(select date('now')) I need to retrieve the iEventDateID from table EventDates.iEventDateID to use on the table EventDateLocations
this is the point where I'm trying to wrap my head around the joins for this query and the syntax...
It seems like you want this:
select distinct edl.iEDLDID
from
tblEventDateLocations edl
join tblEventDates ed on edl.EventDateId = ed.EventDateId
where
ed.EventId = ?
and ed.dEventDate = date('now')
and edl.nClientLocationID = ?
where the ? of course represent the known event ID and location ID parameters.
Since nClientLocationId appears on table tblEventDateLocations you do not need to join table tblLocations unless you want to filter out results whose location ID does not appear in that table.
I want to update whole table from on query. Following is my update functionality is happened:
Database (Database A) stored in the iDevice
Temperory Database (Batabase B) downloads to the device and store in the temp folder inside the device. (Both DB has same database structure)
First I attach temp db to device database. Attached db name is SECOND1
Then I insert new records from temp db to device folder from following Insert code. It is working fine.
INSERT INTO main.fList SELECT * FROM SECOND1.fList WHERE not exists (SELECT 1 FROM main.fList WHERE main.fList.GUID = SECOND1.fList.GUID)
But when I use following code to update it is not working fine. It update same top value for all device db table's rows.
UPDATE fList SET Notes = (SELECT SECOND1.fList.Notes FROM SECOND1.fList WHERE SECOND1.fList.GUID = fDefectList.GUID) WHERE EXISTS (SELECT * FROM SECOND1.fList WHERE SECOND1.fList.GUID = fList.GUID
I found SQL query for bulk update. Following is the code,
UPDATE fTempRank7
SET
fTempRank7.int_key = fRank7.int_key,
fTempRank7.int_rank6 = fRank7.int_rank6,
fTempRank7.title = fRank7.title,
fTempRank7.sequence = fRank7.sequence,
fTempRank7.lastupdated = fRank7.lastupdated
FROM
fTempRank7 INNER JOIN fRank7 ON
fTempRank7.int_key = fRank7.int_key
But in sqlite this code does not work.
Anyone knows bulk update in sqlite?
SQLite does not support joins in an UPDATE statement.
When you use a table name without a database name, it refers to the first matching table in the innermost query. In other words, if you use SECOND1.fList in a subquery, any other occurrence of fList refers to the table in SECOND1.
To ensure that you always refer to the correct table, use the database name in all table references.
The main database is always named main, so all table references should be either main.fList or SECOND1.fList.
Anyway, if you are updating all columns, you can simplify the update by deleting the rows that would be updated, so that all new data can just be inserted:
BEGIN;
DELETE FROM fList WHERE GUID IN (SELECT GUID FROM SECOND1.fList);
INSERT INTO fList SELECT * FROM SECOND1.fList;
COMMIT;
When you have a UNIQUE constraint on the GUID column, this can be simplified into a single statement:
INSERT OR REPLACE INTO fList SELECT * FROM SECOND1.fList;
And here I don't use main. because I know what I'm doing. ☺
I'm searching a way to simulate "create table as select" in Firebird from SP.
We are using this statement frequently in another product, because it is very easy for make lesser, indexable sets, and provide very fast results in server side.
create temp table a select * from xxx where ...
create indexes on a ...
create temp table b select * from xxx where ...
create indexes on b ...
select * from a
union
select * from b
Or to avoid the three or more levels in subqueries.
select *
from a where id in (select id
from b
where ... and id in (select id from c where))
The "create table as select" is very good cos it's provide correct field types and names so I don't need to predefine them.
I can simulate "create table as" in Firebird with Delphi as:
Make select with no rows, get the table field types, convert them to create table SQL, run it, and make "insert into temp table " + selectsql with rows (without order by).
It's ok.
But can I create same thing in a common stored procedure which gets a select sql, and creates a new temp table with the result?
So: can I get query result's field types to I can create field creator SQL from them?
I'm just asking if is there a way or not (then I MUST specify the columns).
Executing DDL inside stored procedure is not supported by Firebird. You could do it using EXECUTE STATEMENT but it is not recommended (see the warning in the end of "No data returned" topic).
One way to do have your "temporary sets" would be to use (transaction-level) Global Temporary Table. Create the GTT as part of the database, with correct datatypes but without constraints (those would probably get into way when you fill only some columns, not all) - then each transaction only sees it's own version of the table and data...