In power query ( version included with exel 2016, PC ), is it possible to refer to a computed column of a related table?
Say I have an sqlite database as follow
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE products (
iddb INTEGER NOT NULL,
px FLOAT,
PRIMARY KEY (iddb)
);
INSERT INTO "products" VALUES(0,0.0);
INSERT INTO "products" VALUES(1,1.1);
INSERT INTO "products" VALUES(2,2.2);
INSERT INTO "products" VALUES(3,3.3);
INSERT INTO "products" VALUES(4,4.4);
CREATE TABLE sales (
iddb INTEGER NOT NULL,
quantity INTEGER,
product_iddb INTEGER,
PRIMARY KEY (iddb),
FOREIGN KEY(product_iddb) REFERENCES products (iddb)
);
INSERT INTO "sales" VALUES(0,0,0);
INSERT INTO "sales" VALUES(1,1,1);
INSERT INTO "sales" VALUES(2,2,2);
INSERT INTO "sales" VALUES(3,3,3);
INSERT INTO "sales" VALUES(4,4,4);
INSERT INTO "sales" VALUES(5,5,0);
INSERT INTO "sales" VALUES(6,6,1);
INSERT INTO "sales" VALUES(7,7,2);
INSERT INTO "sales" VALUES(8,8,3);
INSERT INTO "sales" VALUES(9,9,4);
COMMIT;
basically we have products ( iddb, px ) and sales of those products ( iddb, quantity, product_iddb )
I load this data in power query by:
A. creating an ODBC data source using SQLITE3 driver : testDSN
B. in excel : data / new query , feeding this connection string Provider=MSDASQL.1;Persist Security Info=False;DSN=testDSN;
Now in power query I add a computed column, say px10 = px * 10 to the product table.
In the sales table, I can expand the product table into product.px, but not product.px10 . Shouldn't it be doable? ( in this simplified example I could expand first product.px and then create the px10 column in the sales table, but then any new table needinng px10 from product would require me to repeat the work... )
Any inputs appreciated.
I would add a Merge step from the sales query to connect it to the product query (which will include your calculated column). Then I would expand the Table returned to get your px10 column.
This is instead of expanding the Value column representing the product SQL table, which gets generated using the SQL foreign key.
You will have to come back and add any further columns added to the product query to the expansion list, but at least the column definition is only in one place.
In functional programming you don't modify existing values, only create new values. When you add the new column to product it creates a new table, and doesn't modify the product table that shows up in related tables. Adding a new column over product can't show up in Odbc's tables unless you apply that transformation to all related tables.
What you could do is generalize the "add a computed column" into a function that takes a table or record and adds the extra field. Then just apply that over each table in your database.
Here's an example against Northwind in SQL Server
let
Source = Sql.Database(".", "Northwind_Copy"),
AddAColumn = (data) => if data is table then Table.AddColumn(data, "UnitPrice10x", each [UnitPrice] * 10)
else if data is record then Record.AddField(data, "UnitPrice10x", data[UnitPrice] * 10)
else data,
TransformedSource = Table.TransformColumns(Source, {"Data", (data) => if data is table then Table.TransformColumns(data, {"Products", AddAColumn}, null, MissingField.Ignore) else data}),
OrderDetails = TransformedSource{[Schema="dbo",Item="Order Details"]}[Data],
#"Expanded Products" = Table.ExpandRecordColumn(OrderDetails, "Products", {"UnitPrice", "UnitPrice10x"}, {"Products.UnitPrice", "Products.UnitPrice10x"})
in
#"Expanded Products"
Related
I'm working with very large tables in Hive so I'd like to avoid having to create a whole new table when joining a single column from Table 2 to Table 1.
My first pass using the INSERT and UPDATE statements with the following test data didn't work.
Is there a way to do this or is it simpler to just create Table 3 by joining Table 1 to Table 2 and then dropping Table 1?
DROP TABLE IF EXISTS table_1;
CREATE TABLE table_1 (id VARCHAR(64), cost INT, diag_cd VARCHAR(64));
INSERT INTO TABLE table_1
VALUES ('A0001', 1000, 'A1'), ('A0001', 2000, 'B1'), ('A0001', 3000, 'B1'),
('B0001', 5000, 'A1'), ('B0001', 10000, 'B1'), ('B0001', 15000, 'C1'),
('C0001', 11000, 'B1'), ('C0001', 14000, 'C1'), ('C0001', 20000, 'C1');
DROP TABLE IF EXISTS table_2;
CREATE TABLE table_2 (id VARCHAR(64), prodt_cd VARCHAR(64));
INSERT INTO TABLE table_2
VALUES ('A0001', 'OAP'), ('B0001', 'OAPIN'), ('C0001', 'MOAPIN');
INSERT INTO TABLE table_1 prodt_cd VARCHAR(64);
UPDATE table_1 t1 SET t1.prodt_cd = t2.prodt_cd
INNER JOIN table_2 t2
ON t1.id = t2.id;
After some research and help from Mike67 I found a solution.
It appears Hive does not support COLUMN UPDATE or MERGE statements, but a simple alternative is to create an empty table and then populate it with fields from a join:
DROP TABLE IF EXISTS table_3;
CREATE TABLE table_3 LIKE table_1;
INSERT INTO TABLE table_3
SELECT a.*, b.prodt_cd
FROM table_1 AS a
LEFT OUTER JOIN table_2 AS b
ON a.id = b.id;
I have a table that already has values in it. The value I want to update is g_fuel_prft.billed_qty. I need to multiple a number from this table times a number from another table to get the value.
So table names are inv_header which contains inv_header.rpt_factor and g_fuel_prft which contains g_fuel_prft.billed_qty. The criteria is where inv_header.link=g_fuel_prft.lnk AND inv_header.rpt_factor = 0.
once I have those rows selected I want to update them to billed_qty * rpt_factor
update g_fuel_prft
set billed_qty = (inv_header.rpt_factor * g_fuel_prft.billed_qty)
where exists
(select billed_qty,ivh_rpt_factor from g_fuel_prft,inv_header
where g_fuel_prft.prodlnk = inv_header.ivh_link
and inv_header.ivh_rpt_factor = 0)
I am getting an 201 syntax error
You can't refer to inv_header.rpt_factor like that. You'll need to place it into a sub-query.
UPDATE g_fuel_prft
SET billed_qty = ((SELECT inv_header.rpt_factor
FROM inv_header
WHERE g_fuel_prft.prodlnk = inv_header.ivh_link) *
g_fuel_prft.billed_qty)
WHERE EXISTS (SELECT *
FROM g_fuel_prft
JOIN inv_header ON g_fuel_prft.prodlnk = inv_header.ivh_link
WHERE inv_header.ivh_rpt_factor = 0)
With an EXISTS query, the select-list doesn't matter and * is conventional. You might need more restrictions in the sub-select within the SET clause. It depends on what sort of relationship there is between the joining columns (1:1, 1:N, N:1, M:N).
Given the minimal schema below, the query above is syntactically valid. It's hard to test it without valid sample data.
DROP TABLE IF EXISTS g_fuel_prft;
CREATE TABLE g_fuel_prft
(
prodlnk INTEGER NOT NULL,
billed_qty DECIMAL(8,2) NOT NULL
);
DROP TABLE IF EXISTS inv_header;
CREATE TABLE inv_header
(
rpt_factor DECIMAL(8,4) NOT NULL,
ivh_link INTEGER NOT NULL,
ivh_rpt_factor DECIMAL(8,4) NOT NULL
);
I am new to Gupta Sql Base. I would like to know how to get the last inserted record in Gupta SQL
If you are using SYSDBSequence.NextVal to generate your Primary Key, either within the Insert stmt, or prior to the Insert , then you can retrieve it back immediately after the Insert by Selecting Where [Primary Key] = SYSDBSequence.Currval e.g.
Select Name from Patient Where Patient_Id = SYSDBSequence.Currval
Alternatively, If your Primary Key column has been defined as AUTO_INCREMENT , you can select it back after the Insert using MAX( [Primary Key ] ) e.g.
Select Name from Patient Where Patient_Id = (Select MAX( Patient_Id) from Patient )
Alternatively, if none of the above, then write an Insert Trigger to either return it , or to store the PK in a table so you will always have the latest PK recorded for you.
You may like to join the Gupta users forum at enter link description here or there is much archived information at enter link description here
I have downloaded sqlite manager app in Firefox browser and open sqlite manager app.
1) Created database named DBSQLTEST.
2) Created table named SQLTEST contain has threes fields are SLNO, NAME and AGE
3) Inserted new records
But i want remove a ‘AGE’ column in sqltest table
i using sql command like below
ALTER TABLE SQLTEST DROP COLUMN AGE
SQLiteManager message says
SQLiteManager: Likely SQL syntax error: ALTER TABLE SQLTEST DROP COLUMN AGE [ near "DROP": syntax error ]
Exception Name: NS_ERROR_FAILURE
Exception Message: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [mozIStorageConnection.createStatement]
What is error that?
sqlite does not support DROP COLUMN in ALTER TABLE. You can only rename tables and add columns.
Reference: http://www.sqlite.org/lang_altertable.html
If you need to remove columns, create a new table, copy the data there, drop the old table and rename the table to its intented name.
SQLite does not fully support ALTER TABLE statements. You can only rename table, or add columns.
If you want to drop a column, your best option is to create a new table without the column, and to drop the old table in order to rename the new one.
For example, suppose you have a table named "t1" with columns names "a", "b", and "c" and that you want to delete column "c" from this table. The following steps illustrate how this could be done:
BEGIN TRANSACTION;
CREATE TEMPORARY TABLE t1_backup(a,b);
INSERT INTO t1_backup SELECT a,b FROM t1;
DROP TABLE t1;
CREATE TABLE t1(a,b);
INSERT INTO t1 SELECT a,b FROM t1_backup;
DROP TABLE t1_backup;
COMMIT;
Credits to Benoit.
Create a table
CREATE TABLE SQLTEST ("SLNO" INTEGER PRIMARY KEY ,"NAME" TEXT,AGE INTEGER)
If want to remove a column
ALTER TABLE SQLTEST RENAME TO SQLTESTCOPY
CREATE TABLE SQLTEST ("SLNO" INTEGER PRIMARY KEY ,"NAME" TEXT)
INSERT INTO SQLTEST"SELECT "SLNO","NAME" FROM SQLTESTCOPY
DROP TABLE SQLTESTCOPY
easy.
I am having a scenario, where I am having 5 different tables:
Table 1 - Product, Columns - ProductId, BatchNummer, Status, GroupId, OrderNummer
Table 2 - ProductGrop, Columns - GropId, ProductType, Description
Table 3 - Electronics, Columns - EId, Description, BatchNummer, OrderNummer, OrderData
Table 4 - Manual, Columns - MId, Description, Status, OrderNummer, ProcessStep
Table 5 - ProcessedProduct, columns same as Product with one extra column of datetime
Now, according to business flow, I need to populate all the data from Product table, and have to check if the underlying table (Electronics or Manual, which depends on ProductType column of ProductGoup) has ordernuumer value, then Insert a record in table 5 "ProcessedProduct" else skip the records.
For this requirement, i want to create a procedure. But I am stuck on how to check which underlying table (Electronics/Manual) shall i have to refer and how it can be achieved.
Moreover how should i write the loop for inserting the records.
Note: I cannot change the tables schema.
With a PL/SQL procedure you can just switch within a LOOP, but you don't need an imperative algoritm if you just need to check if OrderNummer is either into Electronics or Manuals.
Supposing the detail table is chosen by ProductType value either "Electronics" or "Manuals", you could:
INSERT INTO ProcessedProduct (ProductId, BatchNummer, Status, GroupId, OrderNummer, TS)
SELECT ProductId, BatchNummer, Status, GroupId, OrderNummer, SYSDATE
FROM Product p
INNER JOIN ProductGroup pg USING (GroupId)
WHERE EXISTS (
SELECT NULL FROM Electronics e
WHERE p.OrderNummer = e.OrderNummer
AND pg.ProductType = 'Electronics'
UNION
SELECT NULL FROM Manuals m
WHERE m.OrderNummer = m.OrderNummer
AND pg.ProductType = 'Manuals')
Plain SQL is always the fastest way, and "WHERE EXISTS" is usually the fastest condition.