Symfony Propel Query - symfony1

I want to make this query on Symfony/Propel
SELECT
folders.NAME,
COUNT(documents.NAME),
COUNT(files.idfiles),
SUM(files.size)
FROM `folders`
LEFT JOIN `documents_has_folders` ON (documents_has_folders.folders_idfolders = folders.idfolders)
LEFT JOIN documents ON (documents_has_folders.DOCUMENTS_IDDOCUMENTS=documents.IDDOCUMENTS)
LEFT JOIN files ON (documents.IDDOCUMENTS=files.DOCUMENTS_IDDOCUMENTS)
GROUP BY folders.idfolders
I do this query
$x = FoldersQuery::create()
->addSelectColumn(FoldersPeer::NAME)
->addSelectColumn("COUNT(".DocumentsPeer::IDDOCUMENTS.")")
->addSelectColumn("COUNT(".FilesPeer::IDFILES.")")
->addSelectColumn("SUM(".FilesPeer::SIZE.")")
->addJoin(DocumentsHasFoldersPeer::FOLDERS_IDFOLDERS, FoldersPeer::IDFOLDERS, CRITERIA::LEFT_JOIN)
->addJoin(DocumentsHasFoldersPeer::DOCUMENTS_IDDOCUMENTS, DocumentsPeer::IDDOCUMENTS, CRITERIA::LEFT_JOIN)
->addJoin(DocumentsPeer::IDDOCUMENTS, FilesPeer::DOCUMENTS_IDDOCUMENTS, CRITERIA::LEFT_JOIN)
->addGroupByColumn(FoldersPeer::IDFOLDERS)
->find();
This returns:
500 | Internal Server Error | PropelException
Unable to execute SELECT statement [SELECT folders.NAME, COUNT(documents.IDDOCUMENTS), COUNT(files.IDFILES), SUM(files.SIZE) FROM LEFT JOIN folders ON (documents_has_folders.FOLDERS_IDFOLDERS=folders.IDFOLDERS) LEFT JOIN documents ON (documents_has_folders.DOCUMENTS_IDDOCUMENTS=documents.IDDOCUMENTS) LEFT JOIN files ON (documents.IDDOCUMENTS=files.DOCUMENTS_IDDOCUMENTS) WHERE folders.REMOVE_DATE IS NULL GROUP BY folders.IDFOLDERS] [wrapped: SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'LEFT JOIN folders ON (documents_has_folders.FOLDERS_IDFOLDERS=folders.IDFOLDERS)' at line 1]
Why there is no table on FROM??? Why it creates a wrong query?
Thanks a lot.
EDITED:
Same problem have and if i do the query like this:
$c = new Criteria();
$c->addSelectColumn(FoldersPeer::NAME);
$c->addSelectColumn("COUNT(".DocumentsPeer::IDDOCUMENTS.")");
$c->addSelectColumn("COUNT(".FilesPeer::IDFILES.")");
$c->addSelectColumn("SUM(".FilesPeer::SIZE.")");
$c->addJoin(DocumentsHasFoldersPeer::FOLDERS_IDFOLDERS, FoldersPeer::IDFOLDERS, CRITERIA::LEFT_JOIN);
$c->addJoin(DocumentsHasFoldersPeer::DOCUMENTS_IDDOCUMENTS, DocumentsPeer::IDDOCUMENTS, CRITERIA::LEFT_JOIN);
$c->addJoin(DocumentsPeer::IDDOCUMENTS, FilesPeer::DOCUMENTS_IDDOCUMENTS, CRITERIA::LEFT_JOIN);
$c->addGroupByColumn(FoldersPeer::IDFOLDERS);
Again after FROM there is no table...
Unable to execute SELECT statement [SELECT folders.NAME, COUNT(documents.IDDOCUMENTS), COUNT(files.IDFILES), SUM(files.SIZE) FROM LEFT JOIN folders ON (documents_has_folders.FOLDERS_IDFOLDERS=folders.IDFOLDERS AND folders.remove_date IS NULL ) LEFT JOIN documents ON (documents_has_folders.DOCUMENTS_IDDOCUMENTS=documents.IDDOCUMENTS AND documents.remove_date IS NULL ) LEFT JOIN files ON (documents.IDDOCUMENTS=files.DOCUMENTS_IDDOCUMENTS AND documents.remove_date IS NULL AND files.remove_date IS NULL ) WHERE folders.remove_date IS NULL AND documents.remove_date IS NULL AND files.remove_date IS NULL GROUP BY folders.IDFOLDERS] [wrapped: SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'LEFT JOIN folders ON (documents_has_folders.FOLDERS_IDFOLDERS=folders.IDFOLDERS ' at line 1]

This is not the Propel way to build queries since Propel 1.6 (maybe before). We don't use Criteria nor Peer classes. The following query:
SELECT
folders.NAME,
COUNT(documents.NAME),
COUNT(files.idfiles),
SUM(files.size)
FROM `folders`
LEFT JOIN `documents_has_folders` ON (documents_has_folders.folders_idfolders = folders.idfolders)
LEFT JOIN documents ON (documents_has_folders.DOCUMENTS_IDDOCUMENTS=documents.IDDOCUMENTS)
LEFT JOIN files ON (documents.IDDOCUMENTS=files.DOCUMENTS_IDDOCUMENTS)
Can be written like that:
$query = FoldersQuery::create())
->joinDocuments('documents')
->joinFiles('files')
->withColumn('COUNT(documents.NAME)', 'CountName')
->withColumn('COUNT(files.IDFILES)', 'CountIdFiles')
->withColumn('SUM(files.size)', 'Sum')
->select(array('Name', 'CountName', 'CountIdFiles', 'Sum'))
;
It may requires adjustments but it's the right way.

Related

Hive lock in transactional table used in joins

We are facing this strange issue with Hive on HDInsight 4.0 - Hive 3.1.0
By default, Hive is set to handle all tables as transactional.
We have 3 tables which join together:
a
b
c
In initial phase 2 of those tables were partitioned by year/month (b,c).
Now, we have repartitioned them by year/month/day (b,c).
Which generates a number of around 200 partitions for each table(b,c).
Now if we do a select from a join b join c we get a transaction lock error.
However, if I do Select a join b - works fine, if I do Select a join c works fine.
Also, if I restrict in the join clause for one of the newly partitioned tables to scan only one partition like
Select a join b on 1=1 join c on 1=1 and c.YEAR=2019 AND c.MONTH=1
it also works fine.
It seems that the increased number of partitions which the join has to scan, or something like that is preventing Hive to take a read lock on those tables... which is very strange, since the tables do not share anything, except the same database.
Any ideas?
Full error:
java.sql.SQLException: Error while processing statement: FAILED: Error in acquiring locks: Error communicating with the metastore
at org.apache.hive.jdbc.HiveStatement.waitForOperationToComplete(HiveStatement.java:401)
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:266)
at com.hortonworks.hivestudio.hive.HiveJdbcConnectionDelegate.execute(HiveJdbcConnectionDelegate.java:56)
at com.hortonworks.hivestudio.hive.actor.StatementExecutor.runStatement(StatementExecutor.java:93)
at com.hortonworks.hivestudio.hive.actor.StatementExecutor.handleMessage(StatementExecutor.java:74)
at com.hortonworks.hivestudio.hive.actor.HiveActor.onReceive(HiveActor.java:45)
at akka.actor.UntypedAbstractActor$$anonfun$receive$1.applyOrElse(AbstractActor.scala:243)
at akka.actor.Actor.aroundReceive(Actor.scala:514)
at akka.actor.Actor.aroundReceive$(Actor.scala:512)
at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:132)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:527)
at akka.actor.ActorCell.invoke(ActorCell.scala:496)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
at akka.dispatch.Mailbox.run(Mailbox.scala:224)
at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

Join with <= in the On clause in Google bigquery

I want to execute a query with join in Google bigquery that has '<=' instead of '=' in it's on clause:
select s.count_value as count_value,s.total as total,sum(p.total) as accumulated from stats s join stats p on p.rn <=s.rn group by count_value,total,s.rn
When I run this query, I receive an error message saying:
Error: ON clause must be AND of = comparisons of one field name from each table, with all field names prefixed with table name.
Any idea how I can implement this query?
You should enable Standard SQL to do such JOINs
See Enabling Standard SQL
in CLI - just add the --use_legacy_sql=false flag to your command line statement.

Ambiguous column error creating table in Aster Studio 6.0

I am new to databases and am posting a problem from work. I am creating a table in Aster Studio 6.0, but got an error about an ambiguous column. I ran the same query in Teradata SQL Assistant and did not get an error.
I have six tables with millions of rows named EDW.SWIFTIQ_TRANS_DTL, EDW.SWIFTIQ_STORE, EDW.SWIFTIQ_PROD, EDW.STORE_XREF, EDW.TDLNX_STR_OUTLT, and EDW.SURV_CWC.
EDW represents the original database, but the columns were labeled with aliases.
I did a trim() on the VARCHAR columns for saving spool space. For the error about TDLNX_RTL_OUTLT_NBR, I performed an INNER JOIN on similar columns from two different tables. Doing a preview in SQL Assistant, there was a temporary table with only one column called TDLNX_RTL_OUTLT_NBR.
Here’s the SQL query:
CREATE TABLE public.table_name
DISTRIBUTE BY HASH (SRC_SYS_PROD_ID) AS (
SELECT * FROM load_from_teradata(
ON public.load_from_teradata_dummy
TDPID(‘database_name')
USERNAME(’user_name')
PASSWORD(’ss')
QUERY ('SELECT e.TDLNX_RTL_OUTLT_NBR, e.OUTLT_ST_ADDR_TXT, e.STORE_OUTLT_ZIP_CD, d.TRANS_ID, d.TRANS_DT,
d.TRANS_TM, d.UNIT_QTY, d.SRC_SYS_STORE_ID, d.SRC_SYS_PROD_ID, d.SRC_SYS_NM, a.SRC_SYS_STORE_ID, a.SRC_SYS_NM, a.STORE_NM,
a.CITY_NM, a.ZIP_CD, a.ST_cd, p.SRC_SYS_PROD_ID, p.SRC_SYS_NM, p.UPC_CD, p.PROD_ID, f.SRC_SYS_STORE_ID, f.SRC_SYS_NM,
f.TDLNX_RTL_OUTLT_NBR, g.SURV_CWC_WSLR_CUST_PARTY_ID, g.AGE_CD, g.HIGH_END_ACCT_FLG, g.RACE_ETHNC_CD, g.OCCPN_CD
FROM EDW.SWIFTIQ_TRANS_DTL d
INNER JOIN EDW.SWIFTIQ_STORE a
ON trim( a.SRC_SYS_STORE_ID) = trim(d.SRC_SYS_STORE_ID)
INNER JOIN EDW.SWIFTIQ_PROD p
ON trim(p.SRC_SYS_PROD_ID) = trim(d.SRC_SYS_PROD_ID)
and p.SRC_SYS_NM = d.SRC_SYS_NM
INNER JOIN EDW.STORE_XREF f
ON trim(f.SRC_SYS_STORE_ID) = trim(a.SRC_SYS_STORE_ID)
INNER JOIN EDW.TDLNX_STR_OUTLT e
ON trim(e.TDLNX_RTL_OUTLT_NBR)= trim(f.TDLNX_RTL_OUTLT_NBR)
INNER JOIN EDW.SURV_CWC g
ON g.SURV_CWC_WSLR_CUST_PARTY_ID = e.WSLR_CUST_PARTY_ID
WHERE TRANS_DT between ''2015-01-01'' and ''2015-03-31''')
num_instances('4') ) );
ERROR: column reference 'TDLNX_RTL_OUTLT_NBR' is ambiguous.
EDIT: Forgot to include a description about the table aliases. a stands for EDW.SWIFTIQ_STORE, p for EDW.SWIFTIQ_PROD, f for EDW.STORE_XREF, e for EDW.TDLNX_STR_OUTLT, g for EDW.SURV_CWC, and d for EDW.SWIFTIQ_TRANS_DTL.
You will get the same error when you try CREATE TABLE AS SELECT in Teradata. There are three column names, SRC_SYS_NM & SRC_SYS_PROD_ID & SRC_SYS_STORE_ID, which are used multiple times (with different table aliases) within the SELECT.
Add column aliases to make those names unique, e.g. trans_SRC_SYS_NM instead of d.SRC_SYS_NM.
Additionally the TRIMs in the joins are a very bad idea. You will probably not save that much spool, but force the optimizer to redistribute all spools for join-preparation.

DB2 SQL Query issue when using Outer Join

I am hoping someone can shed some light to this error I am receiving on a Query script ...
SELECT
...
FROM
LEFT OUTER JOIN DATESA ON DAILY_DATES.ID = DATESA.ID
table3,
table4
WHERE
...
I am getting an error message and stuck; can't figure out the issue on my syntax?!
Why do you put table3, table4 after the join? You specified the tables already in the join
SELECT ...
FROM daily_dates
LEFT OUTER JOIN datesa
ON daily_dates.id = datesa.id
WHERE ...
And you forgot to specify your first table after the select (view my example for the syntax)

Join on Subquery: Google BigQuery

I am trying to join two tables:
select * from
(select *, STRING(ID) as ID_string from Dataset1.Table1 where create_date >= 1388514600) as A left join each Dataset2.Table1 as B on A.ID_string = B.ID
On running the above query, I get the following error:
Field 'ID_string' not found in table 'Dataset1.Table1'
Why is the join not recognizing the newly created column "ID_string"?
Solution: Try the same query, but specifying each field by its name (instead of using *).

Resources