1C:Enterprise Platform: 8.3.17.1851
For a document, the record-level security (RLS) is configured. The document has a table section with additional attributes specified as the document's characteristics.
When rendering the document in a dynamic list with an additional attribute, the database server receives the following query:
SELECT TOP 45 T1.Ref
FROM dbo.Document.Document1 T1
LEFT OUTER JOIN
(
SELECT T11.Ref AS Q_001_F_000RRef,
T11.Value AS Q_001_F_001RRef,
T11.Value AS Q_001_F_002_TYPE,
T11.Value AS Q_001_F_002_L,
T11.Value AS Q_001_F_002_N,
T11.Value AS Q_001_F_002_T,
T11.Value AS Q_001_F_002_S,
T11.Value AS Q_001_F_002_RTRef,
T11.Value AS Q_001_F_002_RRRef
FROM dbo.Document.Document1.AdditionalAttributes T11
INNER JOIN dbo.Document.Document1 T12 ON T11.Ref = T12.Ref
// query constraint description
)
// query constraint description
It turns out that the platform applies a query constraint to the main table several times.
For a non-trivial query constraint, this may result in significant query processing performance degradation. The more additional attributes you include in the query, the greater performance degradation you experience.
Why the platform cannot avoid adding excessive access constraints when applying the inner join or left outer join of the main table to the table section?
In a general case, the construction
FROM dbo.Document.Document1 T1 LEFT OUTER JOIN dbo.Document.Document1 T2 ON
assumes that the system may select different records from T1 and T2.
So, the access constraints should be applied both to T1 and to T2 independently.
In certain cases, depending on the constraint types, the platform can optimize the query.
Probably, in the scenario you described, the query cannot be optimized.
Related
I don't get an except join to work in Cognos-11. Where or what am I missing?
Some understanding for a beginner in this branch would be nice ;-)
What I've tried so far is making two queries. The first one holds data items like "customer", "BeginningDate" and "Purpose". The second query holds data items like "customer", "Adress" and "Community".
What I'd like to accomplish is to get in query3: the "customers" from query1 that are not available in query2. To me it sounds like an except-join.
I went to the query work area, created a query3 and dragged an "except-join" icon on it. Then I dragged query1 into the upper space and query2 into the lower. What I'm used to getting with other joins, is a possibility to set a new link, cardinality and so on. Now double clicking the join isn't opening any pop-up. The properties of the except-join show "Set operation = Except", "Duplicates = remove", "Projection list = Manual".
How do I get query3 filled with the data item "customer" that only holds a list of customers which are solely appearing in query1?
In SQL terms, you want
select T2.C1
from T1
left outer join T2 on T1.C1 = T2.C1
where T2.C1 is null
So, in the query pane of a Cognos report...
Use a regular join.
Join using customer from both queries.
Change the cardinality to 1..1 on the query1 side and 0..1 on the query2 side.
In the filters for query3, add a filter for query2.customer is null.
EXCEPT is not a join. It is used to compare two data sets.
https://learn.microsoft.com/en-us/sql/t-sql/language-elements/set-operators-except-and-intersect-transact-sql?view=sql-server-2017
What you need is an INNER JOIN. That would be the join tool in the Toolbox in Cognos.
According to Hive's documentation it supports NOT IN subqueries in a WHERE clause, provided that the subquery is an uncorrelated subquery (does not reference columns from the main query).
However, when I attempt to run the trivial query below, I get an error FAILED: SemanticException Cartesian products are disabled for safety reasons.
-- sample data
CREATE TEMPORARY TABLE foods (name STRING);
CREATE TEMPORARY TABLE vegetables (name STRING);
INSERT INTO foods VALUES ('steak'), ('eggs'), ('celery'), ('onion'), ('carrot');
INSERT INTO vegetables VALUES ('celery'), ('onion'), ('carrot');
-- the problematic query
SELECT *
FROM foods
WHERE foods.name NOT IN (SELECT vegetables.name FROM vegetables)
Note that if I use an IN clause instead of a NOT IN clause, it actually works fine, which is perplexing because the query evaluation structure should be the same in either case.
Is there a workaround for this, or another way to filter values from a query based on their presence in another table?
This is Hive 2.3.4 btw, running on an Amazon EMR cluster.
Not sure why you would get that error. One work around is to use not exists.
SELECT f.*
FROM foods f
WHERE NOT EXISTS (SELECT 1
FROM vegetables v
WHERE v.name = f.name)
or a left join
SELECT f.*
FROM foods f
LEFT JOIN vegetables v ON v.name = f.name
WHERE v.name is NULL
You got cartesian join because this is what Hive does in this case. vegetables table is very small (just one row) and it is being broadcasted to perform the cross (most probably map-join, check the plan) join. Hive does cross (map) join first and then applies filter. Explicit left join syntax with filter as #VamsiPrabhala said will force to perform left join, but in this case it works the same, because the table is very small and CROSS JOIN does not multiply rows.
Execute EXPLAIN on your query and you will see what is exactly happening.
In Crystal Reports I have 2 tables linked via a LEFT OUTER JOIN - one for budget period and one for the actual expense balance per period.
Although the data in the expense table is correct, the data in the budget table returns no value if the value in the expense table is 0.
The main tables used are the BUDGET table(contains budget balances per GL acc per period) and BALANA table (contains the actual expense balance per GL acc per period)
This is the sql command:
SELECT "FISCALYEAR"."FIYNUM_0", "BALANA"."ACC_0", "BALANA"."DEBLED_0", "BALANA"."CDTLED_0", "BALANA"."DEBLED_1", "BALANA"."DEBLED_2", "BALANA"."DEBLED_3", "BALANA"."DEBLED_4", "BALANA"."DEBLED_5", "BALANA"."DEBLED_6", "BALANA"."DEBLED_7", "BALANA"."DEBLED_8", "BALANA"."DEBLED_9", "BALANA"."DEBLED_10", "BALANA"."DEBLED_11", "BALANA"."DEBLED_12", "BALANA"."CDTLED_1", "BALANA"."CDTLED_2", "BALANA"."CDTLED_3", "BALANA"."CDTLED_4", "BALANA"."CDTLED_5", "BALANA"."CDTLED_6", "BALANA"."CDTLED_7", "BALANA"."CDTLED_8", "BALANA"."CDTLED_9", "BALANA"."CDTLED_10", "BALANA"."CDTLED_11", "BALANA"."CDTLED_12", "BALANA"."LEDTYP_0", "BALANA"."CPY_0", "BALANA"."FCY_0", "BALANA"."BPR_0", "BALANA"."CURLED_0", "BALANA"."CCE1_0", "BUD"."AMT_1", "BUD"."AMT_2", "BUD"."AMT_3", "BUD"."AMT_4", "BUD"."AMT_5", "BUD"."AMT_6", "BUD"."AMT_7", "BUD"."AMT_8", "BUD"."AMT_9", "BUD"."AMT_10", "BUD"."AMT_11", "BUD"."ACC_0", "BUD"."AMT_0"
FROM ("sagex3v7live"."LIVE"."FISCALYEAR" "FISCALYEAR" LEFT OUTER JOIN "sagex3v7live"."LIVE"."BALANA" "BALANA" ON (("FISCALYEAR"."CPY_0"="BALANA"."CPY_0") AND ("FISCALYEAR"."FIYNUM_0"="BALANA"."FIY_0")) AND ("FISCALYEAR"."LEDTYP_0"="BALANA"."LEDTYP_0")) LEFT OUTER JOIN "sagex3v7live"."LIVE"."BUD" "BUD" ON ((((("FISCALYEAR"."FIYNUM_0"="BUD"."FIY_0") AND ("FISCALYEAR"."LEDTYP_0"="BUD"."LEDTYP_0")) AND ("FISCALYEAR"."CPY_0"="BUD"."CPY_0")) AND ("BALANA"."ACC_0"="BUD"."ACC_0")) AND ("BALANA"."CUR_0"="BUD"."CUR_0")) AND ("BALANA"."CCE1_0"="BUD"."CCE1_0")
WHERE "FISCALYEAR"."FIYNUM_0"=4 AND "BALANA"."LEDTYP_0"=1 AND "BALANA"."BPR_0"=N'' AND "BALANA"."FCY_0"<>N'' AND "BALANA"."CPY_0"=N'MAJ' AND "BALANA"."CURLED_0"=N'ZAR' AND ("BALANA"."CCE1_0">=N'BFN' AND "BALANA"."CCE1_0"<=N'PLZ')
ORDER BY "BALANA"."ACC_0", "BALANA"."CCE1_0"
I have the need to join a huge table (10 million plus rows) to a lookup table (15k plus rows) with an OR condition. Something like:
SELECT t1.a, t1.b, nvl(t1.c, t2.c), nvl(t1.d, t2.d)
FROM table1 t1
JOIN table2 t2 ON t1.c = t2.c OR t1.d = t2.d;
This is because table1 can have c or d as NULL, and I'd like to join on whichever is available, leaving out the rest. The query plan says there is a Nested Loop, which I realize is because of the OR condition. Is there a clean, efficient way of solving this problem? I'm using Redshift.
EDIT: I am trying to run this with a UNION, but it doesn't seem to be any faster than before.
If you have a preferred column you can NVL() (aka COALESCE()) them and join on that.
SELECT t1.a, t1.b, nvl(t1.c, t2.c), nvl(t1.d, t2.d)
FROM table1 t1
JOIN table2 t2
ON t1.c = NVL(t2.c,t2.d);
I'd also suggest that you should set the lookup table to DISTSTYLE ALL to ensure that the larger table is not redistributed.
[ Also, 10 million rows isn't big for Redshift. Not trying to be snotty just saying that we get excellent performance on Redshift even when querying (and joining) tables with hundreds of billions of rows. ]
How about doing two (left) joins? With the small lookup table performance shouldn't be too bad even.
SELECT t1.a, t1.b, nvl(t1.c, t2.c), nvl(t1.d, t3.d)
FROM table1 t1
LEFT JOIN table2 t2 ON t1.d = t2.d and t1.c is null
LEFT JOIN table2 t3 ON t1.c = t3.c and t1.d is null
Your original query only returns rows that match at least one of c or d in the lookup table. If that's not guaranteed you may need to add filters...for example rows in t1 where both c and d are null or have values not present in table2.
Don't really need the null checks in the joins, but might be slightly faster.
I have a nightly job that runs and computes some data in hive. It is partitioned by day.
Fields:
id bigint
rank bigint
Yesterday
output/dt=2013-10-31
Today
output/dt=2013-11-01
I am trying to figure out if there is a easy way to get incremental changes between today and yesterday
I was thinking about doing a left outer join but not sure what that looks like since its the same table
This is what it might looks like when there are different tables
SELECT * FROM a LEFT OUTER JOIN b
ON (a.id=b.id AND a.dt='2013-11-01' and b.dt='2-13-10-31' ) WHERE a.rank!=B.rank
But on the same table it is
SELECT * FROM a LEFT OUTER JOIN a
ON (a.id=a.id AND a.dt='2013-11-01' and a.dt='2-13-10-31' ) WHERE a.rank!=a.rank
Suggestions?
This would work
SELECT a.*
FROM A a LEFT OUTER JOIN A b ON a.id = b.id
WHERE a.dt='2013-11-01' AND b.dt='2013-10-31' AND <your-rank-conditions>;
Efficiently, this would span 1 MapReduce job only.
So I figured it out... Using Subqueries and Joins
select * from (select * from table where dt='2013-11-01') a
FULL OUTER JOIN
(select * from table where dt='2013-10-31') b
on (a.id=b.id)
where a.rank!=b.rank or a.rank is null or b.rank is null
The above will give you the diff..
You can take the diff and figure out what you need to ADD/UPDATE/REMOVE
UPDATE If a.rank!=null and b.rank!=null i.e rank changed
DELETE IF a.rank=null and b.rank!=null i.e the user is no longer ranked
ADD if a.rank!=null and b.rank=null i.e this is a new user