How to cross join KsqlDB tables or streams? - join

I want to cross join 2 KsqlDB Tables. For example:
Country Table
data_id
country
101
Malaysia
102
Singapore
Product Table
data_id
product
301
pen
302
book
The result should be:
Country_Product Table
country_data_id
country
product_data_id
product
101
Malaysia
301
pen
101
Malaysia
302
book
102
Singapore
301
pen
102
Singapore
302
book
I tried to google the solution, but I have no luck. All information that I found are inner join or outer join only.

Add another dummy column in both of your tables namely join_id. And populate the column with the value 1 (Both tables, each row). Now do the inner join, you will get the expected result.

Related

Bigquery - LEFT Join two queries with second using most recent dates

I have a query that does two queries. The first query looks at future events and what user is in that event from one table. The second query looks at the historical events and creates stats for the user in that event based off all the previous events that user was in from a second table.
I am having troubles joining the two queries. The goal of the join would be to join the second query to the first query based off the most recent stats for each user. The stat can't be newer than the date that the future event occured on. If that user hasn't been in any previous event it would just return null for the query 2 in the join.
Below is also an example table for the future query, a table for the historic query, and an ideal output of the join.
Future:
user
date
event code
event info
User 1
1/26/2023
5596
info_5596
User 2
1/26/2023
5586
info_5586
User 3
1/26/2023
5582
info_5582
User 1
1/20/2023
5492
info_5492
User 1
1/2/2023
5341
info_5341
User 2
1/2/2023
5333
info_5333
Historical:
user
date
stat 1
stat2
event code
event info
User 1
1/25/2023
10
52
4352
info_4352
User 2
1/25/2023
11
22
4332
info_4332
User 2
1/12/2023
2
45
4298
info_4298
User 3
1/12/2023
8
88
4111
info_4111
User 1
1/12/2023
7
67
4050
info_4050
User 3
1/2/2023
3
91
4000
info_4000
User 1
1/1/2023
6
15
3558
info_3558
Output of the JOIN:
user
date future
stat 1
stat2
event code future
event info future
User 1
1/26/2023
10
52
5596
info_5596
User 2
1/26/2023
11
22
5586
info_5586
User 3
1/26/2023
8
88
5582
info_5582
User 1
1/20/2023
7
67
5492
info_5492
User 1
1/2/2023
3
91
5341
info_5341
User 2
1/2/2023
null
null
5333
info_5333
I tried using a subquery in the join, but bigquery was saying that it is unsupported. Below is my code attempt. I have also tried using MAX() but it was not liking that to be used in the join as well
Another option I am thinking of is to join the two datasets before ever calculating the query 2 stats. Then filtering. I have a large query already written for both though, so I would prefer not to start over.
Select Distict
A.*
B.Stat1, B.Stat2
from future as A
Left Join historic as B
ON (
A.Date = (Select MAX(B.date) FROM historic as recent_historic WHERE recent_historic.user = A.user)
AND
A.user = B.user
)
ORDER BY A.date

How to show 3 nested tables in Google Data Studio?

I am looking for a way to combine 3 Tables:
Contract | Invoice | Payments
with relations 1:n between them, for example:
I have 3 sheets: Contract, Invoice, Payments, with IDs; for simplicity, shown below are only the ID columns:
Contract Invoice Payments
-------- ------- ---------
Contract1 Invoice10 Payment101
Contract1 Invoice10 Payment102
Contract1 Invoice11 Payment103
Contract2 Invoice12 Payment104
Contract2 Invoice13 Null
Contract3 Null Null
I want to create 3 Linked Tables in the Same Page:
When I select one Contract, the other tables only show data related to this Contract, and after I select one Invoice, then the table of Payments only shows data related to this Invoice.
The only way that I found about was blending 3 tables, however, I would need the same key fields (Join Keys) in all tables, but that is not in my case.
=IFERROR(QUERY({A1:A, B1:B, C1:C},
"where Col1='"&F1&"'
and Col2='"&F2&"'", 0),
"no match")
demo spreadsheet

How to join with three tables with multiple conditions?

I want to join three tables with multiple conditions. Below is table structure and expected result
users_id users_first_name
1 rocky
2 James
3 john
meeting_details_id meeting_title users_id meeting_lead close_meeting (NO)
1 newmeet 3 1 No
2 testmeet 2 2 No
Attended_meetings
project_meeting_attendeeid meeting_details_id users_id access_type (attendee)
1 1 2 attendee
Expected output:
Query should check if meeting is attended OR users is meeting lead
meeting_title creator meeting_lead close_meeting (NO)
newmeet john rocky No
testmeet james james No
Try this:
SELECT M.meeting_title,U1.users_first_name as creator,U2.users_first_name as meeting_lead,M.close_meeting
FROM MeetingTable M LEFT OUTER JOIN
Users U1 ON M.creator=U1.users_id LEFT OUTER JOIN
Users U2 ON M.meeting_lead=U2.users_id
Result:
MEETING_TITLE CREATOR MEETING_LEAD CLOSE_MEETING
newmeet john rocky No
testmeet James James No
See result in SQL Fiddle.

Change Data Capture with table joins in ETL

In my ETL process I am using Change Data Capture (CDC) to discover only rows that have been changed in the source tables since the last extraction. Then I do the transformation only for this rows. The problem is when I have for example 2 tables which I want to join into one dimension, and only one of them has changed. For example I have table Countries and Towns as following:
Countries:
ID Name
1 France
Towns:
ID Name Country_ID
1 Lyon 1
Now lets say a new row is added to Towns table:
ID Name Country_ID
1 Lyon 1
2 Paris 2
The Countries table has not been changed, so CDC for these tables shows me only the row from Towns table. The problem is when I do the join between Countries and Towns, there is no row in Countries change set, so the join will result in empty set.
Do you have an idea how to solve it? Of course there might be more difficult cases, involving 3 and more tables, and consequential joins.
This is a typical problem found when doing Realtime Change-Data-Capture, or even Incremental-only daily changes.
There's multiple ways to solve this.
One way would be to do your joins on the natural keys in the dimension or mapping table, to get the associated country (SELECT distinct country_name, [..other attributes..] from dim_table where country_id = X).
Another alternative would be to do the join as part of the change capture process - when a row is loaded to towns, a trigger goes off that loads the foreign key values into the associated staging tables (country, etc).
There is allot i could babble on for more information on but i will be specific to what is in your question. I would suggest the following to get the results...
1st Pass is where everything matches via the join...
Union All
2nd Pass Gets all towns where there isn't a country
(left outer join with a where condition that
requires the ID in the countries table to be null/missing).
You would default the Country ID value in that unmatched join to something designated as a "Unmatched Value" typically 0 or -1 is used or a series of standard -negative numbers that you could assign descriptions to later to identify why data is bad for your example -1 could be "Found Town Without Country".

How to query table to get one row requiring two joins to two separate ID?

Oracle 11g
PERSON table contains both seller and buyer ID. How can I get buyer and seller into single result set? I can get results to list either the buyer or the seller but not both.
Person
=================
PersonID First Last
1 Joe Camel
2 Ronald McFly
3 Barbara Wawa
SalesDetail
=========================
TransID Amount SellerID CustomerID
98 500 1 2
99 700 3 1
Desired Result
===========================================
SellerID SellerLast BuyerID BuyerLast Amount
1 Camel 2 McFly 500
3 Wawa 1 Camel 700
Just join to the Person table twice
SELECT sd.sellerID,
seller.last sellerLast,
sd.buyerID,
buyer.last buyerLast,
sd.amount
FROM salesDetail sd
JOIN person seller ON (sd.sellerID = seller.personID)
JOIN person buyer ON (sd.buyerID = buyer.personID)
You may want outer joins if it is possible that either the buyer or the seller is unknown.
try this
select seller.sellerid,
seller.last,
buyer.buyerid,
buyer.last,
amount
from
person buyer
inner join salesdetail on buyer.personid = salesdetail.cutomerid
inner join person seller on salesdetail.sellerid = seller.personid
unable to test myself at the moment

Resources