I'm trying to join two tables:
Table A has three columns: State, County, and Count (of Farmer's Markets in said county)
Table B has several columns: State, County, and several data columns (like food access score)
I'm trying to combine them in such a way as to put the Count for each State/County combination (since there are multiple counties with the same name) together with the State and County and data columns from Table B.
I've been banging my head on SAS, trying to get a join to cooperate. I read a few other questions on here, but I can't find where the mistake is in my code.
PROC SQL;
CREATE TABLE WORK.QUERY1
AS
SELECT FMDV4.State, FMDV4.County, FMDV4.Count, CFSDV1.GROC14,
CFSDV1.SUPERC14, CFSDV1.CONVS14, CFSDV1.SPECS14, CFSDV1.FOODINSEC_13_15,
CFSDV1.PCT_LACCESS_POP15, CFSDV1.DIRSALES_FARMS12, CFSDV1.FMRKT16,
CFSDV1.FOODHUB16, CFSDV1.CSA12, CFSDV1.POVRATE15, CFSDV1.PERPOV10
FROM FNLPRJT.CFSDV1 AS CFSDV1
INNER JOIN FNLPRJT.FMDV4 AS FMDV4
ON (( CFSDV1.State = FMDV4.State ) AND ( CFSDV1.County =
FMDV4.County ));
QUIT;
I also tried a few variants, like:
PROC SQL;
CREATE TABLE WORK.QUERY1
AS
SELECT FMDV4.State, FMDV4.County, FMDV4.Count, CFSDV1.GROC14,
CFSDV1.SUPERC14, CFSDV1.CONVS14, CFSDV1.SPECS14, CFSDV1.FOODINSEC_13_15,
CFSDV1.PCT_LACCESS_POP15, CFSDV1.DIRSALES_FARMS12, CFSDV1.FMRKT16,
CFSDV1.FOODHUB16, CFSDV1.CSA12, CFSDV1.POVRATE15, CFSDV1.PERPOV10
FROM FNLPRJT.CFSDV1 AS CFSDV1
INNER JOIN FNLPRJT.FMDV4 AS FMDV4
ON CFSDV1.State = FMDV4.State
WHERE CFSDV1.County = FMDV4.County;
QUIT;
I get a table of 0 rows with the columns as they should be (State, County, Count, ). I'm just missing the dang data! Can anyone please help me find my mistake?
Can you try
propcase(CFSDV1.State) = propcase(FMDV4.State)
and
propcase(CFSDV1.County) = propcase(FMDV4.County);
If this doesn't work try character functions like trim and compress to remove any blanks that might be present in the data.
Related
I am not sure how to query my fact tables(covid and vaccinations), I populated the dimensions with dummy data, I am supposed to leave the fact tables empty? As far as I know, they would get populated when I write the queries.
I am not sure how to query the tables I have tried different things, but I get an empty result.
Below is a link to the schema.
I want to find out the "TotalDeathsUK"(fact table COVID) for the last year caused by each "Strain"(my strain table has 3 strain in total.
You can use MERGE to poulate your fact table COVIDFact :
MERGE
INTO factcovid
using (
SELECT centerid,
dateid,
patientid,
strainid
FROM yourstagingfacttable ) AS f
ON factcovid.centerid = f.centerid AND factcovid.dateid=f.dateid... //the join columns
WHEN matched THEN
do nothing WHEN NOT matched THEN
INSERT VALUES
(
f.centerid,
f.dateid,
f.patientid,
f.strainid
)
And for VaccinationsFact :
MERGE
INTO vaccinations
using (
SELECT centerid,
dateid,
patientid,
vaccineid
FROM yourstagingfacttable ) AS f
ON factcovid.centerid = f.centerid //join condition(s)
WHEN matched THEN
do nothing WHEN NOT matched THEN
INSERT VALUES
(
f.centerid,
f.dateid,
f.patientid,
f.vaccineid
)
For the TotalDeathUK measure :
SELECT S.[Name] AS Strain, COUNT(CF.PatientID) AS [Count of Deaths] FROM CovidFact AS CF
LEFT JOIN Strain AS S ON S.StrainID=CF.StrainID
LEFT JOIN Time AS T ON CF.DateID=T.DateID
LEFT JOIN TreatmentCenter AS TR ON TR.CenterID=CF.CenterID
LEFT JOIN City AS C ON C.CityID = TR.CityID
WHERE C.Country LIKE 'UK' AND T.Year=2020
AND Result LIKE 'Death' // you should add a Result column to check if the Patient survived or died
GROUP BY S.[Name]
I recently started using SAS, only receiving a basic training that didn't cover proc sql. I'd like to read up a bit more on SAS sql when I have the time.
For now, I found a solution to what I wanted to do, but I'm having difficulties understanding what is happening.
My issue started when I wanted to find out which subjects in my dataset have a certain value for all their records. I made use of my previously written snippet of code that I thought I understood. I just tried adding a couple more variables and group by statements:
data have;
input subject:$1. myvar:1. mycount:1.;
datalines;
a 1 1
a 0 2
a 0 3
b 1 1
b 0 2
b 1 3
c 1 1
c 1 2 /*This subject has myvar = 1 for all its observations*/
;
run;
*find subjects;
proc sql;
create table want as
/* select*/
/* distinct x.subject */
/* from */
(select distinct subject, count(myvar) as myvar_c
from have where myvar = 1 group by subject) x,
(select distinct subject, max(mycount) as max_c
from have group by subject) y
where x.subject = y.subject and x.myvar_c = y.max_c;
quit;
When removing the commented 'select distinct x.subject from' in the create table statement, the above code works as should.
However, I've previously also created another piece of code, to select all subjects in my dataset that have two types of records:
data have2;
input subject:$1. mytype:1.;
datalines;
a 1
a 0
a 0
b 1
b 0
b 1
c 1
c 1 /*This subject doesn't have two types of records in all its observations*/
;
run;
*Find subjects;
proc sql;
create table want2 as select
distinct x.subject from
have2 x,
(select distinct subject, count(distinct mytype) as mytype_c from have2 group by subject) y
where y.mytype_c = 2 and x.subject = y.subject;
quit;
Which is similar, but didn't require the additional select statement. The first code has 3 select statements, the second code only requires two select statements.
Can someone inform me why this is exactly required?
Or link me some good documentation that lists the specifications of these types of joins - can anyone also inform me of the specific name of this type of join where you only use a comma?
while I'm writing, also see that could've used my code I initially wrote to find subjects that have only 1 type of record and tweak it for my current issue >.< but still would like to know what is happening in the first example.
The SQL join construct
FROM ONE, TWO, THREE, …
is known as a CROSS JOIN and is a join without criteria. The comma (,) syntax is less prevalent today and the following construct is recommended
FROM ONE
CROSS JOIN TWO
CROSS JOIN THREE
The result set is a cartesian product and the number of rows is the product of the number of rows in the cross joined tables.
When the query has criteria (WHERE clause) the join is an INNER JOIN.
The SAS documentation for Proc SQL is a good starting point and includes examples.
joined-table Component
Joins a table with itself or with other tables or views.
…
Table of Contents
Syntax
Required Arguments
Optional Argument
Details
Types of Joins
Joining Tables
Table Limit
Specifying the Rows to Be Returned
Table Aliases
Joining a Table with Itself
Inner Joins
Outer Joins
Cross Joins
Union Joins
Natural Joins
Joining More Than Two Tables
Comparison of Joins and Subqueries
General tip:
If you want to fool around (fiddle) with SQL queries in a browser, try visiting
SQL Fiddle web site.
Given multiple in-resident tables, I'd like to create a new field based on fields in different tables.
table1:
LOAD * INLINE [
id1,val1
a1,car1
a2,car1
];
table2:
LOAD * INLINE [
id2,id1,val2
b1,a1,type1
b2,a2,type2
];
table3:
LOAD * INLINE [
id3,id2,val3
c1,b1,mfr1
c2,b2,mfr2
];
For the sake of argument, assume table1 has ~1M rows, table2 ~1K rows, and table3 ~10 rows. I'd like to create a new field that is either added to table1 or perhaps in a new table linked by id1, resulting in:
id1 val1 newval
a1 car1 car1type1mfr1
a2 car2 car2type2mfr2
Efforts:
newtable:
load val1 & val2 & val3 as newval;
No errors but no newtable or newval.
newtable:
left join (table2)
load val1&val2 as newval resident table1;
Errs with Field not found - <val2>. (Obviously I want to extend this to include table3, but if I can't do it with 2 tables then 3 just won't work.
The real data includes seven tables for this new field (lots of foreign keys). The data is being loaded from QVDs (the data is shared across multiple QVWs), closely mimicking a SQL database; none of the tables are row-wise redundant, so combining db tables into a single QVD table may be inefficient. (Plus refreshing the data is incredibly easier one table at a time.) A colleague suggested I load-join each of the QVDs into one huge table, but that doesn't seem right (nor have I successfully chain-joined even a few tables).
Using QV 12.0 desktop on win10-x64 for deployment on QVS.
#TheBudac's was part of the way there, but it only merged two of the three. Most of the problems were stemming from incorrect multi-table joins. My confusion was in the "join" syntax in Qlik; the docs make sense to me now that I see what's happening, but it wasn't as obvious to me initially.
Here's what eventually worked best for me:
temptable:
load id1 as id1a, val1 as val1a
resident table1;
left join (temptable)
load id2 as id2a, id1 as id1a, val2 as val2a
resident table2;
left join (temptable)
load id2 as id2a, val3 as val3a
resident table3;
newtable:
load id1a as id1,
val1a & val2a & val3a as newval
resident temptable;
drop table temptable;
This produced these tables:
and this tree:
Quick walk-through:
Because I'm using left join, I start with the largest table; other joins would dictate different starting condition requirements. In my case, table1 was representing the largest, so I start with that:
temptable:
load id1 as id1a, val1 as val1a
resident table1;
Each join should be against the temporary table we're working on. Renaming variables is important so that Qlik doesn't create unnecessary synthetic keys.
left join (temptable)
load id2 as id2a, id1 as id1a, val2 as val2a
resident table2;
The use of resident is important in that it does not re-query (SQL) or re-load (QVD or other file).
Repeat with the third and further tables, always joining against temptable with the new table.
Now we use that temporary table to create our new table. You can choose to augment table1 with this data instead (certainly feasible), but for me since I'm generating several new calculated fields (not shown here), it made sense to keep them logically separated.
newtable:
load id1a as id1,
val1a & val2a & val3a as newval
resident temptable;
drop table temptable;
Note that I rename the relevant key back to its original value so that this table correctly links to table. Dropping the temporary table helps clean things up, but it does no harm to keep it around (and doing so helps in debugging/learning).
Your join is the wrong way round and QlikView can only work results after they have been joined,not in process, so you will have to do another resident load to get the values concatenated into Newval. The drop table commands are important or you will end up with massive unintentional syn tables
newtable:
left join (table1)
load * resident table2; drop table 2;
Resulttable:
load id1,
val1&val2 as NewVal
resident newtable; drop newtable;
I'm really struggling at the moment trying to work out how to join multiple tables without duplicating data.
At the moment I have 8 tables that I was wanted to get various information from per member of staff like the below:
SDQ score, Goal scores, CHI score, number of appointments, number of dna appointments
The tables and field I can see to join are as follows
tblSDQ - Assessed_By_Staff_ID
tblGoals - Recorded_By_Staff_ID
tblCHI - Recorded_By_Staff_ID
tblReferral - Staff_ID
tblStaff - Staff_ID
tblDiaryAppointment - needs to connect to tblDiaryAppointmentClinician using Clinician_Invitee_Staff_ID
I hope someone can help or advice. I just don't know if it's even possible to join all these tables using the same field, or if its possible to join them but then return a number of entries but then just count others?
Syntax depends on a rdbms you are using.
You could use join with specified join fields from both tables:
select bla-bla
from table1
join table2 on ( table1.fileld_name1 = table2.fileld_name2 )
https://dev.mysql.com/doc/refman/5.0/en/join.html
if you need outer join (to show nulls for optional tables data) you could use this:
join table2 on ( table1.fileld_name1 = table2.fileld_name2 or table2.field_name2 is null )
to join with couns you could use subqueries like this
join ( select field_name3, coint(*) as cnt from table3 goup by field_name3 ) AS table3_counts
...
where ( table3_counts.field_name3 = ... or table3_counts.field_name3 is null )
https://dev.mysql.com/doc/refman/5.0/en/from-clause-subqueries.html
PS: Joins are often slow. It's better to denormalize tables to eliminate joins and gain performance. Or do simple selects and join in backend code.
In my ETL process I am using Change Data Capture (CDC) to discover only rows that have been changed in the source tables since the last extraction. Then I do the transformation only for this rows. The problem is when I have for example 2 tables which I want to join into one dimension, and only one of them has changed. For example I have table Countries and Towns as following:
Countries:
ID Name
1 France
Towns:
ID Name Country_ID
1 Lyon 1
Now lets say a new row is added to Towns table:
ID Name Country_ID
1 Lyon 1
2 Paris 2
The Countries table has not been changed, so CDC for these tables shows me only the row from Towns table. The problem is when I do the join between Countries and Towns, there is no row in Countries change set, so the join will result in empty set.
Do you have an idea how to solve it? Of course there might be more difficult cases, involving 3 and more tables, and consequential joins.
This is a typical problem found when doing Realtime Change-Data-Capture, or even Incremental-only daily changes.
There's multiple ways to solve this.
One way would be to do your joins on the natural keys in the dimension or mapping table, to get the associated country (SELECT distinct country_name, [..other attributes..] from dim_table where country_id = X).
Another alternative would be to do the join as part of the change capture process - when a row is loaded to towns, a trigger goes off that loads the foreign key values into the associated staging tables (country, etc).
There is allot i could babble on for more information on but i will be specific to what is in your question. I would suggest the following to get the results...
1st Pass is where everything matches via the join...
Union All
2nd Pass Gets all towns where there isn't a country
(left outer join with a where condition that
requires the ID in the countries table to be null/missing).
You would default the Country ID value in that unmatched join to something designated as a "Unmatched Value" typically 0 or -1 is used or a series of standard -negative numbers that you could assign descriptions to later to identify why data is bad for your example -1 could be "Found Town Without Country".