can i do JOIN with two querys with yql, i have two querys:
select *
from yahoo.finance.historicaldata
where symbol in ('YHOO')
and startDate='" + startDate + "'
and endDate='" + endDate + "'&format=json&diagnostics=true&env=store://datatables.org/alltableswithkeys&callback="
and
select symbol,
Earnings_per_Share,
Dividend_Yield,
week_Low,
Week_High,
Last_Trade_Date,
open,
low,
high,
volume,
Last_Trade
from csv where url="http://download.finance.yahoo.com/d/quotes.csv?s=YHOO,GOOG&f=seyjkd1oghvl1&e=.csv"
and columns="symbol,Earnings_per_Share,Dividend_Yield,Last_Trade_Date,week_Low,Week_High,open,low,high,volume,Last_Trade"
i need to convert this two querys to one. how to do this?
Related
I have a model, Lease, with attributes ends_on (date), and notice_days (integer) I would like to write a query that returns all records that have an ends_on date that is less than today's date plus notice_days for that record.
In straight SQL (for my Postgresql db) the query is:
SELECT * FROM leases WHERE ends_on < CURRENT_DATE + notice_days
I can easily write a query to compare against today's date + some fixed buffer
Lease.where(ends_on: ...Date.current + 30).all
But I don't want the fixed buffer, I want a buffer (notice_days) that is unique to each record.
I do realize that I could execute the sql shown above using exec_sql, but I would like to learn how to do this with a straightforward ActiveRecord query.
Edit
Stefanyuk's answer below does in fact answer my question, but I am looking for a way to write the ActiveRecord query with key value pairs as arguments to the where method in the hopes of making the query less database dependent. It seems that this should be possible but I have tried many ways of passing the attribute name without success.
This does not work, but I am looking for something similar that does:
Lease.where(ends_on: ...(Date.current + :notice_days))
Is there a way to replace the symbol :notice_days in that example with a reference to the attribute notice_days?
You can do this by using interval
SELECT * FROM leases WHERE ends_on < CURRENT_DATE + interval '1 day' * notice_days
Or with Rails
Lease.where("ends_on < CURRENT_DATE + interval '1 day' * notice_days")
Or with Arel
Lease.where(Lease.arel_table[:ends_on].lt(Arel.sql("CURRENT_DATE + notice_days")))
I'm trying to implement an event time temporal join in Flink. Here's the first join table:
tEnv.executeSql("CREATE TABLE AggregatedTrafficData_Kafka (" +
"`timestamp` TIMESTAMP_LTZ(3)," +
"`area` STRING," +
"`networkEdge` STRING," +
"`vehiclesNumber` BIGINT," +
"`averageSpeed` INTEGER," +
"WATERMARK FOR `timestamp` AS `timestamp`" +
") WITH (" +
"'connector' = 'kafka'," +
"'topic' = 'seneca.trafficdata.aggregated'," +
"'properties.bootstrap.servers' = 'localhost:9092'," +
"'properties.group.id' = 'traffic-data-aggregation-job'," +
"'format' = 'json'," +
"'json.timestamp-format.standard' = 'ISO-8601'" +
")");
The table is used as a sink for the following query:
Table aggregatedTrafficData = trafficData
.window(Slide.over(lit(30).seconds())
.every(lit(15).seconds())
.on($("timestamp"))
.as("w"))
.groupBy($("w"), $("networkEdge"), $("area"))
.select(
$("w").end().as("timestamp"),
$("area"),
$("networkEdge"),
$("plate").count().as("vehiclesNumber"),
$("speed").avg().as("averageSpeed")
);
Here's the other join table. I use Debezium to stream a Postgres table into Kafka:
tEnv.executeSql("CREATE TABLE TransportNetworkEdge_Kafka (" +
"`timestamp` TIMESTAMP_LTZ(3) METADATA FROM 'value.source.timestamp' VIRTUAL," +
"`urn` STRING," +
"`flow_rate` INTEGER," +
"PRIMARY KEY(`urn`) NOT ENFORCED," +
"WATERMARK FOR `timestamp` AS `timestamp`" +
") WITH (" +
"'connector' = 'kafka'," +
"'topic' = 'seneca.network.transport_network_edge'," +
"'scan.startup.mode' = 'latest-offset'," +
"'properties.bootstrap.servers' = 'localhost:9092'," +
"'properties.group.id' = 'traffic-data-aggregation-job'," +
"'format' = 'debezium-json'," +
"'debezium-json.schema-include' = 'true'" +
")");
Finally here's the temporal join:
Table transportNetworkCongestion = tEnv.sqlQuery("SELECT AggregatedTrafficData_Kafka.`timestamp`, `networkEdge`, " +
"congestion(`vehiclesNumber`, `flow_rate`) AS `congestion` FROM AggregatedTrafficData_Kafka " +
"JOIN TransportNetworkEdge_Kafka FOR SYSTEM_TIME AS OF AggregatedTrafficData_Kafka.`timestamp` " +
"ON AggregatedTrafficData_Kafka.`networkEdge` = TransportNetworkEdge_Kafka.`urn`");
The problem I'm having is that the join works only for the first few second (after an update in the Postgres table), but I need to continuosly join the first table with debezium one. Am I doing something wrong?
Thanks
euks
Temporal joins using the AS OF syntax you're using require:
an append-only table with a valid event-time attribute
an updating table with a primary key and a valid event-time attribute
an equality predicate on the primary key
When Flink SQL's temporal operators are applied to event time streams, watermarks play a critical role in determining when results are produced, and when the state is cleared.
When performing a temporal join:
rows from the append-only table are buffered in Flink state until the current watermark of the join operator reaches their timestamps
for the versioned table, for each key the latest version whose timestamp precedes the join operator's current watermark is kept in state, plus any versions from after the current watermark
whenever the join operator's watermark advances, new results are produced, and state that's no longer relevant is cleared
The join operator tracks the watermarks it receives from its input channels, and its current watermark is always the minimum of these two watermarks. This is why your join stalls, and only makes progress when the flow_rate is updated.
One way to fix this would be to set the watermark for the TransportNetworkEdge_Kafka table like this:
"WATERMARK FOR `timestamp` AS " + Watermark.MAX_WATERMARK
This will set the watermark for this table/stream to the largest possible value, which will have the effect of making the watermarks from this stream irrelevant -- this stream's watermarks will never be the smallest.
This will, however, have the drawback of making the join results non-deterministic.
I have 2 data sources that I am querying, then joining in Crystal Reports on a key string with a Left Outer Join. The intent of the report is to identify purchases made that were not processed. The issue is that CR refuses to show the matching right query records.
Data Source 1: Excel worksheet on my local machine containing raw
credit card purchases. "Left table"
Data Source 2: 2 subqueries from a hosted Oracle database with a
Union join containing processed credit card transactions. "Right
table"
Key String: The last 4 digits of a credit card number concatenated
with the date-time of the transaction, e.g. "223402-06-2019 04:15:00"
The queries return proper values when executed separately. I have verified that many records returned for the Left table actually do have matching Right table records that are not displayed. I did this using a separate report showing only the Right table query results and manually searching for different key strings.
I'm completely buffaloed and any assistance would be appreciated.
The SQL from Crystal Reports:
I:\Dept\DCS\MPOOL\Fleet Management Data\M5\M5 Automation Data Tables\ComData Transaction Data.xls
`SELECT DISTINCT CD.`First Name` AS UNIT_NO,
CD.`HIERARCHY LEVEL3` AS USE_DEPT,
DATEVALUE(MONTH(CD.`Transaction Date`) & "/" & DAY(CD.`Transaction Date`) & "/" & YEAR(CD.`Transaction Date`)) + TIMEVALUE(HOUR(CD.`Transaction Time`) & ":" & MINUTE(CD.`Transaction Time`) & ":" & SECOND(CD.`Transaction Time`)) AS TRANS_DT,
CD.`Odometer` AS ODOMETER,
CD.`Card Number` AS CARD_NO,
RIGHT(CD.`Card Number`, 4) & FORMAT(DATEVALUE(MONTH(CD.`Transaction Date`) & "/" & DAY(CD.`Transaction Date`) & "/" & YEAR(CD.`Transaction Date`)) + TIMEVALUE(HOUR(CD.`Transaction Time`) & ":" & MINUTE(CD.`Transaction Time`) & ":" & SECOND(CD.`Transaction Time`)), "mm-dd-yyyy hh:mm:ss") AS KEYSTRING
FROM `Sheet1$` CD
WHERE ISDATE(CD.`Transaction Date`) AND CD.`Transaction Date` >= FORMAT('02/01/2019', 'mm-dd-yyyy') AND CD.`Transaction Date` <= FORMAT('02/15/2019', 'mm-dd-yyyy')
EXTERNAL JOIN Command.KEYSTRING={?m5oksr: Command_1.KEYSTRING}
m5oksr
SELECT DISTINCT TCC.UNIT_NO,
VUDC.USING_DEPT_NO AS USE_DEPT,
TCC.ISSUE_DT + 2/24 AS TRANS_DT,
TCC.NEW_METER AS ODOMETER,
'COMP' AS STATUS,
TCC.CARD_NO AS CARD_NO,
SUBSTR(TCC.CARD_NO, 16, 4) || TO_CHAR(TCC.ISSUE_DT + 2/24, 'MM-DD-YYYY HH24:MI:SS') AS KEYSTRING,
FROM MFIVE.VIEW_TRIPCARD_COMPLETED_TRANS TCC
LEFT OUTER JOIN VIEW_UNIT_DEPT_COMP VUDC ON TCC.COMPANY = VUDC.COMPANY and TCC.UNIT_NO = VUDC.UNIT_NO
WHERE TCC.ISSUE_DT + 2/24 >= TO_DATE('02/01/2019 00:00:00', 'MM/DD/YYYY HH24:MI:SS') AND TCC.ISSUE_DT + 2/24 <= TO_DATE('02/15/2019 11:59:59', 'MM/DD/YYYY HH24:MI:SS')
UNION
SELECT DISTINCT IR.FIELD2 as UNIT_NO,
VUDC.USING_DEPT_NO AS USE_DEPT,
TO_DATE(IR.FIELD1, 'MM/DD/YYYY HH24:MI:SS') + 2/24 AS TRANS_DT,
IR.METER as ODOMETER,
'FAIL' AS STATUS,
NVL2(IR.FIELD27, CONCAT('XXXX-XXXX-XXXX-', SUBSTR(IR.FIELD27,-4)),'') as CARD_NO,
SUBSTR(NVL2(IR.FIELD27, CONCAT('XXXX-XXXX-XXXX-', SUBSTR(IR.FIELD27,-4)),''), 16, 4) || TO_CHAR(TO_DATE(IR.FIELD1, 'MM/DD/YYYY HH24:MI:SS') + 2/24, 'MM-DD-YYYY HH24:MI:SS') AS KEYSTRING,
FROM INTERFACE_REJECT IR
INNER JOIN INTERFACE_STAT ST ON IR.COMPANY = ST.COMPANY and IR.STAT_ID = ST.STAT_ID
LEFT OUTER JOIN EMP_MAIN E ON IR.COMPANY = E.COMPANY AND IR.FIELD29 = E.TRIPCARD_PIN
LEFT OUTER JOIN VIEW_UNIT_DEPT_COMP VUDC ON IR.COMPANY = VUDC.COMPANY and IR.FIELD2 = VUDC.UNIT_NO
WHERE LENGTH(IR.FIELD1) = 19 AND ST.INTERFACE_NAME = 'M5-TRIP-CARD-INTF' AND TO_DATE(IR.FIELD1, 'MM/DD/YYYY HH24:MI:SS') + 2/24 >=TO_DATE('02/01/2019 00:00:00', 'MM/DD/YYYY HH24:MI:SS') AND TO_DATE(IR.FIELD1, 'MM/DD/YYYY HH24:MI:SS') + 2/24 <= TO_DATE('02/15/2019 11:59:59', 'MM/DD/YYYY HH24:MI:SS')
EXTERNAL JOIN Command_1.KEYSTRING={?I:\Dept\DCS\MPOOL\Fleet Management Data\M5\M5 Automation Data Tables\ComData Transaction Data.xls: Command.KEYSTRING}
Are you sure the join works? If the join doesn't work then you will get nulls and my guess is that this is what is happening. Try to use INNER JOIN instead of Lef join and check if there are any rows returned. If records are returned you may need to cast the values to the same type and trim them. It is possible that the value returned by excel has empty spaces or different value type, which Crystal converts incorrectly
Hi there,
I am trying to fetch last six months data in my query and need to represent 'Month-year' label on x-axis. So query works fine when there is data for a month but if it is unsuccesful in join and no data is returned for that month - there is no label.Hence I am unable to draw it on chart (report Builder 3.0) E.g.
ApptMonthYearname NotCompleteAppointments AppointmentYear AppointmentMonthInt
January-2012 118 2012 1
December-2011 88 2011 12
Query includes a join on three tables and then where clause checks that an appointment is falling between the selected range of month and year or not :
declare #SelectedMonth int
declare #SelectedYear int
declare #careprovider varchar(20)
DECLARE #intFlag INT
let's say
SET #SelectedMonth = 1
SET #SelectedYear =2012
declare #selectedDate datetime
declare #previoussixmonthsdate datetime
IF (#SelectedMonth = Datepart(mm,GETDate()) and #SelectedYear =Datepart(yyyy,GETDate()))
BEGIN
SET #selectedDate = CONVERT(datetime, CONVERT(varchar(2), datepart(DD,GETDATE())+ '/' + Convert(varchar(2),#SelectedMonth) + '/' +Convert(varchar(4),#SelectedYear), 103))
SET #previoussixmonthsdate= DATEADD(month, -6, #selectedDate)
END
ELSE
BEGIN
SET #selectedDate = CONVERT(datetime, '31'+ '/' + Convert(varchar(10),#SelectedMonth) + '/' +Convert(varchar(10),#SelectedYear), 103)
SET #previoussixmonthsdate= DATEADD(month, -6, #selectedDate)
END
select #selectedDate, #previoussixmonthsdate
SELECT dbo.Filteredals_clinicappointment.als_clinicappointmentid [AppointmentID],
dbo.Filteredals_clinicappointment.als_statusname [AppointmentStatus],
dbo.Filteredals_clinicappointment.als_appointmentdatetime [AppointmentBookingTime],
Datepart(mm,dbo.Filteredals_clinicappointment.als_appointmentdatetime) [AppointmentMonth],
Datepart(yyyy,dbo.Filteredals_clinicappointment.als_appointmentdatetime) [AppointmentYear],
DATENAME(month,dbo.Filteredals_clinicappointment.als_appointmentdatetime) [AppointmentMonthName],
DATENAME (year,dbo.Filteredals_clinicappointment.als_appointmentdatetime) [AppointmentyearName]
FROM dbo.Filteredals_clinicappointment LEFT OUTER JOIN
dbo.Filteredrbs_clinicinstance ON dbo.Filteredals_clinicappointment.als_clinicinstance = dbo.Filteredrbs_clinicinstance.rbs_clinicinstanceid LEFT OUTER JOIN
dbo.Filteredrbs_clinic ON dbo.Filteredrbs_clinicinstance.rbs_clinic = dbo.Filteredrbs_clinic.rbs_clinicid LEFT OUTER JOIN
dbo.Filteredrbs_careproviders ON dbo.Filteredrbs_clinic.rbs_careprovider = dbo.Filteredrbs_careproviders.rbs_careprovidersid
WHERE dbo.Filteredrbs_careproviders.rbs_careprovidersid= #careprovider
AND dbo.Filteredals_clinicappointment.als_appointmentdatetime <= #selectedDate AND
dbo.Filteredals_clinicappointment.als_appointmentdatetime >=#previoussixmonthsdate
,dbo.Filteredals_clinicappointment.als_appointmentdatetime)= #SelectedYear
GROUP BY YEAR(AppointmentList.AppointmentBookingTime), MONTH(AppointmentList.AppointmentBookingTime)) as [DNAAppts]
Any help would be greatly appreciated.
One way to guarantee that you always get data for each month in the report range is to populate a temporary/derived table with the set of dates then left join that to the data.
Alternatively you can fake the data eg by storing the results of your query (or a sensible half way stage) into a temporary table then inspect that table to ensure there is data per expected month and if not add it.
Approach 3: union your query with a statement that returns a value for each month expected where there is not real data to match
There are probably even more ways to do this but hopefully that will offer you some inspiration
Some one please explain me how does this query works
SELECT 'WRITEBOARDCOMMENT' AS Type,
wbc.CommentText AS Content,
wb.WBId AS Id,
null AS ToDoListName,
null AS DueDate,
u.FirstName + ' ' + u.LastName AS ActivityBy,
wbc.[Date] as Date,
u.FirstName + ' ' + u.LastName as PartyName,
comp.CompanyId AS CompanyId,
comp.CompanyName AS CompanyName,
p.ProjectName,
p.ProjectId,
wbc.WBCmtId AS SubId,
p.ProjectStartPageId AS ProjectStartPageId
FROM
WriteboardComment AS wbc,
WriteBoardVersions AS wbv,
WriteBoard AS wb,
Project AS p,[user] AS u,
Company AS comp
where
wbc.wbversionid=wbv.wbversionsid and
wbv.WBId=wb.WBId and
wb.ProjectId=p.ProjectId and
p.ProjectId=#projectid and
wbc.CommentedBy=u.UserId and
p.PrimaryCompanyId=comp.CompanyId
What is the advantage of joining the tables like this.I found out this one in one project db code.
There is no advantage, this is an old style join and you can easily shoot yourself in the foot when you mess up/leave out the WHERE clause and then you create a cartesian product/cross join by mistake
This used to be the old style of doing the joins. There is no advantage over the classical join version.