I'm using the following query to get the operationId values from the requests that failed with 400 using AppInsights:
requests
| project timestamp, id, operation_Name, success, resultCode, duration, operation_Id, cloud_RoleName, invocationId=customDimensions['InvocationId']
| where cloud_RoleName =~ 'xxxx' and operation_Name == 'createCase' and resultCode == 400
| order by timestamp desc
I use these operationId values on the following queries to get the logs of what happened:
traces
| union exceptions
| where operation_Id == '35edbc7c13f7ac4c85fa0b8071a12b72'
| order by timestamp asc
With this I'm getting the information I want but I need to write and execute queries several times so I'm trying to do a join between both queries without success as I'm not an expert on querying AppInsights and I'm not sure about how to do the join with a union, can you help me?
Please try the query below:
requests
| project timestamp, id, operation_Name, success, resultCode, duration, operation_Id, cloud_RoleName, invocationId=customDimensions['InvocationId']
| where cloud_RoleName =~ 'xxxx' and operation_Name == 'createCase' and resultCode == 400
| join (
traces
| union exceptions
) on operation_Id
| project-away operation_Id1
| order by timestamp asc
More details on the join operator - https://learn.microsoft.com/en-us/azure/kusto/query/joinoperator
Related
I have a table in a MySQL DB containing a model_name as key and a model_desc as a secondary column.
| model1 | Phone |
| model2 | Tablet |
In influxdb I have a bucket with a series in which model_name is a label
| username | model_name | last_connected |
I have to join these two so for each event in influxdb I can associate it with a model_desc. If there is no match, I want to set model_desc to something like 'unknown'
I got as far as using join on both queries but since join is explicitly 'inner' I only see the intersection. I'd require something like an outer join or a lookup that returns either a match or an 'unknown' for non-matching rows.
I have a model called Event which belongs to Area and Task. I'm attempting to retrieve a collection of events that only contains the most recent event per area and task combination. That is, I only want the most recent event of the events that have the same area_id and task_id. Example collection of events:
|event_id|area_id|task_id| ... |
|--------|-------|-------|-----|
| 5 | 3 | 2 | ... |
| 4 | 3 | 1 | ... |
| 3 | 3 | 2 | ... |
Here I want only event 5 and 4 to be returned since 3 is older.
I've tried using Event.select(:area_id,:task_id).distinct which seems to work, but strips all other attributes of the returned events, including :id. Grateful for any help or suggestions!
You can use raw SQL inside select, so you could try something like this:
Event.select("DISTINCT(CONCAT(area_id, task_id)), id, attr1, attr2")
Where id, attr1 and attr2 are the other attributes from your Event table.
Or you could use .group instead of .distinct and forget about using raw SQL:
Event.all.group(:area_id,:task_id)
You will get the same result as using DISTINCT and all attributes will be available.
UPDATE
To order before grouping, you can use find_by_sql with nested queries (again, raw SQL):
Event.find_by_sql(
"SELECT * FROM (
SELECT * FROM `events`
ORDER BY `events`.`created_at`) AS t1
GROUP BY t1.`area_id`, t1.`task_id`";
)
In another words you need to group events by area_id and task_id, and select recent record in each group. There question about building sql-query for this case. PostgreSQL version for your table will be like this:
SELECT DISTINCT ON (area_id, task_id) events.*
FROM events
ORDER BY area_id ASC, task_id ASC, created_at DESC
And Rails code for this query:
Event.
select("DISTINCT ON (area_id, task_id) events.*").
order("area_id ASC, task_id ASC, created_at DESC")
I have a table where I have multiple records for the same id. I would like to get the latest record for each id using some where clause.
Sample table records
vendor_id | data | created_at | id
-----------+------------+----------------------------+----
1 | some-data | 2014-01-12 16:32:54.084505 | 2
vendor_id | data | created_at | id
-----------+------------+----------------------------+----
1 | Notsome-dat| 2014-01-13 16:32:54.084505 | 3
I have multiple vendors with same data. So I want to get all the latest records for all the vendors where I can filter it with data. I have been using following query
SELECT VENDOR_ID,MAX(CREATED_AT) FROM TABLE WHERE DATA ILIKE '%Not some%'GROUP BY VENDOR_ID;
However, this query also gives me the vendor_id where they have "Not some" data in their second latest record not the latest one.
Please help.
You have to use DISTINCT
SELECT
DISTINCT ON (vendor_id) data
FROM TABLE
ORDER BY vendor_id, "created_at" DESC;
I have the following manual Sphinx query (via the mySQL client), that is producing proper results, and I would like to call it through Thinking Sphinx from Rails. For the life of me, I am struggling with how to make a 'distinct' query work in Thinking Sphinx.
mysql> select merchant_name, count (distinct part_number) from product_core group by merchant_name;
+-----------------------+-----------------------------------------+
| merchant_name | count (distinct part_number) |
+-----------------------+-----------------------------------------+
| 1962041491 | 1 |
| 3208850848 | 1 |
| 1043652526 | 48754 |
| 770188128 | 1 |
| 374573991 | 34113 |
+-----------------------+-----------------------------------------+
Please note: This mySQL query is agaist Sphinx, NOT mySQL. I use the mySQL client to connect to Sphinx, as: mysql -h 127.0.0.1 -P 9306. This works well for debugging/development. My actual db, is Postgres.
Given this, and to add more context, I am attempting to combine a group_by in thinking Sphinx, with a count('Distinct' ...).
So, this query works:
Product.search group_by: :merchant_name
... and, this query works:
Product.count ('DISTINCT part_number')
... but, this combined query throws an error:
Product.search group_by: :merchant_name, count ('DISTINCT part_number')
SyntaxError: (irb):90: syntax error, unexpected ( arg, expecting keyword_do or '{' or '('
...merchant_name, count ('DISTINCT part_num...
Both merchant_name and part_number are defined as attributes.
Environment:
Sphinx 2.2.10-id64-release (2c212e0)
thinking-sphinx 3.1.4
rails 4.2.4
postgres (PostgreSQL) 9.3.4
I have also tried using Facets, but to no avail:
Product.search group_by: :merchant_name, facets: :part_number
Product.facets :part_number, group_by: :merchant_name
For additional information, and to see if this could be accomplished through a Thinking Sphinx call, here is a basic example. I have one product table (and associated index), that lists both merchants, and their products (I agree, it could be normalized, but its coming in from a data feed, and Sphinx can handle it as is):
+-----------------+-------------------+
| merchant | product |
+-----------------+-------------------+
| Best Buy | Android phone |
| Best Buy | Android phone |
| Best Buy | Android phone |
| Best Buy | iPhone |
| Amazon | Android phone |
| Amazon | iPhone |
| Amazon | iPhone |
| Amazon | iPhone |
| Amazon | Onkyo Receiver |
+-----------------+-------------------+
With Thinking Sphinx, I want to: a) group the rows by merchant, and b) create a “distinct” product count for each group.
The above example, should give the following result:
+-----------------+------------------------+
| merchant | count(DISTINCT product |
+-----------------+------------------------+
| Best Buy | 2 |
| Amazon | 3 |
+-----------------+------------------------+
You're not going to be able to run this query through a model's search call, because that's set up to always return instances of a model, whereas what you're wanting is raw results. The following code should do the trick:
ThinkingSphinx::Connection.take do |connection|
result = connection.execute <<-SQL
SELECT merchant_name, COUNT(distinct part_number)
FROM product_core
GROUP BY merchant_name
SQL
result.to_a
end
Or, I think this will work to go through a normal search call:
Product.search(
select: "merchant_name, COUNT(distinct part_number) AS count",
group_by: :merchant_name,
middleware: ThinkingSphinx::Middlewares::RAW_ONLY
)
Events have many shifts.
My shifts table which contains a starts_at and ends_at column which are DATETIME format.
If I query the shifts table, the starts_at and ends_at columns return a string which contains timezone information and is recognised by Rails as an ActiveSupport::TimeWithZone object -- and I can manipulate it accordingly.
If I include the shifts.starts_at or shifts.ends_at column in a more complex query involving a join, I seem to be losing the timezone info in the result set. Consider the following query:
SELECT events.id, events.name, events.default_shift_start,
shifts.id AS shift_id, shifts.starts_at, shifts.ends_at, users.id AS user_id,
users.first_name, users.last_name
FROM "events"
INNER JOIN "shifts" ON "shifts"."event_id" = "events"."id"
INNER JOIN "requests" ON "requests"."shift_id" = "shifts"."id"
INNER JOIN "users" ON "users"."id" = "requests"."user_id"
WHERE (events.default_shift_start > '2012-08-22 05:55:22.069340' AND requests.status = 'accepted')
ORDER BY default_shift_start ASC
EDIT: I'm calling this query in Rails by way of:
Event.joins(:shifts => { :requests => :user}).where(["events.default_shift_start > ? AND requests.status = ?", Time.now, "accepted"]).select("events.id, events.name, events.default_shift_start, shifts.id AS shift_id, shifts.starts_at, shifts.ends_at, users.id AS user_id, users.first_name, users.last_name").order("default_shift_start ASC")
Produces:
+-----+----------+---------------------+---------------------+
| id | shift_id | starts_at | ends_at |
+-----+----------+---------------------+---------------------+
| 17 | 80 | 2012-08-23 00:30:00 | 2012-08-23 07:30:00 |
| 17 | 55 | 2012-08-23 00:30:00 | 2012-08-23 07:30:00 |
+-----+----------+---------------------+---------------------+
The issue is that the columns from the JOINed table (shifts) aren't returning any timezone data, causing Rails to recognize them as String data. The data is stored in the database as UTC. If I include a datetime column from the events table in the same query, it includes timezone data in the result.
I've been searching through the documentation trying to understand what's going on here, but to no avail.
I guess you should start with getting a grasp on the involved data types. The "datetime" data type in PostgreSQL is actually timestamp and there are two variants: with and without time zone. The default is without.
Internally, PostgreSQL always stores a UTC timestamp. Time zone data itself is not saved at all with timestamps, neither with nor without time zone. Those are just decorators for input and output that accommodate for the timezone setting of the client.
Find a more detailed explanation, examples and links at this related answer.
i think if you put starts_at::timestamptz then you are good.