I'm looking to build an analytics dashboard for my data in a rails application.
Let's say I have a list of request types "Fizz", "Buzz", "Bang", "Bar".
I want to display a count for each day based on type.
How should I do this?
Here is what I plan on doing:
Add get_bazz_by_day, get_fizz_by_day, etc to the appropriate models.
In each model get all records of type Fizz, then create an array that stores date and count.
format in view so a JS library can format it into a pretty graph.
Does this sound reasonable?
Depending on number of records, your dashboard can soon get performance problems.
Step 1 is misleading. Don't get the data for each day individually, try to get them all at once.
In Step 2 you can have the database do the the aggregation over days, with the group method.
See http://guides.rubyonrails.org/active_record_querying.html#group
Fizz.select("date(created_at) as fizzed_day, count(*) as day_count").
group("date(created_at)")
In Step 3 you need to take care that days without any fizzbuzz are still displayed, as they are not returned in the query.
Related
Good day to everyone! Hope all is well!
I am looking to run an update query or a group of queries that looks at my Date_Start and Date_End to determine if the Units (quantity of the respective record) fall in my defined current quarter 1/2/3/4 from another table (this table is a master table I’m using to provide the dates that I need to consider for defining the quarters).
I’ve been able to create queries that do this and then join them together to basically display the units out by quarter based on their respective start/end dates. The problem I am running into is this process takes a decent amount of time for the queries to populate that will drastically effect other processes down the line.
Thus we get to my desire. I am trying to no avail to create an update query that will update the quarter fields in my table based off of the queries I built to determine if the records start/end date fall in the respective quarter. I figure that running this update when records change will be an ok run time vs when I’m running reports or running an email script for the reports.
I have tried pulling in the table and query, joining them as equal on ID (the query pulls in the table's IDs), and selecting my field “CQ1” from the table, and setting the Update ether the respective field from the table or the query (which is the same as the field in the table).
All I get are the current values of the field in the data sheet view and an error of “Operation must use an updateable query.”
I have even tried placing a zero to see if that would do it with no luck. I have verified that all the fields are the same data type.
What am I doing wrong? Thanks!
Apologies to everyone.....I think my conscious brain was trying to overly complicate the process and while talking to a buddy about my issue distractedly created a new update query that worked. It all tied down to that I forgot to put a criteria on my quarter filed of is not null I believe. Thanks for anyone that has read this and is responding while I’m typing this or for those of you formulating a response.
In my app users can save sales reports for given dates. What I want to do now is to query the database and select only the latest sales reports (all those reports that have the maximum date in my table).
I know how to sort all reports by date and to select the one with the highest date - however I don't know how to retrieve multiple reports with the highest date.
How can I achieve that? I'm using Postgres.
Something like this?
SalesReport.where(date: SalesReport.maximum('date'))
EDIT: Just to bring visibility to #muistooshort's comment below, you can reduce the two queries to a single query (with a subselect), using the following form:
SalesReport.where(date: SalesReport.select('MAX(date)'))
If there is a lot of latency between your web host and your database host, this could halve execution times. It is almost always the preferred form.
You can get the maximum date to search for matching reports:
max_date = Report.maximum('date')
reports = Report.where(date: max_date)
Due to the PowerShell methods of getting mailbox statistics from Office365 taking about 2 seconds per mailbox, I am working on getting the data from Office 365 Reporting web service, which takes only a few seconds for each 2000 mailboxes.
The problem I'm running into is that the stats are updated periodically and some historical data is kept, so there are numerous records for each user. I only want to get the latest record for each user, but I haven't been able to find a way to do that. The closest I've come is to use $filter=Date ge DateTime'2016-03-10T00:00:00' where the date is concatenated to a couple of days ago. Theoretically, if I sort by Date desc I should get the latest records first, and if there is a user that has a record for 3/10 and 3/11, the 3/11 record would get pulled first, which would work for me. But regardless of how I do the sort it seems to come back with the older records first.
Ideally, I would like to be able to set criteria so that it only returns the latest record for each mailbox, but I can't seem to figure out or find how to do that. The closest I've been able to come is to just start running queries filtered on specific dates, walking the date back a day on each query.
If I can get the latest records to be returned first, I would be able to work with that because I can just discard a record if I've already received a later one.
https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MailboxUsageDetail/
?DelegatedOrg=nnn.onmicrosoft.com&$select=Date,WindowsLiveID,CurrentMailboxSize
&$filter=Date ge DateTime'2016-03-08T00:00:00'&$orderby=Date desc
So the questions are:
Is there a way to specify criteria so that only the latest record for each user is returned?
Is there a way to get it to order by Date descending--what am I doing wrong with the $orderby?
Thanks!
You can use $top=1 to get latest record by applying $orderby on date (desc). $filter and $skip may not require in this case.
https://reports.office365.com/ecp/reportingwebservice/reporting.svc/MailboxUsageDetail/?DelegatedOrg=nnn.onmicrosoft.com&$select=Date,WindowsLiveID,CurrentMailboxSize&$orderby=Date desc&$top=1
Your query looks fine, here is an another example from Odata sample service to get employee detail with most recent birth date.
http://services.odata.org/V4/Northwind/Northwind.svc/Employees?$select=EmployeeID,FirstName,LastName,BirthDate&$orderby=BirthDate%20desc&$top=1
I am to store quite large amount of boolean values in database used by Rails application - it needs to store 60 boolean values in single record per day. What is best way to do this in Rails?
Queries that I will need to program or execute:
* CRUD
* summing up how many true values are for each day
* possibly (but not nessesarily) other reports like how often true is recorded in each of field
UPDATE: This is to store events that may or may not occur in 5 minute intervals between 9am and 1pm. If it occurs, then I need to set it to true, if not then false. Measurements are done manually and users will be reporting these information using checkboxes on the website. There might be small updates, but most of the time it's just one time entry and then queries as listed above.
UPDATE 2: 60 values per day is per one user, there will be between 1000-2000 users. If there isn't some library that helps with that, I will go for simplest approach and deal with it later if I will get issues with performance. Every day user reports events by checking desired checkboxes on the website, so there is normally a single data entry moment per day (or few if not done on daily basis).
This is dependent on a lot of different things. Do you need callbacks to run? Do you need AR objects instantiated? What is the frequency of these updates? Is it done frequently but not many at a time or rarely but a bunch at once? Could you represent these booleans as a mask instead? We definitely need more context.
Why do these need to be in a single record? Can't you use a 'days' table to tie them all together, then use a day_id column in your 'events' table?
Specify in the Day model that it 'has_many :events' and specify in the Event model file that it 'belongs_to :day'. Then you can find all the events for a day with just the id for the day.
For the third day record, you'd do this:
this_day = Day.find 3
Then you can you use 'this_day.events' to get all the events for that day.
You'll need to decide what you wish to use to identify each day so you query for a day's events using something that you understand. The id column I used above to find it probably won't work.
You could use the timestamp first moment of each day to do that, for example. Or you could rely upon the 'created_at' column of the table to be between the start and end of a day
And you'll want to be sure to thing about what time zone you are using and how this will be stored in the database.
And if your data will be stored close to midnight, daylight savings time could also be an issue. I find it best to use GMT to avoid that issue.
Good luck.
I have an API call that returns a JSON format for live statistics - Things like number of people who registered, signed in, performed action a, etc.
Now, I would like to have a history with some presentable graphs.
My plan is to hit the api once a day to get the json and save each result in an activerecord table.
Is there any better suggested way for this. Also, any gem recommendations to make my life easy.
Create an activerecord model to with the report fields and a date column
Hit API to get JSON once a day and populate above table
Plot the above table with various possible dimentions
Highcharts would be something to check out then. It can take the JSON and present good graphs.
It was the easiest I found so far (and still not great). Once with that you can set up a CRON job with the whenever gem to do the data pull once a day.