i have an apps that show event in firebase console and i have done linked project to big query, but in big query only show dataset firebase_crashlytics and firebase_messaging. How i could retrieving event data to big query ?
here are my event
my dataset in big query
Fron the documentation: https://support.google.com/firebase/answer/6318765?hl=en
Once an app is linked to BigQuery, a corresponding dataset is created
in the associated BigQuery project upon the first daily export of
events. Each day, raw event data for each linked app populates a new
table in the associated dataset, and raw event data is streamed into
an intraday BigQuery table in real-time. Data prior to linking to
BigQuery is not available for import.
Apart from that:
check that the connection is not disabled;
if you have recently linked, it may take at least one day to view the data on BQ.
Finally i have done the problems, i do :
Enable integration to Google Analytics
Create dummy apps
Upgrade Blaze plan
Create big query dataset
Set up the export (Enable integration to big query)
full step by step that i follow :
Enable Big query Export
Related
I have an application that, at its core, is a sort of data warehouse and report generator. People use it to "mine" through a large amount of data with ad-hoc queries, produce a report page with a bunch of distribution graphs, and click through those graphs to look at specific result sets of the underlying items being "mined." The problem is that the database is now many hundreds of millions of rows of data, and even with indexing, some queries can take longer than a browser is willing to wait for a response.
Ideally, at some arbitrary cutoff, I want to "offline" the user's query, and perform it in the background, save the result set to a new table, and use a job to email a link to the user which could use this as a cached result to skip directly to the browser rendering the graphs. These jobs/results could be saved for a long time in case people wanted to revisit the particular problem they were working on, or emailed to coworkers. I would be tempted to just create a PDF of the result, but it's the interactive clicking of the graphs that I'm trying to preserve here.
None of the standard Rails caching techniques really captures this idea, so maybe I just have to do this all by hand, but I wanted to check to see if I wasn't missing something that I could start with. I could create a keyed model result in the in-memory cache, but I want these results to be preserved on the order of months, and I deploy at least once a week.
Considering Data loading from lots of join tables. That's why it's taking time to load.
Also you are performing calculation/visualization tasks with the data you fetch from DB, then show on UI.
I like to recommend some of the approaches to your problem:
Minimize the number of joins/nested join DB queries
Add some direct tables/columns, ex. If you are showing counts of comments of user the you can add new column in user table to store it in user table itself. You can add scheduled job to update data or add callback to update count
also try to minimize the calculations(if any) performing on UI side
you can also use the concept of lazy loading for fetching the data in chunks
Thanks, hope this will help you to decide where to start 🙂
I am using GA add on for Sheets to transfer data from GA to Sheets.
I want to transfer user retention cohort data, but I am unable to figure out the right metrics and dimensions for the same.
I tried metric: ga:cohortRetentionRate
and dimension: ga:cohort, ga:cohortNthDay, and some others
But, I keep getting the error "Selected dimensions and metrics cannot be queried together" even though I checked on dev tools that they can be queried together.
Would really appreciate some help!
It is possible to compose a cohort requests by using the Request Composer tool, in the Cohort Request tab.
Programmatically, to make use of these new dimensions and metrics you must construct a V4 cohort request: https://developers.google.com/analytics/devguides/reporting/core/v4/advanced#cohorts
In my ios app, users upload files. I am logging a custom event called "upload_time" because I would like to see approximately how long uploads are taking.
FIRAnalytics.logEvent(withName: "upload_time", parameters: [
kFIRParameterItemID: "upload_time_\(Constants.versionNumber)",
kFIRParameterItemName: val
])
I would like to be able to filter by the version number of the app and see the percentages of upload times. I have divided up times in 10s brackets so "val" is just rounded up to the nearest 10.
Just like how the select_content default event allows you to filter by content_type and then item_id, I would also like to be able to filter by version number and see the percentages for the different brackets of times in the console. At the moment, it seems that what I have setup is just adding up all the values for each day.
How I setup parameters in the console
Would greatly appreciate any help.
There's no way to configure ad hoc reports in the Firebase console.
If you want reports other than those provided in the console, then your best bet here would be to export the results to BigQuery and use a visualization tool.
Once you have these set up, the sky is the limit :)
Firebase Performance Monitoring sounds like a better fit if you're trying to measure upload times. Check out the getting started guide here. Performance Monitoring actually captures a bunch of network data automatically.
In addition, Performance Monitoring lets you filter by a number of parameters, such as device type, OS version, app version, and more. It's still in beta, so if there's some functionality that you'd like to have that isn't there yet, feel free to file a feature request.
To add another way to make this work in GA for Firebase is to Export the Firebase data to Big Query and run a query that calculates the percentage of upload time from all you app instances filtered by version.
Take a look at the Step 6 of this doc on sample query for Big Query data gather by Firebase.
I was concerned about real time stream processing for IOT through GCD pub/sub, Cloud Dataflow and perform analytics through BigQuery.I am seeking help for how to implement this.
Here is the architecture for IOT real-time stream processing
I'm assuming you mean that you want to stream some sort of data from outside the Google Cloud Platform into BigQuery.
Unless you're transforming the data somehow, I don't think that Data Flow is necessary.
Note, that BigQuery has its own Streaming API so you don't necessarily have to use Pub/Sub to get data into BigQuery.
In any case, these are the steps you should generally follow.
Method 1
Issue a service account (and download the .json file from IAM on Google Console)
Write your application to get the data you want to stream in
Inside that application, use the service account to stream directly into a BQ dataset and table
Analyse the data on the BigQuery console (https://bigquery.cloud.google.com)
Method 2
Setup PubSub queue
Write an application that collections the information you want to stream in
Push to PubSub
Configure DataFlow to pull from PubSub, transform the data however you need to and push to BigQuery
Analyse the data on the BigQuery console as above.
Raw Data
If you just want to put very raw data (no processing) into BQ, then I'd suggest using the first method.
Semi Processed / Processed Data
If you actually want to transform the data somehow, then I'd use the second method as it allows you to massage the data first.
Try to always use Method 1
However, I'd usually always recommend using the first method, even if you want to transform the data somehow.
That way, you have a data_dump table (raw data) in your dataset and you can still use DataFlow after that to transform the data and put it back into an aggregated table.
This gives you maximum flexibility because it allows you to create potentially n transformed datasets from the single data_dump table in BQ.
In the context of Google Analytics, I wonder if I can get granular data for an account in the form of a table --or multiple tables that could be joined --containing all relevant information collected per user and then per session.
For each user there should be rows describing in detail the activities and outcomes --micro and macro-- of each session. Features would include source, time of visit, duration of visit, pages visited, time per page, goal conversions etc.
Having the row data in a granular form would enable me to apply machine learning algorithms that would help me explore the data and optimize decisions (web design, budget allocation, biding).
This is possible, however not by default. You will need to set up custom dimensions to be able to identify individual clients, sessions, and timestamps to be able to get list wise user data, rather then pre-aggregated data. A good place to start is https://www.simoahava.com/analytics/improve-data-collection-with-four-custom-dimensions/
There is no way to collect all data per user in one simple query. You will need to run multiple queries, pivot tables, etc. and merge's to get the full dataset you are currently envisaging.
Beyond the problem you currently have, there is also then the problem of downloading the data.
1) There is a 10,000 row limit, so you will need to make a loop to download all available rows.
2) Depending on your traffic, you are likely to encounter sampled data, so you will need to download the data per day, or hour to avoid Google Analytics sampling.