I have Google Sheet which is schedule to run report daily and I have to create Dashboard in AWS QuickSight but I cant find any way to connect Google Sheet as a Data Source in QuickSight.
My google sheet's report is being populated from Google Analytics.
So, Is there any solution for this?
As per aws documentation, it seems we cannot directly integrate google sheets data as a datasource. The data either needs to be present in one of the Database or S3 or any relational data stores of aws.
We might need to make one of the following mechanisms :
Get the data in one of the relational database in aws through a script/cron and then pass it on to QuickSight for analysis.
Get the file in S3 through a script/cron and then pass it on to QuickSight for analysis.
Reference : https://docs.aws.amazon.com/quicksight/latest/user/supported-data-sources.html
You can use a third party product to fill the gap:
https://skyvia.com/data-integration/analyze-google-sheets-with-quicksight
Or any of the other connector that link Analytics with Quicksight, there are plenty such as this:
https://www.dataddo.com/integration/google-analytics/amazon-aurora
Related
I am trying to build this app where I am planning to store my data in a Firestore database. My client needs the data in a Google Sheet.
Can I use Cloud Function to directly populate the records from Firestore to a Google Sheet and whenever a new record will be added to the Firestore, Google Sheet will be updated automatically?
Can anybody suggest any resource that can help on this topic?
Yes, you can use cloud function with Firestore trigger that is notified when your firestore document is changed, in the function you would call google API to update google sheets.
You can find an example of end to end tested cloud function app with firestore triggers.
While the other options provide ways to do it with code, you may want to also look into using a service like Zapier which takes about 30 seconds to set up and requires zero coding.
Setting Firebase updates to trigger to Google Sheets is supported out of the box and should do what you want really easily.
Check this link:
https://zapier.com/app-directory/firebase/integrations/google-sheets
The best way would be to create a service account that can communicate between the Firestore database and the Google Cloud function. The Google Cloud Function can then update a Google Sheet using the Google Sheets API. As for resources to put you on the right track:
How to read/write to a google sheet.
Service account docs.
And lastly, I found this similar case that should answer any other questions you have.
I want to connect a Google Sheets to a new BigQuery table that populates and updates the data automatically from Sheets to BigQuery. I'm using this tutorial from Google itself to do the setup.
My problem: the table connected with spreadsheet was created empty so I had to query it and save the result as another table to see and use the data.
I can't post images yet so I ask you please to check this imgur post, please.
I'm not expert in these things but does not seems to be the best way to do it. I found some spreadsheet add-ons but I'm trying to avoid them.
Any ideas what's the best way to do this kind of setup/connection?
I had to configure each column manually
BigQuery provides a variety of tools which make it pretty simple to connect the external table to BigQuery.
One option is to simply use the WebUi and the Auto Detect option which help you not to enter each column manually
This works perfectly for me also when inserting and adding data to the external table.
You can refer to BigQuery official manual on an external table for more help
I'm using Talend for several ETL-jobs. Main focus on inserting and updating data defined in local Excel files to Salesforce. Excel and Salesforce are used as Input- and Output-Connections massively.
This works like a charm!
Now instead of Excel I need to use Google Spreadsheets as data inputs/outputs. Just manually download as Excel isn't good enough. Reason: I need a highspeed repeatable process executed thousands of times bi-directional. Uploading/Downloading XLS/CSV is not an option.
As an unfortunate, my researches concluded that there is NO Google spreadsheet connector available. I found this legacy-project which seems to be abandoned: https://code.google.com/p/google-talend-components/ - it is outdated from Talend-side as well as from Google (old API versions).
One other thing I've seen, is that Talend comes with Google Big Data Support: https://www.youtube.com/watch?v=BK6G3sTmwVE - Also Google Big Data might be somehow connectable to Google Spreadsheet - but I wasn't able to figure out how. Is this easy or hard? How to start? What to read?
I need to know if there is any viable approach to read and write data with Talend from and to Google-Spreadsheet.
Alternatively is there an other ETL-tool like Talend which has connectors to Google Spreadsheet AND Salesforce out-of-the-box?
Is there a direct google docs connector in Talend, no. But you can do what you are looking to do if you think of the problem a little differently. It sounds like you need to read and write to spreadsheets then upload/download from google. Lets take each problem separately.
Upload/Download
Google storage. If you need to upload and download files from google Talend has excellent integration with their API. As you mentioned, Talend --> Big Data has components to interact with Google Storage. This type of storage is primary meant as back end storage for an application.
In Talend under Cloud-->Google Drive there are components to upload and download from Google Drive. Google Drive is primarily meant as Chrome app and has direct integration with Google Docs.
Writing to Google Docs
Regarding your need to interact with google docs, you do not need any special components for this. Google docs can utilize csv and Excel formats. If you create and modify your documents in Talend using the standard file interaction components ( delimited, Excel, etc.) you can upload and download these documents using one of the two methods I described above (Google Storage or Drive) depending on what type of storage you are using.
I'm working on some .Net code to collect some data on a server and put it in a Google Spreadsheet. The job needs to run once a day, clear out the sheet and repopulate it. I have all that working, but I can't get the authorization right. I can do it with my personal Google credentials, and I can get it to work with OAUTH by allowing the app to manipulate all my spreadsheets using the https://spreadsheets.google.com/feeds scope. But, what I really want to do is allow my script to read and write just one sheet. Is that possible, and how would I do that?
You are using very old scopes. Look at the drive.file scope
https://developers.google.com/drive/web/scopes
Its not possible using the spreadsheets API. That api is very old and doesnt have a scope for indivdual files.
it might be possible by creating a spreadsheet using the drive API by uploading a CSV with conversion, but you will need permission to create new drive files and that also means permission to the entire drive.
Workaround: Create a new google account and share the spreadsheet with it. Do the OAuth using the new account.
#Zig Mandel is right - the API (Gdata style) wants access to all spreadsheets. But you can workaround the problem using a new account.
I want to use Google spreadsheets to store data online so multiple people can enter and maintain data, then publish or export (csv, xls, ods) the sheet for their application. What is the easiest way to process the sheet? If the data can be accessed as a link the updates could be immediately reflected in the client's app.
This article Data Scraping Wikipedia with Google Spreadsheets discusses using Google spreadsheets as an application platform.
It already has features that allow you to share the spreadsheet with multiple editors, as well as the ability to define forms that you can invite people to fill out who's results will be entered into the spreadsheet as a row.
Just in case its still vaguely of interest 5 months later, there is a SQL-like wrapper language for Google Spreadsheets at:
http://gqlx.twyst.co.za
Its very much an alpha release - but maybe its of some use or perhaps the code could give you some ideas.
I think if I am clear on your requirements, you want to store your data online in a Google Spreadsheet where multiple people can access that right? Then you want an app to use that data? Correct me if I am wrong.
It is possible to have Google spreadsheet as a database and the client app can directly access that data through APIs - spreadsheet APIs . If you want a simple link where people can go and update - you can also have a web app using google apps scripting. very simple to use and would solve your purpose too , why to export your data to ods, xls, csv, make your app directly over Google Spreadsheet. Would be maintainable too.