I am currently working on a Power BI report, which uses Google Sheets as the data source. As there are no official connectors available, there are two ways to fetch data from sheets to Power BI:
1. Publish to web
2. Link sharing
In both methods, we need to provide a link to Power BI which consists of the ID of the particular spreadsheet. In 'publish to web', any individual with the link can view my data.
So my main concern is, how secure the link sits in power BI? Are there any chances of breach?
Your concern should not be how the link is secured in Power BI Service, but the existence of this link itself. Lets say the probability of leaking it from your IM/e-mail/correspondence is much higher than leaking from Power BI. You are sharing sensitive data for anonymous access. Don't do that! Find another way for reporting. Either use charting, etc. from Google Sheets, or if you need/must use Power BI, store the data in more convenient place. For example a database in the cloud (Azure SQL Database is a good one). Then build an ETL process to read your data from Google Sheets and push it to the database. This process will be fully authenticated and you will not expose your data.
Related
I have a Google Spreadsheets with data connected to a Data Studio Panel. I'm using the following data flow to get the data:
Google SpreadSheets --> BigQuery External Table --> View To the External Table --> Data Studio (Updated every 10 minutes)
But for some reason that I don't know, sometimes, when executing a select on the BigQuery External Table I get the following error:
Resources exceeded during query execution: Google Sheets service overloaded for spreadsheet id:XXX
The Google SpreadSheet has only 1500x10 Columns, which I think is pretty small. Also, there are about 6 users.
What can cause that error? Any idea about how to solve this?
Thanks
The Google documentation has information about this error:
A BigQuery query can overload Sheets, resulting in an error like Resources exceeded during query execution: Google Sheets service overloaded. Consider simplifying your spreadsheet; for example, by minimizing the use of formulas.
It seems that along with size of the Sheet, the "complexity" also matters. We cannot know how complex is your spreadsheet without seeing it but consider reducing your formula usage. This article also mentions a max result size of 10MB and other pivot table limits. You could also try to divide the data or if the error rate is manageable you could also use some kind of retry strategy to query again until you get the results.
If this is not enough then you may have reached the limits of what you can do with Sheets. Digging deeper I found this Google issue tracker post which has a quote from their engineering team:
The BigQuery Engineering Team has stated that the current suggested approach is to simplify the spreadsheet. Sheets is designed for Web/Mobile use cases and not as a DB backend. Even a couple of thousand rows is large in this context, especially if there are formulas involved.
The post is a feature request to the Google engineering team to allow for more complexity, but these requests can take time and if they don't intend Sheets to be used that way it's also possible that they won't implement it. If you cannot reduce the spreadsheet's complexity enough to stop getting the error you may want to consider querying the data from a different source.
I was wondering if there was a method whereby I can get CMS data from Avaya to PowerBI in real time.
I read that this can be accomplished by Avayas custom real-time data connectors (RT_sockets/Generic-RTA etc), but those are all licensed.
Is there any other way (i.e. not needing a license) to pull in CMS data from Avaya, in real-time to Power BI?
Informix will allow you to connect and pull data, but it's not real-time. It is near real-time and great for historical data analysis.
For real-time, I use the vbs scripting in Avaya CMS Supervisor (dump the returning data into a file or database). I've used this method to create nice reader boards in powerpoint and web apps.
I am really would like to be able to pull out the cost of a click from adwords with a gclid. I can upload this to adwords but we have rich meta data about the customer after they have completed our online application, I would like to be able to do some analysis on this data, but need to find out the cost of the customer.
Unfortunately, Google do not release the cost information at that granular level. They do provide a click performance report (details here) but performance metrics such as cost, conversion, etc. are all infuriatingly absent.
You can, of course, match the gclId to a unique criteriaId, device, date, etc. then get the cost for that combination but, at least in general, you'll lose a little information in the aggregation unless there happens to be just a single click in that segment.
I have most of my big tables (tables with over 200 M records) sitting at Google's Big Query servers and would like to use Power Bi (Desktop) for doing analytics. Found Simba´s ODBC driver, installed and configured and gained access to the datasets, but Power Bi is not giving a choice for Direct Querying the data. Instead, is trying to download it, which of course will not do it, firstly because of storage limitations on the client side, but mostly because the only reason I have the data at Big Query is to be able to use Google´s processing power. Simba´s driver specs says it should allow Direct Querying (called Direct BI) but that did not work for me.
Has anyone out there ever tried to connect BQ from Power Bi? If so could direct querying? Have any ideas or suggestion for doing this without transferring the data?
Thanks
There are only a certain number of data sources supported by Power BI for DirectQuery. You can find the full list here: https://powerbi.microsoft.com/en-us/documentation/powerbi-desktop-directquery-data-sources/.
However, support for Google Big Query (using either Import or DirectQuery mode) was added to Power BI Desktop in the August 2017 update. More information here: https://powerbi.microsoft.com/en-us/blog/power-bi-desktop-august-2017-feature-summary/#GoogleBigQuery
Per Miguel Llopis (program manager for Power BI), support for Google Big Query in the Power BI Service (allowing you to refresh reports using Big Query) will be coming by the end of the 2017 calendar year. See here: https://ideas.powerbi.com/forums/265200-power-bi-ideas/suggestions/7043266-support-google-bigquery
I would not recommend using Power BI for Big Query analysis. As of Q1 2019 Power BI still do not support project billing unless manually editing the connexion script, it does not support Sheets federated tables and do not support nested and repeated fields. Microsoft do not seems to be willing to develop the compatibility between Power BI and Big Query further.
Learn more in this article (i am the author):
https://medium.com/#remy_david/which-bi-tool-for-big-query-d9eb838ff7ad
I plan to write a thesis about using sentiment information to enhance the predictivity of some financial trading model for currency.
The sentiment data should be twitter threads including some keyword, like "EUR.USD". And I will filter out some sentiment words to identify the sentiment. Simple idea. Then we try to see whether here is any relation between the degree of sentiment and the movement of EUR.USD.
My big concern is on twitter data. As we all know that the twitter set up the limit to see the history data. You could only browser back for like 5 days. It is not enough since our strategy based on daily sentiment.
I noticed that google have some fantastic thing like timeline about the twitter updates: http://www.readwriteweb.com/archives/googles_twitter_timeline_lets_you_explore_the_past.php
But first of all, I am in Switzerland and seems I have no such function on my google which is too smart to identify my location and may block some US google version function like this. Secondly, even I could see some fancy interactive google timeline control on my firefox, How could I dig out data from my query and save them? Does google supply such api?
The Google service you mentioned has shut down recently so you won't be able to use it. (http://www.searchenginejournal.com/google-realtime-shuts-down-as-twitter-deal-expires/31007/)
If you need a longer timespan of data to analyze I see the following options:
pay for historical data :) (https://dev.twitter.com/docs/twitter-data-providers)
if you don't want to pay, you need to fetch tweets containing EUR/USD whatever else (you could use the streaming API for this) and store them somehow. Run this service for a while (if possible) and you'll have more than just 5 days of data.