I made a BigQuery table backed by Google spreadsheet and it worked great until I shared it with some project members.
All the project members have permission to access all the resources in the BigQuery project and the Google spreadsheet, but the only person who can execute SQL on the table backed by a Google spreadsheet is the one who made the table.
When other people run queries, they get the error message below.
Error: Access Denied: BigQuery BigQuery: Permission denied while globbing file pattern.
Is it specification (or restriction) of BigQuery? Or is there any configuration that I should activate when I share such a table with others?
Related
I want to create external table in BigQuery and the data source is from Google Sheet. Is it possible to do it using dbt? In the yml file, where should I put the URI?
The main problem is, I don’t have the access to create it directly in BigQuery.
One way to handle a Google Sheet as a source is by creating a new table out of it in BigQuery via Connected Sheets.
Then, you create a new source in dbt that relies on that table, and start building your downstream models from there.
As far as I know, you cannot create a source directly from dbt, unless it is a seed file, which I woul not recommend unless it is a rather static file (e.g. country names and ISO codes, which is not prone to change over time).
We have a similar situation where the data source is from Google Sheet.
The end user updates the Google sheet on a periodical basic and we replicate it using Fivetran to our Snowflake datastore.
DBT can then pick up the data seamlessly.
I would like to setup our organisation's Bigquery IAM so that some users would have access to views under our "Report" dataset, which are queried on Google Sheets. Users with edit rights on these Google Sheets should be able to hit the refresh button and access latest data.
We achieved this by granting some user access to the entire project, but that's very poor security. Instead, I want to limit their access to either the entire "Report" dataset or to specific views under that dataset.
We tried to define some custom role and share the dataset but there is no "refresh" button on the sheet for the user to refresh.
Sharing the dataset with a viewer roles showed the Refresh button but they were not allowed to refresh:
What did we miss?
For access to BigQuery data in Google Sheets, one should have:
An Enterprise Plus, G Suite Enterprise for Education, Enterprise
Standard, or Enterprise Essentials account.
Access to BigQuery.
A project with a billing setup in BigQuery.
A BigQuery Job Creator role on the selected billing project.
BigQuery Data Viewer role on the datasets containing the selected table.
According to this document:
If you share a sheet with someone who doesn't meet the criteria above,
they'll be able to see analysis created with Connected Sheets and
perform regular Sheets operations, but they won't be able to refresh
it or create their own connected sheet.
If you share a sheet with someone who doesn't have access to the correct data tables, they won't be able to create or refresh Connected Sheets analysis, and will need to contact the BigQuery admin.
I have several tables in BigQuery that are sourced from Google Sheets tables. When the Google Sheets table is updated then automatically the table in BigQuery is also updated. I am trying to understand how the log of this event looks like in the Operations Logging. My end idea is to create a sink of theses logs in order to create a Pub/Sub and run scheduled queries based on these events.
Thank you
When you use external Table (Google sheet or other) the data are never stored in BigQuery native storage. It's always external.
Therefore, when you update your Google Sheet, nothing happens in BigQuery. It's only when you query the data, you will read (again) the sheet document and get the latest data.
Therefore, there is no insert log that you can track when you update the data in Google Sheet. The only log that you have is when you perform a request in BigQuery to read the data (external or not), as mentioned by Sakshi.
When the external data source(Google Sheet or other) is updated and the BigQuery table associated with it is queried, BigQuery initiates an insert job which is visible in Cloud Logging.
You can find this log by applying filter resource type as BigQuery Project in Cloud Logging console, ie. you will see protoPayload.methodName set to google.cloud.bigquery.v2.JobService.InsertJob.
For more information on BigQuery Logs you can refer to this documentation.
I have a sheet that gets data from BigQuery and everybody can edit. I tried to remove permissions in the tab with a query, but it didn't work because it's connected to a database. Here is the error:
Can't add protection to this sheet because it's connected to a database.
How do I remove query edit permission without removing edit access in all the sheet?
As pointed out in the documentation for this feature under "Change who can see & edit queries or results", to edit the BigQuery query, the user needs both a corresponding BigQuery role and at least the editor permission in the Sheet.
When users don't have a proper BigQuery role, they will get an error like this:
So, as long as the users don't have the required permissions for BigQuery in your project, then they won't be able to edit the query. A user with no role in BigQuery can only see fetched data from a BigQuery query and view the query itself.
If you have users who have BigQuery roles in your project, you can separate the Google Sheet to a file where the users only have viewer permissions for the Sheets file.
I created one table in BigQuery from Google Sheets, when I tried importing it in Cloud Data Prep it says that there are no tables in the dataset.
I'm not sure whether it's an issue with the Google sheet integration, because when I check the details of the table it says there are 0 rows even though there are 3 rows (it's test data).
I already tried giving Cloud Data Prep viewer permission for both the dataset and the project and nothing changes.
Print screens:
Data Prep
BigQuery table info
BigQuery entries
Well apparently when you create a table from google sheets it is not recognized as an actual table, so I made a query to replicate it to a second table and it works!