How to automatically import CSV into Google Sheets? - google-sheets

I get a CSV data file from Google Adwords API. It is formatted like this: Keyword;"Monatliches Suchvolumen";Competition;"Suggested Bid" and has .csv file ending. Here an original data sample:
+--------------------------+-------------------------+------------------+---------------+
| Keyword | Monatliches Suchvolumen | Competition | Suggested Bid |
+--------------------------+-------------------------+------------------+---------------+
| rechtsschutzversicherung | 110000 | 0.84808008438954 | 6404121 |
+--------------------------+-------------------------+------------------+---------------+
| rechtsschutz | 12100 | 0.7662246934702 | 6050097 |
+--------------------------+-------------------------+------------------+---------------+
Then i upload it to Google Drive with File Upload. After upload the file is recognized as Google Spreadsheets file - but after opening i have all data in one column, column separation doesn't work.
If after downloading the file from AdWords API i open it in Notepad++, remove "" and replace ; with tabs, save the file and then upload it to Google Drive, column separation works and all data is in its correct column. But i can't do it manually - i have too many files.
What should i do to reach automatic column recognition in Google Spreadsheets?
Was trying: played with Google Drive settings, like Convert uploaded files to Google Docs editor format, but without success. I thought about the localization settingsdirectly in Google Spreadsheets - but this will not work, because these settings can be edited only after spreadsheets is already created - and here is all data already in one column.

I believe your current situation and goal as follows.
You have a CSV file on your Google Drive.
CSV data is as follows.
Keyword;"Monatliches Suchvolumen";Competition;"Suggested Bid"
,
,
,
You want to parse the CSV data and put to the Spreadsheet.
In this case, how about using Google Apps Script? I thought that when Google Apps Script is used, the CSV file on your Google Drive can be retrieved and parsed. The sample script is as follows.
Sample script:
Please copy and paste the following Google Apps Script to the script editor of Google Spreadsheet, and set the variable of fileId. And, when you run the function of myFunction, the authorization screen is opened. So please authorize the scopes. By this, the script retrieves the CSV data from the CSV file, and parse the CSV data and put to the active sheet.
function myFunction() {
const fileId = "###"; // Please set the file ID of the CSV file on your Google Drive.
const csv = DriveApp.getFileById(fileId).getBlob().getDataAsString();
const values = Utilities.parseCsv(csv, ";");
const sheet = SpreadsheetApp.getActiveSheet();
sheet.getRange(1, 1, values.length, values[0].length).setValues(values);
}
Note:
When you want to retrieve the CSV data from the CSV file using the filename, you can also use the following script.
function myFunction2() {
const filename = "sample.csv"; // Please set the filename of CSV file on your Google Drive.
const file = DriveApp.getFilesByName(filename);
if (!file.hasNext()) {
throw new Error(`"${filename}" was not found.`);
}
const csv = file.next().getBlob().getDataAsString();
const values = Utilities.parseCsv(csv, ";");
const sheet = SpreadsheetApp.getActiveSheet();
sheet.getRange(1, 1, values.length, values[0].length).setValues(values);
}
This is a simple sample script. So please modify this for your actual situation.
References:
getFileById(id)
parseCsv(csv, delimiter)
setValues(values)

Related

Unexpected behavior while finding and deleting files in Google Drive: file disappears

I want to find and delete files in Google Drive, which contain special string in the file name.
I tried it in the following way:
created a spreadsheet named testDeleteByName, where I will run Google App script,
created a project in the Cloud Console,
activated Google Drive API for the project, created OAuth and added the spreadsheet testDeleteByName to this project in the project setting of the spreadsheet.
In the spreadsheet I added the script
function SearchFiles() {
var searchFor ='title contains "untitled"';
var files = DriveApp.searchFiles(searchFor);
while (files.hasNext()) {
files.next().setTrashed(true);
}
}
Then I created a spreadsheet named untitled testsheet to test the script.
After this I executed the script in the testDeleteByName.
As result I get
An error in execution log
Exception: Access denied: DriveApp. SearchFiles # Code.gs:5
The test file untitled testsheet is disappeared (not even in the bin)
The file testDeleteByName with the script is removed into the bin.
What I'm doing wrong? How looks the correct way?

How to fix incorrect data when loading from azure storage using external table in snowflake

I was trying to load data from azure blob storage using snowflake's external table like this after creating the stage area(blob_tb2_434) :
CREATE OR REPLACE EXTERNAL TABLE mydb.public.tb2_434
WITH LOCATION = #mydb.public.blob_tb2_434/
FILE_FORMAT = (TYPE = CSV SKIP_HEADER = 1 FIELD_OPTIONALLY_ENCLOSED_BY='"' )
PATTERN='.*.tsv';
The above external table will load every file with tsv extension from the blob defined in the stage area. I used FIELD_OPTIONALLY_ENCLOSED_BY property because there are tabs, and other characters inside some of the fields enclosed with double quotes.
One of the files in the blob has "A"" where it was supposed to be A""" and the entire record is being read as null in the external table.
How can I use replace function from snowflake or other preprocessing to parse the data correctly?
I was trying to load the entire data using '\n' as the FIELD_DELIMITER and parsing it locally but it isn't working.

Unable import text using importxml and xpath inside div

i'm Using Google Sheets with IMPORTXML to scrape a download count information from a japanese website via XPath in google sheet. I want to save the number/text inside this red box
here's the link
https://www.photo-ac.com/main/detail/4465781?title=%E3%82%A2%E3%82%B2%E3%83%8F%E8%9D%B6%E3%81%A8%E3%83%92%E3%83%A3%E3%82%AF%E3%83%8B%E3%83%81%E3%82%BD%E3%82%A6
here's my function
=IMPORTXML("https://www.photo-ac.com/main/detail/4465781?title=アゲハ蝶とヒャクニチソウ", "/html/body/div[17]/div/div/div/div[2]/div[7]/div[1]/div[1]/div/div[3]/div[2]/div[1]//text()")
the function doesn't work? why?
thank you
When I tested your formula, I confirmed that an error of Could not fetch url: occurred. But, fortunately, when Google Apps Script is used, I confirmed that the URL can be requested using UrlFetchApp. So, in this answer, I would like to propose to use Google Apps Script. The sample script is as follows.
Sample script:
Please copy and paste the following script to the script editor of Google Spreadsheet, and save it, and put a formula of =SAMPLE("URL") to a cell. If the function name is not found, please reopen the Google Spreadsheet and test it again. This script is used as the custom function.
function SAMPLE(url) {
const value = UrlFetchApp.fetch(url).getContentText().match(/ダウンロード:.+/);
if (!value) throw new Error("Value was not retrieved.");
return value;
}
Result:
When above script is used, the following result is obtained.
Note:
This sample script is for the current HTML of the URL of https://www.photo-ac.com/main/detail/4465781?title=アゲハ蝶とヒャクニチソウ. And, when the structure of HTML of the URL is changed, above script might not be able to be used. Please be careful this.
References:
Custom Functions in Google Sheets
fetch(url)

Auto upload a csv from google driver folder?

Hope you can help. Im looking to create a script to upload csv into a google sheet from a folder in the drive? is there a way to auto upload this once i have added to the folder?
thanks in advance
You can use Google Apps Script to create a google sheet version of your csv file in your Google Drive. You can use the script below:
function myFunction() {
var file = DriveApp.getFilesByName(your-filename-here).next();
var csvData = Utilities.parseCsv(file.getBlob().getDataAsString());
var sheet = SpreadsheetApp.getActiveSheet();
sheet.getRange(1,1,csvData.length, csvData[0].length).setValues(csvData)
}

Using google query to download parts of a published sheet

This works:
curl 'https://docs.google.com/spreadsheets/d/e/2PACX-1vS3iBtVf4i_won5zAN9NGPqhcd6CcTb-4QHxpisSjCmlgV95B6mFmZvtMaC9GPvD7m8kD-6XLkVAhfc/pub?gid=911257845&single=true&output=csv'
however I want to only pick up rows where count > 300.
The query before encoding would be
select * where F > 300
After encoding
select%20*%20where%20F%3E300
So the url becomes
https://docs.google.com/spreadsheets/d/e/2PACX-1vS3iBtVf4i_won5zAN9NGPqhcd6CcTb-4QHxpisSjCmlgV95B6mFmZvtMaC9GPvD7m8kD-6XLkVAhfc/pub?gid=911257845&output=csv&tq=select%20*%20where%20F%3E300
The line above works retrieves a file, but it returns the whole file, and doesn't filter.
Note that a published web sheet has the form
https://docs.google.com/spreadsheets/d/e/KEY/pub?gid=GID
https://docs.google.com/spreadsheets/d/e/2PACX-1vS3iBtVf4i_won5zAN9NGPqhcd6CcTb-4QHxpisSjCmlgV95B6mFmZvtMaC9GPvD7m8kD-6XLkVAhfc/pub?gid=911257845
This works. Adding &output=csv to it (no space before the &) works, and it downloads as a csv file. This opens in excel and shows the data in the table.
I tried this:
https://docs.google.com/spreadsheets/d/e/2PACX-1vS3iBtVf4i_won5zAN9NGPqhcd6CcTb-4QHxpisSjCmlgV95B6mFmZvtMaC9GPvD7m8kD-6XLkVAhfc/pub?gid=911257845&output=csv&tq=select%20*%20where%20F%3E%20300
and
https://docs.google.com/spreadsheets/d/e/2PACX-1vS3iBtVf4i_won5zAN9NGPqhcd6CcTb-4QHxpisSjCmlgV95B6mFmZvtMaC9GPvD7m8kD-6XLkVAhfc/gviz/tq?gid=911257845&output=csv&tq=select%20*%20where%20F%3E300
and get errors -- resource not available.
The page above should be public for people who want to try.
This may be an issue between publishing a sheet, and sharing a whole spread sheet to anyone who has the link.
I've created a new page that uses importrange() that slurps up the page from the main sheet, and that one is public.
https://docs.google.com/spreadsheets/d/1-lqLuYJyHAKix-T8NR8wV8ZUUbVOJrZTysccid2-ycs/edit?usp=sharing
How about this modification?
Modification points :
When it uses query, please use like https://docs.google.com/spreadsheets/d/### file ID ###/gviz/tq?gid=###&tq=### query ###.
When select%20*%20where%20%F%3E300 is decoded, it is select * where %F>300.
select * where F > 300 is select%20%2a%20where%20F%20%3e%20300.
In order to output CSV, please use tqx=out:csv.
Please share the Spreadsheet.
On Google Drive
On the Spreadsheet file
right-click -> Share -> Advanced -> Click "change" at "Private - Only you can access"
Check "On Anyone with the link"
Click "Save"
At "Link to share", copy URL.
Retrieve file ID from https://docs.google.com/spreadsheets/d/### file ID ###/edit?usp=sharing
Modified curl command :
curl 'https://docs.google.com/spreadsheets/d/### file ID ###/gviz/tq?gid=911257845&tq=select%20%2a%20where%20F%20%3e%20300&tqx=out:csv'
Reference :
Query Language Reference
If I misunderstand your question, I'm sorry.
Edit :
The following 2 URLs are the comparison between your URL and my answer. The URL of my answer was matched to your URL.
1. Your URL
https://docs.google.com/spreadsheets/d/e/2PACX-1vS3iBtVf4i_won5zAN9NGPqhcd6CcTb-4QHxpisSjCmlgV95B6mFmZvtMaC9GPvD7m8kD-6XLkVAhfc/gviz/tq?gid=911257845&output=csv&tq=select%20*%20where%20F%3E300
When above URL is separated,
https://docs.google.com/spreadsheets/d/e/
e/ is not required.
2PACX-1vS3iBtVf4i_won5zAN9NGPqhcd6CcTb-4QHxpisSjCmlgV95B6mFmZvtMaC9GPvD7m8kD-6XLkVAhfc
This is not the file ID of spreadsheet.
/gviz/tq
gid=911257845
output=csv
tq=select%20*%20where%20F%3E300
2. In my answer matched to your URL
https://docs.google.com/spreadsheets/d/### file ID ###/gviz/tq?gid=###&tqx=out:csv&tq=### query ###
When above URL is separated,
https://docs.google.com/spreadsheets/d/
### file ID ###
You can see the detail of the file ID of spreadsheet at here.
/gviz/tq
gid=###
You can use gid=911257845.
tqx=out:csv
This has to be used instead of output=csv.
tq=### query ###
You can use tq=select%20*%20where%20F%3E300.
Note :
Each number corresponds.
And please share the Spreadsheet as follows. This is difference from "Publish to the web" on Spreadsheet.
On Google Drive
On the Spreadsheet file
right-click -> Share -> Advanced -> Click "change" at "Private - Only you can access"
Check "On Anyone with the link"
Click "Save"
At "Link to share", copy URL.
Retrieve file ID from ``https://docs.google.com/spreadsheets/d/###

Resources