I'm using Talend for several ETL-jobs. Main focus on inserting and updating data defined in local Excel files to Salesforce. Excel and Salesforce are used as Input- and Output-Connections massively.
This works like a charm!
Now instead of Excel I need to use Google Spreadsheets as data inputs/outputs. Just manually download as Excel isn't good enough. Reason: I need a highspeed repeatable process executed thousands of times bi-directional. Uploading/Downloading XLS/CSV is not an option.
As an unfortunate, my researches concluded that there is NO Google spreadsheet connector available. I found this legacy-project which seems to be abandoned: https://code.google.com/p/google-talend-components/ - it is outdated from Talend-side as well as from Google (old API versions).
One other thing I've seen, is that Talend comes with Google Big Data Support: https://www.youtube.com/watch?v=BK6G3sTmwVE - Also Google Big Data might be somehow connectable to Google Spreadsheet - but I wasn't able to figure out how. Is this easy or hard? How to start? What to read?
I need to know if there is any viable approach to read and write data with Talend from and to Google-Spreadsheet.
Alternatively is there an other ETL-tool like Talend which has connectors to Google Spreadsheet AND Salesforce out-of-the-box?
Is there a direct google docs connector in Talend, no. But you can do what you are looking to do if you think of the problem a little differently. It sounds like you need to read and write to spreadsheets then upload/download from google. Lets take each problem separately.
Upload/Download
Google storage. If you need to upload and download files from google Talend has excellent integration with their API. As you mentioned, Talend --> Big Data has components to interact with Google Storage. This type of storage is primary meant as back end storage for an application.
In Talend under Cloud-->Google Drive there are components to upload and download from Google Drive. Google Drive is primarily meant as Chrome app and has direct integration with Google Docs.
Writing to Google Docs
Regarding your need to interact with google docs, you do not need any special components for this. Google docs can utilize csv and Excel formats. If you create and modify your documents in Talend using the standard file interaction components ( delimited, Excel, etc.) you can upload and download these documents using one of the two methods I described above (Google Storage or Drive) depending on what type of storage you are using.
Related
I have an application that runs benchmarks and generates a CSV file with the report contents.
To interpret this report, I have created a template GSheets document with queries, drop down selections, graphs, etc.
Consumers must:
Run the benchmarking tool to generate the data CSV
Go to the template document in gsheets (outlined in the readme)
Copy the template into their own gdrive account
Insert the contents of the CSV into the unprotected data sheet of that document
This is a lot of manual handling and I would rather be able to, like a csv, programmatically produce a file that contains the filled out sheet where the consumer just needs to open the sheet with the Google Sheets account.
From what I can see, Google Sheets doesn't have a specific file format that it uses and the gsheet files are just empty magic links that the web app knows to respond to.
I can imagine that I can use the Google Sheets web API to publish a new file to the user's Google Drive but I don't want to mess around with Google OAuth authentication as I feel that's overkill.
Is it possible to write a file that Google Sheets can interpret directly?
Perhaps I can write an older MS Office .xls file and Google Sheets can interpret it via the import functionality?
My backup is writing the report to an html file with a web app that visualizes the data using some charting JS library - though it would be a shame because everything I need is already in Google Sheets.
Google sheets have an API: gspread.
If it's only a matter of switching data, you could create main spreadsheet with graphs etc. set up, and then using an API copy it and paste new data into it.
I'd like to read data from a Google Drive Spreadsheet in Dart code (running as a console app in the VM, not in the browser). I'm somewhat confused by the names of the API; Sheets is part of Google Drive, but all the Google Drive API docs I can find relate to integrating the UI of Drive into your app.
There's also this question on SO, but it's over 2 years so I'm hoping that it's no longer correct, and there is an API.
I found this Dart Google API library, but it says:
THIS LIBRARY IS DEPRECATED
Please use the Google API Dart libraries in http://pub.dartlang.org, the Dart package manager
Search for 'google' or the API of your choice. For example:
http://pub.dartlang.org/packages/google_drive_v2_api
However I'm struggling to find which package I'm supposed to use. The Drive API one doesn't have a lot of info, and there doesn't appear to be one with "Sheets" or "Docs" in the name.
You need to use the Spreadsheet API https://developers.google.com/google-apps/spreadsheets/.
There isn't a Dart lib for it, but it's a well formed REST API so very easy to program against.
Sheets is not exactly part of Google Drive, it is part of the Google Docs productivity suite that is stored in Google Drive. Google Drive is the general cloud storage layer. The Drive API is for interacting with files of all types that are stored in Drive. You can interact with Sheets files using that API to modify metadata or import or export to/from other formats like csvs. That is possible to do from Dart (I'm not sure if there is a Dart-specific client library, but this API is just rest calls so its always possible.)
If you want to interact with a Sheet at a lower level to do things like modify a single cell, you can use AppScript, however that cannot be done from Dart.
Usually for Google APIs support generators that create access libraries for every language.
For example https://pub.dartlang.org/packages/google_drive_v2_api.
You can generate them yourself (also for your own endpoints) using https://pub.dartlang.org/packages/discovery_api_client_generator but for most APIs there are packages already in pub.dartlang.org.
So, a quick background. I make productivity apps (specifically CRM and Project Management). And I love the docs, spreadsheet and presentation products made by Google. Not surprisingly, my products have done a lot of "things" with Google Docs for a long time:
Create "native" (ie. Docs/Spreadsheets/Presentations) documents
Use native documents as templates
Link and modify permissions of any file in Docs/Drive
Upload any arbitrary file
etc.
What I'm confused about is what does Google want me to do on the labels on the buttons in my app. Right now, they all say "Google Docs". You're linking any arbitrary file to a presentation, you're linking it from "Google Docs". You're exporting a spreadsheet of time sheet entries, you're exporting it to "Google Docs". You upload a PDF, you uploaded it to Google Docs. Etc.
What I'm confused about is that, and correct me if I'm wrong, but I don't think it is a complete switch over to "Drive." I still see labels on the Google site for Google Docs. So, this is what I think the breakdown is:
If it is a Google "native" file, then it is Docs, else it is Drive. Thus, if your uploading any arbitrary file, that button should refer to drive. But if you are exporting a spreadsheet of data to the Google Spreadsheets format, then that is Docs.
Is this right at all? Does Google have some information somewhere?
Disclaimer: personal opinion
I would use Drive everywhere, except when specifically talking about the collaborative word processor provided in Google Drive, that is the Google Doc.
I would also make sure that all my integrations use the new Google Drive API.
There is reasonably good guidance here: https://developers.google.com/drive/branding
Google Docs and Google Drive are two seperate products from Google. They can work together, but they are still their own individual products and should be called their respective names when being used
I'm designing a new app for iPad for a small company. This app will use Core Data to store a local database and a database of products and prices. The last database needs to be always up to date since the prices can change.
This company, has a Excel file to keep this database of prices. So they don't have a SQL db to which I can interact from my app.
One option could be to export the Excel file in CVS and put that file in their server (accessible by internet). Then my app should parse the file.
I don't like very much this idea, though.
Do you have any suggestion?
I had a very similar problem to you. I recently got into a project where the client wanted to import information from an Excel file into an app. I know a lot of people say, just transform it into a CSV and parse it that way, but I really didn't want the client to go through yet another step and introduce a different file format - as simple as that may be.
I also really don't like having the information in the cloud, especially Google. Privacy is something that's important to most companies and I'd doubt they'd approve of you using Google to parse the info.
In order to parse the file, I created QZXLSReader. It's a drag-and-drop solution so it's a lot easier to use. I don't think it's as feature complete, but it worked for me.
It's basically a library that can open XLS files and parse them into Obj-C classes. Once you have the classes, it's very easy to send them to Core Data or a dictionary or what have you.
I hope it helps!
Here are a couple of options for you:
Use Google Doc as the intermediary. When the pricing Excel is updated by someone, simply upload the updated Excel to Google Docs. From your iPad app, you can read the latest data via the Goole API. If the company is up for it, they can move to Google Doc altogether and just modify the online Google Spreadsheet directly.
Use services like StackMob as the intermediary. You will have to write a tool to sync the pricing Excel with SrackMob but you can easily access the data via StackMob's iOS SDK.
I want to use Google spreadsheets to store data online so multiple people can enter and maintain data, then publish or export (csv, xls, ods) the sheet for their application. What is the easiest way to process the sheet? If the data can be accessed as a link the updates could be immediately reflected in the client's app.
This article Data Scraping Wikipedia with Google Spreadsheets discusses using Google spreadsheets as an application platform.
It already has features that allow you to share the spreadsheet with multiple editors, as well as the ability to define forms that you can invite people to fill out who's results will be entered into the spreadsheet as a row.
Just in case its still vaguely of interest 5 months later, there is a SQL-like wrapper language for Google Spreadsheets at:
http://gqlx.twyst.co.za
Its very much an alpha release - but maybe its of some use or perhaps the code could give you some ideas.
I think if I am clear on your requirements, you want to store your data online in a Google Spreadsheet where multiple people can access that right? Then you want an app to use that data? Correct me if I am wrong.
It is possible to have Google spreadsheet as a database and the client app can directly access that data through APIs - spreadsheet APIs . If you want a simple link where people can go and update - you can also have a web app using google apps scripting. very simple to use and would solve your purpose too , why to export your data to ods, xls, csv, make your app directly over Google Spreadsheet. Would be maintainable too.