I downloaded the Flicker8k_Dataset (a folder with lots of pictures). I am looking for a way to load that into Google Colab without using Google Drive (it would take 5 hours to upload to Google Drive). What would be the best way to do this?
You can use wget command to download the dataset into your google drive and use it from there. The other way to use colab on your local dataset would be to run the colab on a local machine.
Related
I’m learning opencv for object detection and trying to do all the coding using google colab. And so far it has been working well: I can mount the gdrive and upload file and read the video file just fine.
The selecting ROI by passing video frame into cv2.selectROI() crashed the Colab. The runtime log complained about a plugin xcb not initialising. Further reading reveals the plug in require to be run on local machine, so I wonder if there is a way to run cv2.selectROI or equivalent on colab?
I would like to load my own data set with images in order to test my CNN algorithm. I've got two folders, one for training and another one for validation. I have loaded that folders into the Google Drive. I would like to ask how to download folders from Drive into the Colab ? Or maybe there are another better ways to do that ? I look forward to hearing from You soon. Best wishes!
Since you already have the folder in drive , go to google colab and run the following:
#Mounting your Google Drive in the runtime's virtual machine
from google.colab import drive
drive.mount('/content/drive/')
On running the above code you will find a URL from where you can obtain the authorisation code. Paste the code in the space provided and click enter
If you are able to access Google Drive, your google drive files should be all under:
You can then copy the path and move to the directory using :
%cd /content/drive/MyDrive/foldername
I am working on a class project where we're working with a large dataset in Google Colabs. Notably, the file is not saved if the runtime disconnects. I thus downloaded the dataset to my personal drive, but am struggling to get it accessible to the Colabs project. I also can't just mount my drive because there are other people working on this.
Is there some way to download a link-sharing=on drive file in Colabs?
A way to save the trained model would also be useful, but the same issue of sharing it and the saved files still applies.
There are 2 ways that I know.
If you only want to share with your friends, not making it public, you can share via google drive. Each friend will need to auth.authenticate_user() and then use pydrive to load the file given its FILE_ID.
If you can make it public, it's even easier. Anyone can download the file with
!gdown --id xxxxxxxxx
Where xxxxxxx is the FILE_ID.
Is it possible to read/write data on local without using DirectPipelineRunner?
Suppose I create a dataflow template on cloud and I want it to read some local data. Is this possible?
Thanks..
You will want to stage your input files to Google Cloud Storage first and read from there. Your code will look something like this:
p.apply(TextIO.read().from(gs://bucket/folder)
where gs://bucket/folder is the path to your folder in GCS, and assuming you are using the latest Beam release (2.0.0). Afterwards, you can download the output from GCS to your local computer.
I am dealing with mobile and desktop applications. I am trying to use Google Drive as a permanent storage folder. Unfortunately I am at a loss because I cannot find any examples. Can someone point me at an example of storing or retrieving a file from Google Drive?
I would really like to see a simple example in code or maybe a library which can handle this. I am stuck at Retrieve and Use OAuth 2.0 Credentials https://developers.google.com/drive/credentials
Did you look at the sample app on the drive API page?
edit
google moved the examples to github
I don't know if I figured out your problem. What I have understood is that you are looking for a way that makes you able to work on the same project from different computers.
If so, you simply have to download Drive. Once install it will ask you what folder you want to keep syncronized with Drive. If you select your workspace, it will be syncronized with Drive. So, on another pc, if you have installed Drive, you will be able to use the workspace folder too, with the same files.
I hope this is helpful.
Marco
You should look into Documents List API.
The Google Drive API & SDK are meant to be a mean to integrate applications into the Web-UI of Google Drive. To protect the security of the user's files, it puts some restrictions to the application developer, like:
The app has to be installed from the Chrome Web Store
The app can only access files created from the app itself,
or files selected by the user using the Google Picker UI.
All these restrictions make the Google Drive API not a good choice for most non-web applications.
E.g. installing the app from the Chrome Web Store is probably something you would find cumbersome to request from your users for a mobile or desktop application. Nonetheless, without being installed from the Chrome Web Store your application will always receive an error when accessing a file from Google Drive.
The Documents List API allows you to store and load files (not only Google Documents!) into Google Drive.
This question is a little bit old, still. You can look at this example: https://code.google.com/p/google-api-dotnet-client/wiki/OAuth2
They do the OAuth2 for accessing task. But I just used the same code to access the files using the v2 API. http://code.google.com/p/google-api-dotnet-client/wiki/APIs#Drive_API
I only retrieved the file list, but it is working (from .NET, desktop app).