I am creating an application which uploads file to gmail account avilable space
can any one please tell me the best way to do it ?
i read somewhere about using IMAP protocol is the best way to upload files or is there any other good way around ?
regards
GMail is not suitable for keeping generic files. Google offers Documents and Data API mechanisms for storing files. You might want to look at them. BTW we have products, which combined let you create a virtual drive with Google backend storage in a couple of hours. Callback File System offers a virtual drive, and CloudBlackbox lets you store data on Amazon S3, MS Azure and on GMail storages.
Related
I would like to use some publicly available data from a government website as a source of data in an iOS app. But I am not sure what is the best / most polite / scalable way have a large number of users request data from this website with the least impact on their servers and best reliability for me.
It is 1-50kb of static XML with a fixed URL scheme
It updates with a new XML once a day
New users would need to download past data
It has a Last-Modified header but no caching headers
It does not use compression or a CDN
It's a government website, so if someone even replies to my email I doubt they are going to change how they host it for me...
I'm thinking I could run a script on a server to download this file once a day and re-host for my app however my heart desires. But I don't currently run a server which I could use for this and it seems like a lot just for this. My knowledge of web development is not great, so am I perhaps missing something obvious and I just don't know what search terms I should be using to find the answer.
Can I point a CDN at this static data somehow and use that?
Is there something in CloudKit I could use?
Should I run a script on AWS somehow to do the rehosting without needing a full server?
Should I just not worry about it and access the data directly??
You can use the AWS S3 service (Simple Storage Service).
The flow is somewhat like this:
If the file doesn't exist on S3 yet, or, if the creation date of the file on S3 is yesterday, the iOS app downloads the XML from the gov site and stores it in S3.
If the file exists on S3 and is up to date, download it from S3.
After that, the data can be presented by the app without overloading to the site.
I think the best way for you is to create an intermediary database where you can store your data in a secure manner.
Create a pipeline that does some data transformation and store in you newly created database.
Create an api with pagination and you desired filters
Also make sure you are not violating any data policies in the process.
I hope this helps.
I'm really struggling here as I'm super new to Dot Net Core as well as Google Cloud Storage. I have looked over a lot of the available documentation online but I still can't understand on how to build the architecture.
So what I'm trying to build is a dot net core MVC application that has a form to upload a video file to Google Cloud storage (Google bucket probably?). The controller will take the data from the form and the Model layer is Google Storage.
Some pointers will be really helpful on how can I proceed about this task. Also some links to tutorials or any documentation if you guys think would be useful. Thanks a lot!!
It sounds like you're trying to get end users to upload files into Google Cloud Storage from their web browser. The trick here is that allowing any random anonymous user write access to your GCS bucket is a bad idea, but you also don't want to require that your users have Google Cloud accounts, either.
To resolve this, Google Cloud Storage offers a feature called "signed URLs." Your server uses its credentials to create a URL that is valid for a limited amount of time and, when presented to GCS by the end user, allows it to do a very specific thing as if it is your application's service account (in this case, uploading an object).
The flow goes like this:
Your app signs a URL for uploading an object to GCS and serves it as part of the page to the user.
The user does an upload to GCS using whatever JavaScript libraries you prefer.
If you want the user to use a literal POST web form, the signature is a little different than other cases. Look at the "policy document" section here: https://cloud.google.com/storage/docs/xml-api/post-object#usage_and_examples
Here's a sample that help answer half your question. It demonstrates how to upload a file to Google Cloud Storage:
https://github.com/GoogleCloudPlatform/dotnet-docs-samples/blob/master/storage/api/Storage/Program.cs#L117
Im struggling to find an answer to this. I have a website that is deployed in a shared hosting environment. I want to allow people to upload files to my azure blob storage account.
I have this working locally, using the storage emulator, however when I publish the site I get a Security Exception.
Is this actually possible under a shared hosting envrionment ?
Cheers
A bit more detail would help, in understanding how these uploads are taking place. That said, I'll make the assumption that people are uploading directly to Blob Storage, and not through your Website (or Web Service).
To allow direct uploads, you need to provide either a public blob or container (which everyone in the world can see), or create a temporary Shared Access Signature (SAS) on a specific blob or container, that grants access for a short time window.
If your app is Silverlight, then you are probably running into a cross-domain issue (and you'll need to correct that with an access policy).
If you provide more details around the way uploads are being sent, as well as the client and server technology, I can edit my answer to be more specific.
just recently I was thinking and wondered, how does Dropbox upload my files to its S3 storage and how might that one be organized?
Let's just completely forget about the sync aspect for a second and scale the problem down to one S3 bucket.
Say, in that bucket's root directory you have lots of folders, each belonging to an arbitrary user.
Now if that user wants to upload a file to his folder... how does that happen internally? I mean, Dropbox can't just store the Amazon S3 access credentials/keys hard-coded into the application (be it on ios or windows) as it might get reverse-engineered and thus exposed.
Any thoughts on this?
Thanks!
Some guys from EADS did reengineering on Dropbox, the presentation slides are available for download: A CRITICAL ANALYSIS OF
DROPBOX SOFTWARE SECURITY
In the same way websites don't allow users to directly access their databases but rather provide interfaces that can control permissions and handle authentication, I'm sure Dropbox has some kind of application that the client on your computer interacts with. Their server daemon will have permissions to write to the disk, but your computer has to go through it (and it's security procedures) before anything your computer sends is written.
I have a Rails application that I want to add file upload to, so that the users have access to a "resources" section where they can upload and share (although not publicly) any type of file. I know I could build a solution using paperclip and S3 for example, but to try and avoid the admin overhead of all that I'm looking at API interfaces to drop.io and box.net. Does anyone have any experience of these? I've got a basic demo working rather well to drop.io, but I was just wondering if anyone had any better ideas or experiences.
Many thanks
D
I use attachment_fu with S3 backend. For User Interface goodness, I use YUI's file uploader.
Some of the files are uploaded with world read access, others with no public read access.
I use Attachement_fu to create self-signed urls to enable clients to access the private S3 files.
I did write some small helper routines for the S3 library for re-connecting after a timeout, handling various errors that the S3 library can raise, etc.
Building your own library for drop.io and/or box.net
Your idea of using the API for a commercial service is interesting but I haven't run into any problems with the above config. And the price for direct S3 access is very low.
If you do decide to go this route, you may want to open source your code. You'd benefit by getting testing, ideas, and possible code contributions from the community.
Note that if you have a lot of uploads, you can end up with a performance issue if the uploads are synchronous with the Rails thread--the rails process is busy uploading and can't do anything else until the upload is done.
HTH,
Larry