Exploring around S3's UI, it seems they only enjoy file uploads from my local box. Is there a way to push the files directly from my production server to an s3 account?
As you've tagged your question ruby-on-rails I will suggest the AWS::S3 gem. I use it myself to copy daily MySQL database dumps from my production server to S3 (blog entry with the details if you're interested).
Amazon doesn't offer a pull service for FTP to S3, and I haven't seen anyone else advertising one either. (It's not a terrible business idea though.)
That having been said, there are a ton of tools to help with this!
A lot of people use something like s3fs with their favorite backup utility (ie, cron + rsync).
Lately, I've had great success with boto and some custom Python scripts. The reason I like this is because it integrates nicely with whatever other services you're running on the box, and gives status updates.
Hope that helps!
Best,
Zach
I had done massives upload with the s3fox add of firefox.
Related
I would like to ask you a question about deploying a rails application onto a physical server.
Briefly, our dinosaur client doesn't want to store anything on the cloud hence he wants us to deploy everything onto his servers instead of heroku.
What should be the most efficient way to solve this problem? Won't he be able to reach the source code of our application?
That would be great if you can help me,
Thanks!
You might think on the variant when you keep only production database (and assets if client want) on the client's server and deploy your code to Heroku or whatever else.
I'm making a small photo-gallery app. The photos will be hosted locally. Right now I use dreamhost but their rails implementation looks horrible. So I'm looking for other options. I know Heroku gives you one 'web dyno' for free, but they don't say anything about how much space you get. As I said, I want my photos stored locally with the app, I don't want to deal with s3 or other cloud storage.
there is no local storage with heroku - only temporary space. you'll need to use S3 or some off-site storage with heroku.
(and I agree, rails on DH is awful, even if you enable passenger)
Use Openshift to deploy your app
checkout this deploying rails app in openshift
openshift provides one permanent data directory to store data and its free
If you are interested in VPS, Digital Ocean - https://www.digitalocean.com/, provides excellent hosting starting from $5. And you can store your photos on the local disc.
There are very good tutorials on their site to get you started with.
Check out Shelly Cloud: https://shellycloud.com/ You get persistent storage (so you don't lose data in case of disk failure) and the deployment is optimized for Rails applications.
With Heroku you can host images that you store in the repository of your project. Just try this:
rails new mytest
create a simple page and link to a test image in your /app/assets/images
heroku create mytest123 # <-- mytest1234 must be your unique app name
and now push the repo to Heroku:
git push origin master # origin points automatically to Heroku after you created this
This is the easiest way to host small projects for free. Sometimes the Dyno takes some time to startup, and you need to point the domain to the right proxy, but these issues can be dealt with later.
S3 comes into play when you deal with uploaded content / images. For this use case, you need S3 which is out of the scope of your question too.
I can't figure out where I can go create a domain for my simpleDB.
I've gone to AWS Management Console, I was able to create a bucket for S3, but I don't have a simple DB tab.
Am I missing something here?
SimpleDB is not yet supported by AWS console. They may eventually add it, but for now you have to use direct service calls (with SDK it is the easiest).
SDB Tool is a great FOSS Firefox plugin:
http://code.google.com/p/sdbtool/
You can use SDB Explorer. SDB Explorer has been made as an industry leading graphical user interface (GUI) to explore Amazon SimpleDB service thoroughly and in a very efficient and user friendly way.
http://www.sdbexplorer.com/
You can try the Javascript Scratchpad for Amazon SimpleDB
Or if you're a Python lover, try out boto library, designed to access AWS resources. Find more information here: http://boto.readthedocs.org/en/latest/simpledb_tut.html
I have a Rails application that I want to add file upload to, so that the users have access to a "resources" section where they can upload and share (although not publicly) any type of file. I know I could build a solution using paperclip and S3 for example, but to try and avoid the admin overhead of all that I'm looking at API interfaces to drop.io and box.net. Does anyone have any experience of these? I've got a basic demo working rather well to drop.io, but I was just wondering if anyone had any better ideas or experiences.
Many thanks
D
I use attachment_fu with S3 backend. For User Interface goodness, I use YUI's file uploader.
Some of the files are uploaded with world read access, others with no public read access.
I use Attachement_fu to create self-signed urls to enable clients to access the private S3 files.
I did write some small helper routines for the S3 library for re-connecting after a timeout, handling various errors that the S3 library can raise, etc.
Building your own library for drop.io and/or box.net
Your idea of using the API for a commercial service is interesting but I haven't run into any problems with the above config. And the price for direct S3 access is very low.
If you do decide to go this route, you may want to open source your code. You'd benefit by getting testing, ideas, and possible code contributions from the community.
Note that if you have a lot of uploads, you can end up with a performance issue if the uploads are synchronous with the Rails thread--the rails process is busy uploading and can't do anything else until the upload is done.
HTH,
Larry
Greetings,
has anyone of you used the paperclip plugin on Engine Yards hosting service? Are there any problems to solve or anything to pay special attention to?
Is the paperclip on heroku example of Pedro Belo fitting?
Thanks for your comments,
Joe
Heroku and Engine Yard uses Amazon EC2. That's where the similarities stop, though.
Heroku has a very special setup where you don't have write access to the file system. They do that in order to be able to scale easily with their setup, prompting users to use HTTP caching (varnish), S3 and the likes, instead of serving dynamic content via the file system.
Engine Yard has a pretty standard virtual host-ish setup, where you have write access to the file system.
So, the answer to your question is: no, there are no problems or special cases you have to worry about.
Using it now on a production site, and I've had no problems thus far.