I want a user to upload multiple images (+ thumbs) and give a description about their pics.
What do i need to do to create this the ruby way?
Do i manually create the tables (and which are these) or what gem do i require?
I want to store the file physical on a path and store the link (+ attr. information) in the db (if it is the best solution).
I am open to any alternatives to seek my best solution! :-)
Look at paperclip. Other great solution for handling multiple images for an item is paperclippolymorph
I'm not sure if there is a "best" solution, whether or not you store the images in the database is a tradeoff. Storing images on the server's filesystem and keeping the file's path information in the database will keep your DB smaller, but it will also add one more folder/location that you need to keep backed up and can provide security problems (if the image storage folder is not properly secured, it can be easier for an attacker to pull images off of a filesystem than extract them out of a database).
Related
is it recommended for creating each folder for each user to store the image uploaded? or should i just create an image folder and put everything inside?
Note: for web application such as ebay where each user will not have a a lot of image uploaded, but they might be millions of users using it.
There are other considerations to decide whether to create a folder for each user or
just a big image folder containing all images.
What is important? Ease of maintenance? Performance? Any Resources limitation ?
Will you ever need to persist those images?
What are the operations you will normally perform on the images?
How often?
Having a folder will be more organized, but you may need to maintain millions of folders and it will require more resources.
It will be difficult to maintain if the images are not followed any naming convention based on the user id or name if you put them in one single folder.
Before you decide which way to go, you need to look at the overall operations.
I'm new to Rails, I wanted to make a website for uploading files: music, videos, pictures, text. What is a better way to store files? I've read about different methods: Database, as a file, Amazon S3?
There will be a lot of files around 1 kb to 20Mb each.
Thanks!
Storing files in a database is not bad, per se. It depends on the kind of database.
Storing files in a relational database is not view as a good practice by the reasons explained by bassneck.
But there are other kind databases that are specifically designed to store any kind of data, in a non-relational way, for example files of any kind. The answer of Dhruva highlight that, MongoDB is pretty good and its support for storing files using GridFS is awesome.
GridFS is very good, for example it can stream only parts of a file, pretty useful for video.
In your specific case – many small files of many kinds of data – GridFS is an real option. I use Heroku & mongohq.com and they work like a charm.
I don't think there is a right answer for that. I use heroku, so my only option is Amazon S3 (which has free quotas for the first year) and I'm pretty satisfied with it. I use carrierwave gem for uploading files and it's really easy to use. I really prefer it over Paperclip.
If your hosting provides a lot of space and bandwidth then you could give it a try.
But for the database, I really don't like the idea of storing files in it.
Updated
Reading a filename from the table should be faster than reading the file itself. The bigger the DB, the longer it will take to make a backup. And if you take heroku for example, you only get 20gb for a shared db. That's not much if you're gonna store 10-20mb files in there. Moving project to another hoster might be easier if you store files in the DB. But if you use an external service (such as S3), there would be no difference for you.
Amazon S3 integration on Heroku is relatively easy to setup and get working with the paperclip gem. Heroku has some documentation on how to get this up and running.
Take a look at their documentation and see if this is the kind of thing your looking for.
http://devcenter.heroku.com/articles/s3
I will suggest you to also check out & consider MongoDB GridFS - http://www.mongodb.org/display/DOCS/GridFS+Specification. My experience with GridFS has been good. I cannot give you a comparative analysis with the other options, however I would like to know the same.
I would recommend you use amazon s3 to store your files. It is the best.
I'm creating a filesystem and I think I'll be storing files in a DB (http://sietch.net/ViewNewsItem.aspx?NewsItemID=124 and http://blog.druva.com/2009/01/25/file-systems-vs-databases/ seem to indicate it's a good idea).
Since it's a filesystem, I'll need A LOT of I/O and REALLY fast. If I'm hosting on EC2, will Amazon SimpleDB be a decent solution for this?
SimpleDB has a maximum record size of a 1,000 BYTES so it is VERY poorly suited to storing files/blobs (unless they are tiny).
It is fairly common for people to use SimpleDB to index files and then to store the files in S3 which is much better suited for storing large objects.
SimpleDB (and recent DynamoDB) aren't suited to store files. Why don't you just make the versioning control on one of them, indexing files stored on S3?
You don't need to override the files on S3, as you can name them whatever you want and get the original name (and other info) from the database. You can even, for text files, for example, have a preview or the begining of the file on the DB or, for images, have a thumbnail on S3 and also get this info from the DB, so when users list the files they get only the thumbnail and, if they want, they download the full-size file.
Take a look at http://aws.amazon.com/en/dynamodb/faqs/#When_should_I_use_Amazon_DynamoDB_vs_Amazon_S3 and http://aws.amazon.com/en/running_databases/
I am developing a web application to accept a bunch of text and attachments (1 or more) via email, web and other methods.
I am planning to build a single interface, mostly a web service to accept this content.
What design considerations should I make?
I am building the app using ASP.NET MVC 2.
Should the attachments be saved to disk or in the database?
Should the unified single interface be a web service?
Pros and cons to using web services to upload files
as with any acceptance of files i'd be checking them for viruses or the like. i'm very nervous about files transmitted from the internet.
i always like putting my files in a database because it's neater i find. i hate having files over the network with folders needing rights etc. i know there are people that prefer it the other way so i guess my answer is also depends on personal preference.
i like the db approach because i can more easily tie files to records and do searches. if you have a file system then you still need to store info about the file plus the extra work of storing it.
then if you need to move files around you also need to possibly modify references in the database.
then again, you need to allocate enough space to grow the database and then cater for multiple databases perhaps as storage runs out.
so i guess if you're downloading large files then yeah maybe i can see the point of a file system as it's easier to grow it. if you have small text files then maybe a database will work.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
Using Rails, is there a reason why I should store attachments (could be a file of any time), in the filesystem instead of in the database? The database seems simpler to me, no need to worry about filesystem paths, structure, etc., you just look in your blob field. But most people seem to use the filesystem that it leaves me guessing that there must be some benefits to doing so that I'm not getting, or some disadvantages to using the database for such storage. (In this case, I'm using postgres).
This is a pretty standard design question, and there isn't really a "one true answer".
The rule of thumb I typically follow is "data goes in databases, files go in files".
Some of the considerations to keep in mind:
If a file is stored in the database, how are you going to serve it out via http? Remember, you need to set the content type, filename, etc. If it's a file on the filesystem, the web server takes care of all that stuff for you. Very quickly and efficiently (perhaps even in kernel space), no interpreted code needed.
Files are typically big. Big databases are certainly viable, but they are slow and inconvenient to back up etc. Why make your database huge when you don't have to?
Much like 2., it's really easy to copy files to multiple machines. Say you're running a cluster, you can just periodically rsync the filesystem from your master machine to your slaves and use standard static http serving. Obviously databases can be clustered as well, it's just not necessarily as intuitive.
On the flip side of 3, if you're already clustering your database, then having to deal with clustered files in addition is administrative complexity. This would be a reason to consider storing files in the DB, I'd say.
Blob data in databases is typically opaque. You can't filter it, sort by it, or group by it. That lessens the value of storing it in the database.
On the flip side, databases understand concurrency. You can use your standard model of transaction isolation to ensure that two clients don't try to edit the same file at the same time. This might be nice. Not to say you couldn't use lockfiles, but now you've got two things to understand instead of one.
Accessibility. Files in a filesystem can be opened with regular tools. Vi, Photoshop, Word, whatever you need. This can be convenient. How are you gonna open that word document out of a blob field?
Permissions. Filesystems have permissions, and they can be a pain in the rear. Conversely, they might be useful to your application. Permissions will really bite you if you're taking advantage of 7, because it's almost guaranteed that your web server runs with different permissions than your applications.
Cacheing (from sarah mei below). This plays into the http question above on the client side (are you going to remember to set lifetimes correctly?). On the server side files on a filesystem are a very well-understood and optimized access pattern. Large blob fields may or may not be optimized well by your database, and you're almost guaranteed to have an additional network trip from the database to the web server as well.
In short, people tend to use filesystems for files because they support file-like idioms the best. There's no reason you have to do it though, and filesystems are becoming more and more like databases so it wouldn't surprise me at all to see a complete convergence eventually.
There's some good advice about using the filesystem for files, but here's something else to think about. If you are storing sensitive or secure files/attachments, using the DB really is the only way to go. I have built apps where the data can't be put out on a file. It has to be put into the DB for security reasons. You can't leave it in a file system for a user on the server/machine to look at or take with them without proper securty. Using a high-class DB like Oracle, you can lock that data down very tightly and ensure that only appropriate users have access to that data.
But the other points made are very valid. If you're simply doing things like avatar images or non-sensitive info, the filesystem is generally faster and more convenient for most plugin systems.
The DB is pretty easy to setup for sending files back; it's a little bit more work, but just a few minutes if you know what you're doing. So yes, the filesystem is the better way to go overall, IMO, but the DB is the only viable choice when security or sensitive data is a major concern.
I don't see what the problem with blobstores is. You can always reconstruct a file system store from it, e.g. by caching the stuff to the local web server while the system is being used.
But the authoritative store should always be the database. Which means you can deploy your application by tossing in the database and exporting the code from source control. Done.
And adding a web server is no issue at all.
Erik's answer is great. I will also add that if you want to do any caching, it's much easier and more straightforward to cache static files than to cache database contents.
If you use a plugin such as Paperclip, you don't have to worry about anything either. There's this thing called the filesystem, which is where files should go. Just because it is a bit harder doesn't mean you should put your files in the wrong place. And with paperclip (or other similar plugins) it isn't hard. So, gogo filesystem!
Unable to find an up-to-date answer to this question I have implemented an
database service for Active Storage available since Rails 5.2 that works just like any other Active Storage service, but stores file content in a special database column instead of a cloud service.
The implementation is based on a standard Rails Active Storage service, adding a migration with a new model: an extra table that stores blob contents in a binary field. The service creates and destroys records in this table as requested by Active Storage.
Therefore, this service, once installed, can be consumed via a standard Rails Active Storage API.
https://github.com/TitovDigital/activestorage-database-service
Please be aware of all pros and cons of using a database for storing files.
With the right database it will provide full ACID support and can wrap file storage and deletion into transactions. It is also much easier in DevOps as there is one less service to configure.
Large files or large traffic are the risky cases. Either will put an unnecessary strain on the app and database servers.