I'm making a Rails App, which I need to be online, but probably 1 or 2 people will use it at most. The app will organize photos + description uploaded by this user. The photos are aprox. 2 GB. I don't want to spend a lot of money on this, so my question is what is the best solution here? I hear Heroku is free but I can't upload photos in my App on Heroku and S3 costs money. Any advice would be appreciated.
host
You can go with heroku, free plan should be enough for 2-5 person.
image storage
As you mentioned about S3 is not free or cheap, best solution is alternate of S3 is Cloudinary. It will allow you to upload around 5-7GB of data including bandwidth, and super fast.
Cloudinary Link
Alternates: Google Drive, Dropbox, Pcloud and etc... These all cloud storages maybe little slow in uploading or fetching but still free solution...
Related
Background - My app is related to Chef where people will post pics and videos. These numbers would be huge.
Tech stack - RoR, nginx, Postgresql, AWS.
My question :
Since there would be lot of files, what would be the suggestion. Should I use s3 to store the images. Have CDN implemented?
I would be tagging the files so that they would be searched on those tags. What is the best way to store the tags?
Load Balancing?
I was looking at how FB is storing the images and videos. Not sure if I have to do it because mine is a small app.
P.S - I have already done the MVP with basic s3 setup but there are performance issues.
I was wondering to use another DB since images and videos are unstructured. (Correct me if I am wrong)
I am posting this because I have an actual working problem.
This is a vast question, and answers may vary depending on different opinions and experiences. But On the lines that you are thinking, AWS S3 is a good option to manage files, especially media. You can also search through s3, and associate meta information with files which you might want to call tags. To Scale it more for reads, you can also setup elastic search or opensearch server to index the image distributions in s3. The best way is indeed indexing, whatever tools or platforms you use. But Using indexing cant be achieved always, due to project constraints including costs. Infact you might not even want CDN because S3 is itelf geographically distributed in wider regions and you can leverage that.
Edit---------
Yes, you index the stored images information. You can set up elastic search server whose index will always be updated. If you are not keen on investing in elasticsearch, you can setup a simple database of your choice without worrying future's future's issues. May be you can setup NoSQL (DynamoDB AWS). On every store in s3, create a trigger to update the DynamoDB with its information storing tags and other meta information. You can retrieve that information and display image from s3 URI stored in that DynamoDB map.
My team and I are building an iOS application. We allow technicians in the field to upload images for certain issues they are resolving on technical equipment. It will be important to zoom in (so keep quality relatively high) when these images are uploaded to S3.
Recently we decided to add thumbnails because it will be much faster when others browse the iOS app, rather than downloading a 1.5-2.5mb image.
My co-worker decided the best way to handle this is to generate a 200-500kb thumbnail in iOS then upload the image and the thumbnail to s3.
I voiced my concern that some of our technicians may be in some parts of the world where internet is slow and data usage is limited. So doing all this additional work on the device and uploading makes no sense to me. However the team considers this a good solution and will move forward. I've shown them easy examples of how to generate thumbnails from S3 and Lambda automatically on the server... allowing us to either upload higher fidelity images with the additional bandwith or just increase the speed of the app by uploading much less. Sometimes a user may upload as many as 100 images... meaning an additional 20-50mb...
Anyways I wanted to hear some answers about how you guys think the best way to handle this is, mainly for my own sanity check.
I don't completely comprehend the intricacies of your project, but from experience, I have one word for you - Cloudinary. As opposed to S3, which a general purpose Cloud storage solution, cloudinary is designed to handle images.
We have a 200,000 hits a day online classified app that handles tens of thousands of photos daily. And cloudinary provides an extremely mean solution for all our needs. We have uploads by users from their mobile and desktop devices, bookmarking of those images, CDN based serving, and thumbnail generation.
Did I mention they have thumbnail generation built in? They have lots of other features as well, including
Resize and Crop
Optimized JPEG Custom Crop
Face Thumbnail
Rotated Circular Thumbnail
Zoom Effects and Zoom Image Overlay
Watermark Image
Optimized WebP
Overlay, Border, Shadow Text Overlay, Border, Shadow etc.
The admin console is pretty kickass too, with all of the above features available for you to configure over the cloud. And it fits well with pretty much any application (we use it in our internal Ruby, Go, NodeJS services, our Web Application and our iOS and Android apps as well).
I'm not paid to sell Cloudinary to you, but I can vouch that if it is image based services I needed, I would go for Cloudinary any day over S3. Major players like EBay and TED etc. use it for their image requirements.
I have zips in s3 with videos.. some are small (3Mb), but others can be bigger (like 30Mb).. I need to download these zips.. I don't know what is the best way to do it, talking about performance.. Put these videos in a unique zip and download just one zip with all the videos in it or download every file separately.. Joining and compacting the files to a zip in a temp folder isn't expensive in performance or much work for the server?
PS.: The user will choose the videos that he wants to be downloaded and click on a button to download them (I think it would be like 10-100 videos)..
From a strictly performance standpoint, downloading straight from S3 beats everything else hands down as it's no load on the rails box. It doesn't often meet other considerations like security, logging, access control, etc.
I am new to AWS services, so forgive the simple question:
I'm building an iOS app that is currently using RDS and EC2 instances. I now want to store photos and audio files in an S3 bucket. I know there is an iOS SDK which makes it easy to download from and upload to an S3 bucket directly from the iOS side. However, should I also be using Cloudfront to decrease latency in addition? What's the best way to go about this?
I recently built an app that moved photos back and forth between S3. I looked into using Amazon's SDK but it was just too bulky.
Instead I used Matt Thompson's AFAmazonS3Manager.
Worth looking into if you are building an app with AWS.
Well, it depends. You should use cloudfront if you need to reduce latency, and especially if your users a spread out all over the globe - cloudfront will help with that.
On the otherhand, there is an extra cost in using it (sometimes substantial), so if you are not having a latency problem, why bother?
If you decide you need cloudfront, it is extraordinarily easy to add in.
I am solving the question about downloading a large amounts of picture from the web. I have XML file with links to images on web sites and I have to download this images. Every image have ca ±3MB and the count of images is tens of thousands, so is not possible to store these images on hosting (100.000 x 3MB)...
And these images I need to display on sites. I don't worked yet with so large amounts of data yet, so I would like to ask you, what could be the best idea for displaying these images on "my" page.
My first ideas:
- store only links into my database table and then for displaying images use just image_tag URL_OF_IMAGE
- some a way of caching images/links of images (I don't know specifically)
Can you help me, please, what you think will be the fastest way for displaying images from foreigners sources?
Thank you in advance,
M.
Amazon S3, but as ezkl says, make sure you have licensing rights to the images or you'll land yourself in trouble (and in any case it is wrong to use the content without permission).