My heroku application resizes images as thumbnails: these thumbnails are supposed to be stored by Cloudfront.
Requesting the thumb to Heroku will cause a image to be generated, which takes time, and should only be done once for each image.
Our application always access these images through Cloudfront: so the images should be generated once, then they'd be stored by Cloudfront which would serve us as long as the cache would be deemed valid.
We receive mails everytime a thumb is generated. The problem is, when we try to access those thumbs, our Heroku server is asked to generate the thumbnail again: only then the thumb is properly cached, and we can freely access it without any traffic being sent to our server.
Does anyone know why such a thing would happen ?
Related
I'm using ActiveStorage for the first time.
Everything works fine in development but in production (Heroku) my images disappear without a reason.
They were showing ok the first time, but now no image is displayed. In the console I can see this error:
GET https://XXX.s3.amazonaws.com/variants/Q7MZrLyoKKmQFFwMMw9tQhPW/XXX 403 (Forbidden)
If I try to visit that URL directly I get an XML
<Error>
<Code>AccessDenied</Code>
<Message>Request has expired</Message>
<X-Amz-Expires>300</X-Amz-Expires>
<Expires>2018-07-24T13:48:25Z</Expires>
<ServerTime>2018-07-24T15:25:37Z</ServerTime>
<RequestId>291D41FAC6708334</RequestId>
<HostId>lEVGuwA6Hvlm/i40PeXaje9SEBYks9+uk6DvBs=</HostId>
</Error>
This is what I have in the view
<div class="cover" style="background-image: url('<%= rails_representation_path(experience.thumbnail) %>')"></div>
This is what I have in the model
def thumbnail
self.cover.variant(resize: "300x300").processed
end
In simple words, I don't want images to expire but to be always there.
Thanks
ActiveStorage does not support non-expiring link. It uses expiring links (private), and support uploading files only as private on your service.
It was a problem for me too, and did 2 patches (caution) for S3 only, one simple ~30lines that override ActiveStorage to work only with non-expiring (public) links, and another that add an acl option to has_one_attached and has_many_attached methods.
Hope it helps.
Your question doesn't say so, but it's common to use a CDN like AWS CloudFront with a Rails app. Especially on Heroku you probably want to conserve compute power.
Here is what happens in that scenario. You render a page as usual, and all the images are requested from the asset host, which is the CDN, because that's how it is configured to integrate. Its setup to fetch anything it doesn't find in cache from origin, which is your application again.
First all image requests are passed through. The ActiveStorage controller creates signed URLs for them, and the CDN passes them on, but also caches them.
Now comes the problem. The signed URL expires in 5 minutes by default, but the CDN caches usually much longer. This is because usually you use digest assets, meaning they are invalidated not by time but by name, on any change.
The solution is simple. Increase the expiry of the signed URL to be longer than the cache's TTL. Now the cache drops the cached signed URL before it becomes invalid.
Set the URL expiry using ActiveStorage::Service.url_expires_in in 5.2 or directly in Rails.application.config.active_storage.service_urls_expire_in in an initializer see this answer for details.
To set cache TTL in CloudFront: open the AWS console, pick the distribution, open the Behavior tab, scroll down to these fields:
Then optionally issue an invalidation to force re-caching of all contents.
Keep in mind there is a security trade-off. If the image contents are private, then they don't belong into a CDN most likely, and shouldn't have long lasting temp URLs either. In that case choose a solution that exempts attachments from CDN altogether. Your application will have to handle the additional load of signing all attached assets' URLs on top of rendering the relevant page.
Further keep in mind, that this isn't necessarily a good solution, but more of a workaround. With the above setup you will cache redirects, and the heavier requests will hit your storage bucket directly. The usual scenario for CDNs is large media, not lightweight redirects. You do relieve the app of handling a lot of requests though. How much that is a valid optimization should be looked into.
I had this same issue, but after I corrected the time on my computer, the problem was resolved. It was a server time difference, that the aws servers did not recognize.
#production.rb
Change
config.active_storage.service = :local
To
config.active_storage.service = :amazon
Should match aws/amazon whatever you defined it as in storage.yml
I'm planning to upload images to facebook to my account first, get their "src" and then show them in my Rails app where img src will point to the location of the images at facebook that I've uploaded them.
Is there overhead in this approach as opposed having images in my own website? Will that slow down the server? And will this approach do in general? Is it legal?
No there is no overhead, in fact this could actually speed up your app by reducing the load of request your server will receive. This is basically like using a distributed CDN for your javascript and css.
Typically your rails server will serve an html response with links to css, javascript, and images. The user's browser then starts rendering this html and will make requests when it encounters these links. If all these links point back to your server, then your rails server has to handle serving these static assets (and it can only handle so many requests per second).
In production its common to put your assets on a CDN such as Amazon Web Services to decrease the load on your rails server. As long as your facebook image is public, I believe this is actually a good idea.
Hi I'm using fragment caching extensively in my rails application. I've cached fragments which have user images in them. So when a user changes his/her avatar(image), a lot of fragments have to be expired. To solve this, I've made some settings so that a user's avatar URL is always constant ( "/avatars/:user_id/thumbs" ).
Im using amazon s3 for storage and the expires header for the images is "no-cache".
Sample image URL : https://s3.amazonaws.com/bucket_name/avatars/388/thumbs
In IE and Chrome, Im facing a peculiar problem. When a user changes his avatar, it is not reflected immiediately on client side. It takes 2-3 page refresh for the image to change. Has anyone faced similar issues before?
You're going about this the wrong way in my opinion. Expire the fragment as you're doing but allow rails' cache busting timestamps to force the browser to download new images.
e.g. s3.amazonaws.com/bucket_name/avatars/388/thumbs/filename.jpg?1230601161
In my web app each user will be having a profile image those images are stored in Amazon s3. If the user signs in i need to show that image and that will stay in the side bar in all the pages he enters. Once he signs in is there anyway i can cache the image so that i no need to get back from Amazon s3 every time ? when he again updates the image i need to clear the cache.
You can use standard Http Caching for this.
You should set the Cache-Control and/or expires headers depending on your needs.
All the major S3 clients support setting these headers or you can set using the S3 API's or SDKs/Libraries etc.
In order to re-download the image if it has changed, you can add a querystring to the url.
eg
http://mypath/myfile.ext?v=1
http://mypath/myfile.ext?v=2
I would like to know if Google App Engine can be used as a Content delivery network like aws S3. I'm running a RoR app on Heroku and I would like store my uploaded files on GAE instead of s3.
If it's possible what would be the best way to do it?
http://24ways.org/2008/using-google-app-engine-as-your-own-cdn
It won't be able to host files over 1MB though.
Make sure to read through the comments on that blog post as well, some have concerns about the terms of service.
GAE in itself isn't meant to be a CDN... that doesn't, however, stop you from writing a CDN application on top of it. The only limit you'll need to worry about is the 50 MB limit on the size of the blobstore. Such an app will have to provide a URL that you can hit to get the upload URL, which could then be used to upload the file. The download url can also be generated with the upload URL, and used to access the content.