I currently using carrierwave with fog to store and upload images using an s3 bucket but the images load much slower than they should. These images load almost instantly when stored as part of the application - but stored with carrierwave and fog it takes a few seconds.
Is this a problem with my s3 setup or carrierwave/fog? My carrierwave config is the following:
CarrierWave.configure do |config|
config.fog_credentials = {
:provider => 'AWS', # required
:aws_access_key_id => '***', # required
:aws_secret_access_key => '***', # required
}
config.cache_dir = "#{Rails.root}/tmp/uploads" # To let CarrierWave work on heroku
config.fog_directory = 'bucketname' # required NB: having '.' in the bucket name creates an untrusted certificate
config.fog_public = false # optional, defaults to true
config.fog_attributes = {'Cache-Control'=>'max-age=315576000'} # optional, defaults to {}
end
I do have my s3 bucket configured for the US and I'm located in Australia so that might pose a few problems - but my heroku app is also configured to the US and it loads the same images blazingly quick when they're stored as part of the app itself. Maybe aws isn't the best solution?
Anyway any solutions on how I can improve the speed of image load time would be great. It just seems unnecessarily slower than it should be.
It sounds like you want to use CloudFront, Amazon's CDN (content delivery network) service that integrates with S3. Using a CDN will globally replicate the content you're storing in CDN (for a price), which should improve your load times.
After you set up a CloudFront account and link it to S3, add a line like the following to your CarrierWave configuration:
config.asset_host = "http://1234567.cloudfront.net"
With the URL that you get during CloudFront setup.
Unfortunately it looks like you may also need to set config.fog_public = true for Carrierwave to be able to use Amazon's CDN.
Related
I am using ruby 2.4.0p0 and Rails 5.2.3
In the production.rb file I have done the following setting:
# Force all access to the app over SSL, use Strict-Transport-Security, and use secure cookies.
config.force_ssl = true
if Rails.application.config.force_ssl
Rails.application.routes.default_url_options[:protocol] = 'https'
end
But still the resource are getting rendered on http rather then https do I need to do any thing extra, please provide the desired thing to be done to get all assets getting loaded from s3 loads over https.
The website is live here at: https://tukaweb.com/asset/garments
The s3 resources are at http
ex: http://tukaweb.s3.amazonaws.com/uploads/three_d_garment/thumbnail/7/Womens_Dress_35-41_Thumbnail.png?X-Amz-Expires=600&X-Amz-Date=20200918T060705Z&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIRDA3IQIVTEPMN6Q%2F20200918%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-SignedHeaders=host&X-Amz-Signature=1792bd4cc2437abd950b7d16d360d09e64423bdef89f41c24a5386d35e982dfa
need them over https.
The required change should be done inside the carrierwave.rb inside the webapp/config/initializers directory modified the settings as:
CarrierWave.configure do |config|
config.fog_provider = 'fog/aws'
config.fog_credentials = {
provider: 'AWS',
aws_access_key_id: 'XXXXXXXXXX',
aws_secret_access_key: 'xxxxxxxxxx',
use_iam_profile: false,
region: 'us-west-2', # optional, defaults to 'us-east-1'
# host: 'ec2-xx-xxx-xx-xx.us-west-2.compute.amazonaws.com', # optional, defaults to nil
:endpoint => 'https://s3.amazonaws.com',
}
config.fog_directory = 'tukaweb' # required
config.fog_public = false # optional, defaults to true
# config.fog_attributes = { cache_control: "public, max-age=#{365.days.to_i}" } # optional, defaults to {}
end
The line which is responsible for changing s3 resource to be downloaded from https instead of http
:endpoint => 'https://s3.amazonaws.com' ## earlier it was 'http://s3.amazonaws.com'
Force SSL only works for the incoming requests to the rail's routes. If you have an image link set to http://image-domain.com/image it's going to use the http, and you'll get a mixed content warning. You need to ensure anything external to the app's routes is going to be using SSL or a secure connection as well.
First thing I do when I see a mixed content warning is to do a global search of the codebase for http:// to find everywhere that isn't using https://. I may or may not do a global find + replace depending on what I see, there are cases where it needs to be http:// or it won't work right (if the site doesn't have an https:// version).
Next thing is to work out what is causing the insecure url, here it is S3, so I would be looking at what uses S3, and working out how I can tell it to use SSL or a secure connection.
Note: The other answer does well explaining what your actual issue is, but this may be more useful to others for general troubleshooting of mixed content issues, and would lead to the same result.
I am using sitemap_generator gem with rails 6 on heroku. I am told the easiest way is to use an s3 on amazon and bridge with aws_fog.
The implementation is well documented on the gem side... but I am struggling to make sure the amazon config is correct.
I search a lot and couldn't find anything so I was hoping someone could help
I configure an s3 instance lets name it example and I add it to region US East(Ohio). This is all pretty simple.
The Properties tab... what should and shouldn't be selected? i select nothing.
The Permissions tab. I make public, although this feels wrong... the bucket is for a sitemap, so it should be public right?
I set up my region as per the doumentation
SitemapGenerator::Sitemap.default_host = "https://www.example.com"
SitemapGenerator::Sitemap.public_path = 'tmp/'
SitemapGenerator::Sitemap.sitemaps_host = "https://example.s3.amazonaws.com/"
SitemapGenerator::Sitemap.sitemaps_path = 'sitemaps/'
SitemapGenerator::Sitemap.adapter = SitemapGenerator::S3Adapter.new(fog_provider: 'AWS',
aws_access_key_id: Rails.application.credentials.aws[:access_key_id],
aws_secret_access_key: Rails.application.credentials.aws[:secret_access_key],
fog_region: 'us-east-2')
when i hit rake sitemap:refresh:no_ping on my local host I get :status_line => "HTTP/1.1 301 Moved Permanently\r\n"
I think maybe i need to add the sitemaps folder to the s3 instance, so i do but i still get the :status_line => "HTTP/1.1 301 Moved Permanently\r\n".
Any tips would be great...
I am also using sitemap-generator gem on my rails application (heroku hosted and rails 6). I have the following code inside config/sitemap.rb, before SitemapGenerator::Sitemap.create. I have configured it with aws-sdk-s3 gem and it goes like this:
require 'aws-sdk-s3'
SitemapGenerator::Sitemap.default_host = "https://www.example.com"
SitemapGenerator::Sitemap.sitemaps_host = 'https://example.s3.eu-west-2.amazonaws.com/'
SitemapGenerator::Sitemap.adapter = SitemapGenerator::AwsSdkAdapter.new(Rails.application.credentials.dig(:amazon, :s3, :bucket),
aws_access_key_id: Rails.application.credentials.dig(:amazon, :s3, :access_key_id),
aws_secret_access_key: Rails.application.credentials.dig(:amazon, :s3, :secret_access_key),
aws_region: Rails.application.credentials.dig(:amazon, :s3, :region)
)
I have a ruby on rails application where users can upload their avatar or change it. First I stored the images in Amazon s3 but then I realized that content contents were being served slowly and decided to use Amazon cloudfront.
There is no problem for uploading and getting avatar. However, I can see that an updated photo changes immediately but I expect to invalidate it through cloudfront api. And uploading an image takes a lot of time.
At this point I can't decide whether I use cloudfront correctly or not.
This my carrierwave.rb file inside config/initializer:
CarrierWave.configure do |config|
config.fog_provider = 'fog/aws'
config.fog_credentials = {
provider: 'AWS',
aws_access_key_id: 'key',
aws_secret_access_key: 'value',
region: 'us-east-1'
}
config.storage :fog
config.asset_host = 'http://images.my-domain.com'
config.fog_directory = 'bucket_name'
config.fog_public = true
config.fog_attributes = { cache_control: "public, max-age=315576000" }
end
I can't see what I'm missing ? How can I be assure that I'm using cloudfront properly ?
Thanks.
Your images aren't being stored in CloudFront, they're being served through CloudFront's CDN.
First request for an image served through CF looks like this:
Browser -> CloudFront -> S3
|
Browser <- CloudFront <-
The second request for an image just looks like this:
Browser -> CloudFront
|
Browser <-
The second request never hit's CF because CF has cached the result for that URL.
NOW, your avatar's updating immediately is simply probably because it's being uploaded to S3 and resulting in a new URL, and thusly, an immediate update. This is how you want it to work.
Just switched over to Refile for image uploads on my Rails application.
I have the uploads going directly to my s3 bucket. I have two buckets configured (with same settings), one for testing and one for production.
Everything in my local development works fine, and uploading to my bucket in production works, but on all the uploaded images I get the following when rendering them on a webpage.
Resource interpreted as Image but transferred with MIME type text/html:
Also in production, the images are not showing up.
I've looked into permissions for the buckets, but they seem to be good to go. I've also looked at others questions/answers regarding this warning, but have been unable to find any that pertain here.
If any code is needed please let me know.
config/initializers/refile.rb
require 'refile/backend/s3'
aws = {
access_key_id: Rails.application.secrets.aws['access_key_id'],
secret_access_key: Rails.application.secrets.aws['secret_access_key'],
bucket: Rails.application.secrets.aws['s3_bucket_name'],
use_ssl: true
}
Refile.cache = Refile::Backend::S3.new(max_size: 5.megabytes, prefix: 'cache', **aws)
Refile.store = Refile::Backend::S3.new(prefix: 'store', **aws)
Gist of the image helper
attachment_image_tag(avatar, :image, :fill, size, size)
Thanks for taking a look.
I'm using Carrierwave with Fog for managing uploads in a Rails App. And recently, some pages of the app were changed to be served over HTTPS only. So my Fog Host was changed from
config.fog_host = "a%d.cdn.com"
to
config.fog_host = "https://my-bucket.s3.amazonaws.com"
Is it possible to change Carrierwave fog_host to a Proc that takes in a request and decides which fog host to use, just like with Rails asset_host as its described here
http://api.rubyonrails.org/classes/ActionView/Helpers/AssetTagHelper.html
config.fog_host = Proc.new { |req|
if req.ssl?
"ssl_asset_host"
else
"non_ssl_asset_host"
end
end
How do I go about it?