My asset_sync gem does not upload to s3 when I precompile assets
asset_sync.rb
if defined?(AssetSync)
AssetSync.configure do |config|
config.fog_provider = 'AWS'
config.aws_access_key_id = ENV['AWS_ACCESS_KEY_ID']
config.aws_secret_access_key = ENV['AWS_SECRET_ACCESS_KEY']
# To use AWS reduced redundancy storage.
# config.aws_reduced_redundancy = true
config.fog_directory = ENV['FOG_DIRECTORY']
# Invalidate a file on a cdn after uploading files
# config.cdn_distribution_id = "12345"
# config.invalidate = ['file1.js']
# Increase upload performance by configuring your region
config.fog_region = 'ap-southeast-2'
#
# Don't delete files from the store
# config.existing_remote_files = "keep"
#
# Automatically replace files with their equivalent gzip compressed version
# config.gzip_compression = true
#
# Use the Rails generated 'manifest.yml' file to produce the list of files to
# upload instead of searching the assets directory.
# config.manifest = true
#
# Fail silently. Useful for environments such as Heroku
# config.fail_silently = true
end
end
production.rb
config.assets.enabled = true
config.assets.digest = true
config.action_controller.asset_host = "//#{ENV['FOG_DIRECTORY']}.s3.amazonaws.com"
config.assets.initialize_on_precompile = true
application.rb
config.assets.enabled = true
config.assets.digest = true
When I precompile, I'm not even getting any message to show it is uploading.
Is there a reason why this is happening?
may be you have:
group :assets do
gem 'asset_sync'
end
you should put that gem out from that group
I know this issue is old... but I just inherited a Heroku app and am reading how to asset_sync with Heroku and came across this, which states ENV variables are not available when compiling assets on Heroku.
Related
I am in the process of upgrading our product from rails 4.1 to 5.2. I'm hung up on what appears to be an asset pipeline related issue. When I load the app in our browser, I see the following error in the server log and the app is missing all styling and javascript code that it normally has.
DEPRECATION WARNING: The asset application.js" is not present in the asset pipeline.Falling back to an asset that may be in the public folder.
This behavior is deprecated and will be removed.
To bypass the asset pipeline and preserve this behavior,
use the `skip_pipeline: true` option.
When I wget the index.html of the application, I see that all of the images and javascript urls are lacking the fingerprint digest they should have. Interestingly enough, they do have the S3 asset_host we have, so they are in fact being processed by the helpers.
To demonstrate, I have two servers which have identical configurations, but one is running 4.1 and the other 5.2. On both, I am using S3 as an asset_host and digest is turned on. I've run the following commands in their consoles:
=== Rails 4 ===
Loading qatest environment (Rails 4.1.0)
irb(main):001:0> Rails.application.config.action_controller.asset_host
=> "https://redacted.s3.amazonaws.com"
irb(main):002:0> Rails.application.config.assets.digest
=> true
irb(main):003:0> ActionController::Base.helpers.asset_path('application.js')
=> "https://redacted.s3.amazonaws.com/assets/application-042f2014ca329c79c304ab1332557040d3f7b922247202f40c28acc950f30ef8.js"
=== Rails 5 ===
Loading sean environment (Rails 5.2.1)
irb(main):001:0> Rails.application.config.action_controller.asset_host
=> "https://redacted.s3.amazonaws.com"
irb(main):002:0> Rails.application.config.assets.digest
=> true
irb(main):003:0> ActionController::Base.helpers.asset_path('application.js')
=> "https://redacted.s3.amazonaws.com/application.js
As requested, production.rb
require "#{File.dirname(__FILE__)}/includes/s3_assets"
Rails.application.configure do
# Settings specified here will take precedence over those in config/application.rb
config.eager_load = true
# In the development environment your application's code is reloaded on
# every request. This slows down response time but is perfect for development
# since you don't have to restart the web server when you make code changes.
config.cache_classes = false
# Log error messages when you accidentally call methods on nil.
config.whiny_nils = false
config.log_level = ENV['SHOW_SQL'] != 'false' ? :debug : :info
# Show full error reports and disable caching
config.consider_all_requests_local = true
config.action_controller.perform_caching = false
config.action_mailer.raise_delivery_errors = true
config.action_mailer.perform_deliveries = true
config.action_mailer.delivery_method = :smtp
config.action_mailer.smtp_settings = {
user_name: Rubber.config.email_username,
password: Rubber.config.email_password,
address: Rubber.config.email_server,
port: Rubber.config.email_port,
enable_starttls_auto: true, # detects and uses STARTTLS
authentication: 'login' # Mandrill supports 'plain' or 'login'
}
# Print deprecation notices to the Rails logger
config.active_support.deprecation = :log
# Only use best-standards-support built into browsers
config.action_dispatch.best_standards_support = :builtin
# Raise exception on mass assignment protection for Active Record models
# config.active_record.mass_assignment_sanitizer = :strict
# Log the query plan for queries taking more than this (works
# with SQLite, MySQL, and PostgreSQL)
# config.active_record.auto_explain_threshold_in_seconds = 0.5
# Compress JavaScripts and CSS
config.assets.compress = true
# Don't fall back to assets pipelin if a precompiled asset is
# missed.
config.assets.compile = false
# Generate digests for assets URLs
config.assets.digest = true
config.assets.enabled = true
# Nginx will serve as an asset proxy to s3, where assets are
# stored.
config.serve_static_assets = false
# instead of bundling the assets, include multiple css style includes
config.assets.debug = true
config.assets.logger = false
# Enable serving of images, stylesheets, and JavaScripts from an asset server
# NB: The exclusion of any protocol in the asset host declaration above will allow browsers to choose the transport mechanism on the fly.
# So if your application is available under both HTTP and HTTPS the assets will be served to match.
configure_s3_assets config
config.action_controller.asset_host = "https://#{Rubber.config.s3_assets_bucket}.s3.amazonaws.com"
config.action_mailer.default_url_options = { :host => Rubber.config.external_host }
config.sequel.search_path = %w(public audit)
end
config/initializers/assets.rb
# Be sure to restart your server when you modify this file.
# Version of your assets, change this if you want to expire all your assets.
Rails.application.config.assets.version = '1.0'
# Add additional assets to the asset load path
# Rails.application.config.assets.paths << Emoji.images_path
# Precompile additional assets.
# application.js, application.css, and all non-JS/CSS in app/assets folder are already added.
# Rails.application.config.assets.precompile += %w( search.js )
Rails.application.config.assets.precompile += %w( *.js *.png *.eot *.woff *.ttf *.svg *.gif)
config/environments/includes/s3_assets.rb
def configure_s3_assets(config)
# Do not compress assets
config.assets.compress = false
# Disable Rails's static asset server (Apache or nginx will already do this)
config.serve_static_assets = false
# Generate digests for assets URLs
config.assets.digest = true
config.assets.enabled = true
# Precompile additional assets (application.js, application.css, and all non-JS/CSS are already added)
config.assets.precompile += %w(
)
# find javascripts/ stylesheets/ -type f | sed 's/\.scss//' | sed 's/\.coffee//' |
# sed 's/^.*\/\///'| sort
# defaults to application.js, application.css
# application.css has home.css ?
config.assets.precompile += %w(
handlebars.js
jquery-1.7.js
json2.js
)
end
After digging around in the sprockets code, I discovered that my list of resolvers was empty. I'm not sure what was missing from my configuration, but I fixed the problem by adding the following to config/application.rb
config.assets.resolve_with = [:manifest] // in production
config.assets.resolve_with = [:manifest, :environment] // in development
I was doing some changes in my app locally and everything was working fine, but after I deployed my app, the stylesheet files are not working. I already cloned the app to check if the files are present, and they are. So I imagine that the problem is in any of my config files.
How is:
How should be:
Here are my config files:
environments/production.rb
Rails.application.configure do
# Settings specified here will take precedence over those in config/application.rb.
# Code is not reloaded between requests.
config.cache_classes = true
# Eager load code on boot. This eager loads most of Rails and
# your application in memory, allowing both threaded web servers
# and those relying on copy on write to perform better.
# Rake tasks automatically ignore this option for performance.
config.eager_load = true
# Full error reports are disabled and caching is turned on.
config.consider_all_requests_local = false
config.action_controller.perform_caching = true
# Ensures that a master key has been made available in either ENV["RAILS_MASTER_KEY"]
# or in config/master.key. This key is used to decrypt credentials (and other encrypted files).
# config.require_master_key = true
# Disable serving static files from the `/public` folder by default since
# Apache or NGINX already handles this.
config.public_file_server.enabled = true
# Compress JavaScripts and CSS.
config.assets.js_compressor = :uglifier
# config.assets.css_compressor = :sass
# Do not fallback to assets pipeline if a precompiled asset is missed.
config.assets.compile = false
# `config.assets.precompile` and `config.assets.version` have moved to config/initializers/assets.rb
# Enable serving of images, stylesheets, and JavaScripts from an asset server.
# config.action_controller.asset_host = 'http://assets.example.com'
# Specifies the header that your server uses for sending files.
# config.action_dispatch.x_sendfile_header = 'X-Sendfile' # for Apache
# config.action_dispatch.x_sendfile_header = 'X-Accel-Redirect' # for NGINX
# Store uploaded files on the local file system (see config/storage.yml for options)
config.active_storage.service = :local
# Mount Action Cable outside main process or domain
# config.action_cable.mount_path = nil
# config.action_cable.url = 'wss://example.com/cable'
# config.action_cable.allowed_request_origins = [ 'http://example.com', /http:\/\/example.*/ ]
# Force all access to the app over SSL, use Strict-Transport-Security, and use secure cookies.
# config.force_ssl = true
# Use the lowest log level to ensure availability of diagnostic information
# when problems arise.
config.log_level = :debug
# Prepend all log lines with the following tags.
config.log_tags = [ :request_id ]
# Use a different cache store in production.
# config.cache_store = :mem_cache_store
# Use a real queuing backend for Active Job (and separate queues per environment)
# config.active_job.queue_adapter = :resque
# config.active_job.queue_name_prefix = "cronograma_capilar_#{Rails.env}"
config.action_mailer.perform_caching = false
# Ignore bad email addresses and do not raise email delivery errors.
# Set this to true and configure the email server for immediate delivery to raise delivery errors.
# config.action_mailer.raise_delivery_errors = false
# Enable locale fallbacks for I18n (makes lookups for any locale fall back to
# the I18n.default_locale when a translation cannot be found).
config.i18n.fallbacks = true
# Send deprecation notices to registered listeners.
config.active_support.deprecation = :notify
# Use default logging formatter so that PID and timestamp are not suppressed.
config.log_formatter = ::Logger::Formatter.new
config.cache_classes = true
config.serve_static_assets = true
config.assets.compile = true
config.assets.digest = true
# Use a different logger for distributed setups.
# require 'syslog/logger'
# config.logger = ActiveSupport::TaggedLogging.new(Syslog::Logger.new 'app-name')
if ENV["RAILS_LOG_TO_STDOUT"].present?
logger = ActiveSupport::Logger.new(STDOUT)
logger.formatter = config.log_formatter
config.logger = ActiveSupport::TaggedLogging.new(logger)
end
# Do not dump schema after migrations.
config.active_record.dump_schema_after_migration = false
#config.web_console.whiny_requests = false
config.paperclip_defaults = {
storage: :s3,
s3_credentials: {
bucket: ENV.fetch('S3_BUCKET_NAME'),
access_key_id: ENV.fetch('AWS_ACCESS_KEY_ID'),
secret_access_key: ENV.fetch('AWS_SECRET_ACCESS_KEY')
},
s3_region: ENV.fetch('AWS_REGION')
}
Paperclip.options[:image_magick_path] = "/opt/ImageMagick/bin"
Paperclip.options[:command_path] = "/opt/ImageMagick/bin"
end
config/application.rb
require_relative 'boot'
require 'rails/all'
# Require the gems listed in Gemfile, including any gems
# you've limited to :test, :development, or :production.
Bundler.require(*Rails.groups)
module CronogramaCapilar
class Application < Rails::Application
config.assets.initialize_on_precompile = false
config.assets.paths << "#{Rails}/vendor/assets/fonts"
# Initialize configuration defaults for originally generated Rails version.
config.load_defaults 5.2
# Settings in config/environments/* take precedence over those specified here.
# Application configuration can go into files in config/initializers
# -- all .rb files in that directory are automatically loaded after loading
# the framework and any gems in your application.
end
end
I also tried run the commands bundle exec rake assets:precompile, RAILS_ENV=production rake assets:precompile and not working :(
For context, I essentially followed this guide to get my rails app set up: https://medium.com/#jatescher/how-to-set-up-a-rails-4-2-app-on-aws-with-elastic-beanstalk-and-postgresql-3f9f29c046e2
My app is now running on AWS:
http://ems-heroes-dev.elasticbeanstalk.com/
64bit Amazon Linux 2015.03 v2.0.1 running Ruby 2.2 (Puma)
However, I can't get any of my assets to load.
Here is my config/environments/production.rb:
EmsHeroes::Application.configure do
config.cache_classes = true
config.eager_load = true
config.consider_all_requests_local = false
config.action_controller.perform_caching = true
config.serve_static_assets = true
# Compress JavaScripts and CSS.
config.assets.js_compressor = :uglifier
config.assets.compress = true
config.assets.compile = true
# Bower asset paths
root.join('vendor', 'assets', 'components').to_s.tap do |bower_path|
config.sass.load_paths << bower_path
config.assets.paths << bower_path
end
# Precompile Bootstrap fonts
config.assets.precompile << %r(bootstrap-sass/assets/fonts/bootstrap/[\w-]+\.(?:eot|svg|ttf|woff2?)$)
# Minimum Sass number precision required by bootstrap-sass
::Sass::Script::Value::Number.precision = [8, ::Sass::Script::Value::Number.precision].max
config.less.paths << "#{Rails.root}/lib/less/protractor/stylesheets"
config.less.compress = true
# Generate digests for assets URLs.
config.assets.digest = true
config.assets.enabled = true
# Version of your assets, change this if you want to expire all your assets.
config.assets.version = '1.0'
# Specifies the header that your server uses for sending files.
# config.action_dispatch.x_sendfile_header = "X-Sendfile" # for apache
config.action_dispatch.x_sendfile_header = 'X-Accel-Redirect' # for nginx
# Force all access to the app over SSL, use Strict-Transport-Security, and use secure cookies.
config.force_ssl = false
config.log_level = :info
config.i18n.fallbacks = true
config.active_support.deprecation = :notify
config.log_formatter = ::Logger::Formatter.new
end
My assets are loading fine via heroku though. (However, I dont plan on using heroku any more)
EDIT
It looks like its an issue with rake assets:precompile providing the wrong file names. Ex: My compiled css file is: "public/assets/application-79dc234c01a4f604b52fc53ff49ac89d.css" but Im getting a 404 looking for "application.css". Renaming the file to "public/assets/application.css" will load it properly.
Any idea why this would be the case and how I can get it to precompile properly?
This string after file name is called "digest", it's used to notify browser when file content changes. You need to use view helpers (asset_path, etc) to get name with digest, or you can turn off this feature with config.assets.digest = false
Has anyone got the asset_sync gem to work with Rails 4? I have never had a problem with it with Rails 3, but i cannot precompile my assets to my S3 bucket anymore, what happens is everything is just compiled into my public folder.
Could anyone offer advice on resources to look at or summarise the key differences between Rails 3 and 4 that would cause this to fail. Some examples of configuration used would be helpful from those who have got it working. I am at a loss on how to start debugging this
Any advice and help appreciated
Thanks
EDIT
Current config
asset_sync.rb # Within Initializer
if defined?(AssetSync)
AssetSync.configure do |config|
config.fog_provider = ENV['FOG_PROVIDER']
config.aws_access_key_id = ENV['AWS_ACCESS_KEY_ID']
config.aws_secret_access_key = ENV['AWS_SECRET_ACCESS_KEY']
config.fog_directory = ENV['FOG_DIRECTORY']
config.fog_region = ENV['FOG_REGION']
config.existing_remote_files = "delete"
config.gzip_compression = true
config.manifest = true
config.custom_headers = { '.*' => { cache_control: 'max-age=315576000', expires: 1.year.from_now.httpdate } }
end
end
Production.rb
YmcaView::Application.configure do
config.action_controller.asset_host = "//#{ENV['FOG_DIRECTORY']}.s3.amazonaws.com"
config.cache_classes = true
config.eager_load = true
config.consider_all_requests_local = false
config.action_controller.perform_caching = true
config.serve_static_assets = true
config.assets.js_compressor = :uglifier
config.assets.compile = true
config.assets.digest = true
config.assets.version = '1.0'
config.log_level = :info
config.i18n.fallbacks = true
config.active_support.deprecation = :notify
config.log_formatter = ::Logger::Formatter.new
end
Output when running rake assets:precompile RAILS_ENV=production
I, [2014-03-04T13:04:41.176230 #6085] INFO -- : Writing /home/richardlewis/Rails/ymca_view/public/assets/760x380_10-477c3cdc939905d6b32f9997e7f93072.jpg
I, [2014-03-04T13:04:41.177963 #6085] INFO -- : Writing /home/richardlewis/Rails/ymca_view/public/assets/aboutus-d9ad504fcd86071255015d24780caef8.jpg
I, [2014-03-04T13:04:45.018794 #6085] INFO -- : Writing /home/richardlewis/Rails/ymca_view/public/assets/application-90d317f561a8b0e84124ce7bb872f867.js
I, [2014-03-04T13:04:46.666640 #6085] INFO -- : Writing /home/richardlewis/Rails/ymca_view/public/assets/application-b422f803b56ae2f5c56a648891ec553e.css
[fog][WARNING] Unable to load the 'unf' gem. Your AWS strings may not be properly encoded.
Yep we've got it working in Rails 4
You need to do this:
#GemFile (might need aws_sdk gem too)
gem "asset_sync", "~> 1.0.0"
#config/environments/production.rb
config.action_controller.asset_host = "//#{ENV['FOG_DIRECTORY']}.s3.amazonaws.com"
$-> rake assets:precompile RAILS_ENV=production
We use Figaro to store ENV vars locally, here's what we got:
#config/application.yml
FOG_DIRECTORY: "***BUCKET_NAME****"
FOG_PROVIDER: "AWS"
FOG_REGION: "eu-west-1"
ASSET_SYNC_GZIP_COMPRESSION: "true"
ASSET_SYNC_MANIFEST: "true"
ASSET_SYNC_EXISTING_REMOTE_FILES: "delete"
I didn't need to use the 1.0.0 version. I used the latest gem, it seems that it doesn't actually upload (at least on EY) when it is in the :assets group. I had to add it to the general group in my Gemfile and it worked fine.
asset_sync gem error :(
The first error I started out with was related to the unf gem, so I added gem "unf", "~> 0.1.3" to my gemfile and tried things again. No luck :( After more research I may have an issue with another setting related to my aws bucket. My dns is hosted via route53 and I wanted to use a custom domain to serve my assets. This means my bucket will look like assets.domain.com instead of just a plain name. when I attempt to run rake assets:precompile it aborts with the error rake aborted!
hostname "assets.domain.com.s3-us-west-1.amazonaws.com" does not match the server certificate (OpenSSL::SSL::SSLError)" however that error appears to be wrong. In order to use your own sub domain i.e. assets. I read that the bucket must be setup as a static website. This means the url looks like assets.domain.com.s3-website-us-west-1.amazonaws.com which does not seem to match the error code.
am I missing a setting here? maybe I am crazy... thank you for your help.
my production.rb settings
# Enable serving of images, stylesheets, and JavaScripts from an asset server.
AssetSync.config.run_on_precompile = true
config.action_controller.asset_host = "http://assets.domain.com"
config.assets.prefix = "/data"
config.assets.enabled = true
config.assets.compile = true
config.assets.initialize_on_precompile = true
# Generate digests for assets URLs.
config.assets.digest = true
my current initializer file:
if defined?(AssetSync)
AssetSync.configure do |config|
config.fog_provider = 'AWS'
config.aws_access_key_id = ENV['MY_S3_ID']
config.aws_secret_access_key = ENV['MY_S3_SECRET']
# To use AWS reduced redundancy storage.
# config.aws_reduced_redundancy = true
config.fog_directory = ENV['MY_S3_BUCKET']
# Invalidate a file on a cdn after uploading files
# config.cdn_distribution_id = "12345"
# config.invalidate = ['file1.js']
# Increase upload performance by configuring your region
config.fog_region = ENV['MY_S3_ENDPOINT']
#
# Don't delete files from the store
# config.existing_remote_files = "keep"
#
# Automatically replace files with their equivalent gzip compressed version
# config.gzip_compression = true
#
# Use the Rails generated 'manifest.yml' file to produce the list of files to
# upload instead of searching the assets directory.
# config.manifest = true
#
# Fail silently. Useful for environments such as Heroku
# config.fail_silently = true
end
end
Fixed by adding: Fog.credentials = { path_style: true }
if defined?(AssetSync)
AssetSync.configure do |config|
config.fog_provider = 'AWS'
config.aws_access_key_id = ENV['MY_S3_ID']
config.aws_secret_access_key = ENV['MY_S3_SECRET']
# To use AWS reduced redundancy storage.
# config.aws_reduced_redundancy = true
config.fog_directory = ENV['MY_S3_BUCKET']
# Invalidate a file on a cdn after uploading files
# config.cdn_distribution_id = "12345"
# config.invalidate = ['file1.js']
Fog.credentials = { path_style: true }
# Increase upload performance by configuring your region
config.fog_region = ENV['MY_S3_ENDPOINT']
#
# Don't delete files from the store
# config.existing_remote_files = "keep"
#
# Automatically replace files with their equivalent gzip compressed version
# config.gzip_compression = true
#
# Use the Rails generated 'manifest.yml' file to produce the list of files to
# upload instead of searching the assets directory.
# config.manifest = true
#
# Fail silently. Useful for environments such as Heroku
# config.fail_silently = true
end
end
Reference issues: https://github.com/rumblelabs/asset_sync/issues/236, https://github.com/fog/fog/issues/2357