I'm having some problems with assets on Heroku (rails) and am hoping someone can point me in the right direction. I've got the asset_sync gem installed, and after many hours of debugging I've finally got it working. However, when I first run (with an empty S3 bucket) "git push heroku master", I get about 4 copies of every file uploaded to s3 (each with a different hash appended). Also, somehow a lot of files I previously deleted (and are no longer in my app/assets/images directory) are still somehow getting uploaded. I've deleted the public/assets folder on my local copy & pushed to git, but perhaps that folder is still there on heroku? How do I debug this? I want my assets to be properly sync'd, so if I delete an image while developing locally, it will also be removed from s3 when I next deploy.
Another possibly related problem, my static error pages (public/404.html) are not getting served on heroku, yet work fine on development- are these static html files treated as assets and meant to be uploaded to S3 too?
Running heroku run rake assets:precompile does nothing. My asset_sync.rb initializer is:
if defined?(AssetSync)
AssetSync.configure do |config|
config.fog_provider = 'AWS'
config.aws_access_key_id = 'key'
config.aws_secret_access_key = 'key'
config.fog_directory = 'bucketname'
config.fog_region = 'us-east-1'
config.existing_remote_files = "delete"
end
end
I know I should be using environment variables but it shoudln't make any difference hardcoding my access details at least while I'm testing
Thanks for any help.
Related
I'm trying to make a Rails app server its assets at S3. I'm not using any gems because want to do it manually to better understand the process. My questions regarding serving the static assets are:
Which assets should I upload to my S3 bucket: from my_app/assets or from my_app/public/assets?
If the second option, it means I have to precompile them first, right? And then upload the whole folder my_app/public (or my_app/public/assets?) ?
Since right now I'm deploying the website at heroku and the max size of the repo at heroku is 300Mb and the size of my assets is much bigger, how and should I at all make the app somehow "understand" that its assets are located at S3 so that when I say "git push heroku master" it won't upload all them from its my_app/assets or my_app/public/ folder?
You will need to change the config.action_controller.asset_host option on the Rails Configuration. Take a look at http://guides.rubyonrails.org/configuring.html#rails-general-configuration
You will indeed need to precompile the assets and sync them to S3. Remember to use RAILS_ENV=production. A good place to look at and then replicate manually would be in the asset_sync gem.
My rails app is working but its slug size is huge (220mb).
I am storing all assets on cloudfront with config.assets.digest = true and config.action_controller.asset_host = "something.cloudfront.net"
heroku run bash
In the slug, there is 30 times application-onedigest.js in public/assets, and same thing for stylesheets and images, so now the slug size is 220 mb.
Maybe one of my gem is doing middleware things (Rails asset_host, cloudfront and heroku) or maybe heroku is trying to optimize things (https://devcenter.heroku.com/changelog-items/328), but i don't think so. I don't want to exceed the maximum heroku slug size that is 300 mb, and i feel that is not normal.
How to remove old rails assets on heroku ?
EDIT
I made little commits in a css file, push to heroku, and compared slug. In the deploy process, there is those lines:
Running: rake assets:precompile
INFO -- : Writing /tmp/build_ff4eb6d7-303e-444d-9c88-938ab504ea8a/public/assets/application-700c7e1849c55312a94a353e60312500.css
Asset precompilation completed (9.60s)
Cleaning assets
Running: rake assets:clean
INFO -- : Removed application-48375ba5495e14b36afab9d4b9d97033.css
This is what i want. But when i compare the old and the new slug, there is the new application-700....css, and an another new mysterious application-a9b33ae58934ad161038cb3ebcee146c.css
And this one is actually used is the heroku webapp (when i explore css resources from the browser, after clearing cache).
So i guess precompile is done twice somewhere?
Temporary solution : heroku fork the app, i am back to 40 mb but there is still the problem.
EDIT
It seems it works when i precompile locally, but i don't like it as it violates 12factors.
Some things you'll want to consider:
rake assets:clobber
Heroku's file system
CDN
rake assets:clobber is an inbuilt way to remove old asset files from your public/assets directory. By running this command, either locally or on Heroku, you'll be able to actively remove the precompiled asset files on your system, allowing you to repopulate the folder with new files
Secondly, you have to remember Heroku runs an ephemeral file system. This means that each time you push your application to Heroku, it will just overwrite any of the files you had before, meaning (importantly), that it's not going to store your last assets. Each time you push, it will precompile the assets for you - which should ensure minimal space taken up with them.
--
Finally, if you find that your assets are taking up too much space on your application server, you may consider using a content delivery network (CDN) to serve the assets you need. You can use the asset_sync gem to push your assets to your CDN of choice
Most people tend to use a CDN with storage system to serve their assets. We use S3 with Cloudfront currently, but are looking to move to Rackspace's file serve system!
You can use the Heroku repo plugin (https://github.com/heroku/heroku-repo) and the repo:purge command to clean all the build assets out of your slug.
Using asset_sync gem is not ideal as you can run into deploy timeouts pushing all the assets over to S3 plus you end up with assets in two places. Remember S3 is not a content delivery network, you're better off using a CloudFront distribution and have it cache the assets directly from your application. See https://devcenter.heroku.com/articles/using-amazon-cloudfront-cdn for more details.
This solve it for me, in production.rb :
config.assets.compile = false
Heroku is no longer generating useless assets. I don't know how to remove the old assets in the same heroku app, but i used "heroku fork" to create a brand new heroku app with the same config / database and without old assets.
heroku run rake assets:clobber
I'm running a Rails 4 app on Heroku, using CloudFront as the CDN for some image assets. I have a Production app and a Staging app. On CloudFront, I have separate distributions for Production and Staging. I'm precompiling assets locally and pushing them to Heroku.
My production.rb has the following:
# production.rb
config.action_controller.asset_host = 'xxxx.cloudfront.net'
# Use special staging CDN if pushing to staging
config.action_controller.asset_host = 'yyyy.cloudfront.net' if ENV['PUSH_TO'] && ENV['PUSH_TO'].downcase == 'staging'
So I'm specifying a different asset_host depending on if I'm pushing to staging or production.
However, when I push to staging, the resulting images are missing and pointing to the Production CloudFront distribution.
I put some logging statements in the app to ensure that config.action_controller.asset_host is set to the correct Staging distribution.
If I manually access the image urls by changing the host from the Production host to the Staging host they work, so I know the fingerprints updated correctly.
My .css.scss.erb file contains:
.importance1 { background-image: image-url("<%= asset_path 'myImage1.png' %>"); }
I made sure to clear my public/assets and tmp folders.
So what could be causing my assets on Staging to be pointing to the wrong CloudFront distribution?
Update: I've discovered that only the assets that are referred to with asset_path from other assets (css files) are getting the wrong CDN host. Meanwhile, assets that are displayed from views using image_tag are working fine. This makes me think that the problem is during precompiling assets, the wrong url's are getting output from asset_path in the css files. Not sure why though. I'm calling rake RAILS_ENV=production assets:clean assets:precompile and like I've said, I tried modifying production.rb to explicitly use the staging CDN only as a test and it still didn't use it (the assets still point to the Production CDN.)
I finally figured out my problem.. Even though my staging app was using the correct asset_host, and the files were being served from the correct CDN, the urls within those files were pointing to the wrong CDN (the production CDN.) This was because while precompiling assets, the PUSH_TO env variable wasn't set, so the logic in my production.rb was using the production CDN. The solution was to do:
rake RAILS_ENV=production PUSH_TO=staging assets:clean assets:precompile
The key was the PUSH_TO=staging in there. This fixed all my problems!
It's clear that your conditional on this line is returning false:
config.action_controller.asset_host = 'yyyy.cloudfront.net' if ENV['PUSH_TO'] && ENV['PUSH_TO'].downcase == 'staging'
Either that or the asset_host won't allow being set twice. Don't know why that would be. Might be better to put that in an else block.
So fix that and you'll be set.
BUT....
Isn't this what environment files are for? Why don't you have this in your environments/staging.rb file? If it doesn't work there, you've fugged up your environment configuration and kinda defeated the whole purpose of multiple environments...
So I started investigating the asset pipeline in Rails3 and I have a desired use case that is somewhat off the beaten path... so I'm looking for a recipe.
I often run webrick locally for development and then run
Passenger+Apache for deployed instances of the app.
the app is configured with a suburi path, e.g. http:// server/approot/...
to make webrick paths work like deployed instances, I added map '/approot' do run app to config.ru. Now webrick is also at http:// local:3000/approot/...
The Confusion
Given this setup, I tried to use rake assets:precompile and have been having a lot of configuration problems between local, deployed -- missing files, incorrect paths, 404s in firebug, etc. Here's a smattering of solutions I've tried:
config.assets.initialize_on_precompile = false to application.rb to prevent trying to initialize the app for production, (we have several deployed environments and call them different names, ug.) when precompiling the assets.
config.assets.precompile += %w( *.js *.css ) to application.rb to include things like jquery.js and ujs and rails.js that were missing.
config.assets.prefix = "/approot/assets" to correct a problem where the map above (in config.ru) doesn't apply to assets, so assets had to be precompiled to ./public/approot/assets, but I'm not sure if that only works locally, i.e. if I deploy, will my asset paths be http:// server/approot/approot/(js|css|...)?
When precompiling assets, the rake task switches to env production, but then it is unclear from the Rails3 doc whether sprockets continues to compile on the fly locally and use those dynamic assets, or whether it will serve the static precompiled assets instead?
I tried putting config.serve_static_assets = true in environments/development.rb, but I'm not sure how this works with config.assets.compress = false and config.assets.debug = true. Setting the assets.debug to false just seems to hide the GET requests in the webrick log, although I saw a post saying that "solved the problem" [sic].
Requirments for a Recipe
So I'd like a recipe that does the following:
assets are consistently and correctly served from a path http://server/approot/assets/... whether run in passenger or webrick (i.e. deployed or local). If this isn't possible, then I can switch my local dev environment to use Passenger+Apache as well, it's not a big deal, but I just want to know if it's possible.
raw assets exist in /app/assets like normal Rails3, but when I precompile them, they work exactly the same way in deployed envs so that asset file references don't break (i.e. right now, there are a lot of refs looking for /assets/image/... when the path is clearly set up as /approot/assets/image.... (It's unclear from the Rails3 doc whether there are assumptions about deploying to root vs a suburi, e.g. http:// server/assets/... vs. http:// server/approot/assets/...)
sprockets can't be used in deployed environments (our restriction, sorry). So this means the rails3 app has to effectively look like a static asset app. I know this is what precompiled assets are supposed to do, but the pathing issues are preventing me from getting this working as advertised.
TL;DR - I feel like I'm trying a lot of separate things that might work if I only knew the right combination of them.
References
http://guides.rubyonrails.org/asset_pipeline.html#precompiling-assets
http://blog.55minutes.com/2012/02/untangling-the-rails-asset-pipeline-part-2-production/
http://blog.55minutes.com/2012/02/untangling-the-rails-asset-pipeline-part-3-configuration/
https://github.com/rails/rails/pull/3946
Ok, here's a potentially horrible answer, but it seems to work with webrick in two contexts now and it's the day after Halloween (although I haven't tried this approach in deployed slots yet).
Configuration
Unless otherwise mentioned, everything is defaults from a rails new app generation.
config/application.rb
config.assets.initialize_on_precompile = false
environments/production.rb (not really production, only used for rake asset:precompile)
config.assets.css_compressor = :yui
config.assets.js_compressor = :uglifier
environments/stage.rb (this is one of our deployed envs)
config.serve_static_assets = true
config.ru
This is the horrible part. I duplicated the map so that Rack would serve both the suburi and the root. So the controller action that shows the layout can have http:// server/approot/foo/index, while the assets within the layout can be loaded from http:// server/assets/...
map '/approot' do
run AppRoot::Application
end
map '/' do
run AppRoot::Application
end
Running it locally
$ rake assets:precompile
$ rails s
and in firebug I see the separate parts served by sprockets (all 200 OK):
GET /approot/
GET /assets/application.css?body=1
GET /assets/jquery.js?body=1
GET /assets/jquery.ujs.js?body=1
GET /assets/application.js?body=1
Ok, so now test a 'deployed' slot locally and see if the compiled assets work?
$ rails s -e stage
and then I see the correct precompiled assets (all 200 OK):
GET /approot/
GET /assets/application-xxxxxxxxxxxxxxx.css
GET /assets/application-xxxxxxxxxxxxxxx.js
Ok, so this isn't as nice as a real suburi solution and I think I'm going to have problems in deployed slots. Round 2, fight!
I have a rails app where I precompile my assets and send them to a remote AWS S3 bucket.
I would like to add my public/assets folder to my gitignore because it is starting to be pretty large and I don't want git saving all changed to the compiled files. But if I do rails complains that it is not finding the precompiled files and basically crashes the whole app.
Is there a way around this ?
Thanks !
Try to ignore public and set config.assets.enabled = false