Heroku Slug Size Reduction - .slugignore issue - ruby-on-rails

I got this error again... I don't remember how I fixed the first time this time I tried using the .slugignore file to exclude things I don't need my main folder that is taking for slug size is vendor, and node modules, I used
Heroku run bash -a businessappk3400
» Warning: Heroku update available from 7.42.5 to 7.42.8.
Running bash on ⬢ businessappk3400... up, run.8800 (Standard-1X)
~ $ du -sh * | sort -hr
and got :
So I am looking into reducing the vendor and node module files I tried putting them in the .slugignore folder I created in the root folder like this:
.slugignore:
# Ignore bundler config.
/.bundle
# Ignore the default SQLite database.
/db/*.sqlite3
/db/*.sqlite3-journal
# Ignore all logfiles and tempfiles.
./log
./tmp
./storage
./public
./node_modules
./yarn-error.log
./heroku
./vendor
./vendor/bundle
./kevin
./kevin.pug
./package-lock.json
./config/environments/database.yml
./package-lock.json
./vendor/database.yml
./public/packs
./public/packs-test
./yarn-error.log
./yarn-debug.log
.yarn-integrity
but I still pass the 500mb limit on slug... anyone has resolved this issue?
error:
WARNING in entrypoint size limit: The following entrypoint(s) combined asset size exceeds the recommended limit (244 KiB). This can impact web performance.
Entrypoints:
builder (1.02 MiB)
js/builder-ba7328e435e703a663b7.js
Asset precompilation completed (268.67s)
Cleaning assets
Running: rake assets:clean
-----> Detecting rails configuration
###### WARNING:
You have not declared a Ruby version in your Gemfile.
To declare a Ruby version add this line to your Gemfile:
```
ruby "2.6.6"
```
For more information see:
https://devcenter.heroku.com/articles/ruby-versions
###### WARNING:
Removing `Gemfile.lock` because it was generated on Windows.
Bundler will do a full resolve so native gems are handled properly.
This may result in unexpected gem versions being used in your app.
In rare occasions Bundler may not be able to resolve your dependencies at all.
https://devcenter.heroku.com/articles/bundler-windows-gemfile
###### WARNING:
Detecting rails configuration failed
set HEROKU_DEBUG_RAILS_RUNNER=1 to debug
-----> Discovering process types
Procfile declares types -> web, worker
Default types for buildpack -> console, rake
-----> Compressing...
! Compiled slug size: 565.7M is too large (max is 500M).
! See: http://devcenter.heroku.com/articles/slug-size
! Push failed
The complete error can be found here: https://codepen.io/apesyntax/pen/oNLvWdB

To reduce the Heroku slug size of my app (Rails 6.0.4.4, Ruby 3.1), I did the following:
Restrict 'wkhtmltopdf-binary' gem to development/test environments & add 'wkhtmltopdf-heroku' gem to production environment in my Gemfile (ref: https://www.eyupatis.com/how-to-decrease-heroku-slug-size-wkhtmltopdf-edition/):
gem 'wkhtmltopdf-binary', group: [:development, :test]
gem 'wkhtmltopdf-heroku', group: :production
Add 'heroku-buildpack-post-build-clean' AFTER the nodejs and ruby buildpacks in Heroku (ref: https://www.eyupatis.com/how-to-decrease-heroku-slug-size-post-build-clean-buildpack-edition/):
heroku buildpacks:add --index 3 https://github.com/Lostmyname/heroku-buildpack-post-build-clean
After this step, running heroku buildpacks listed my buildpacks as:
heroku/nodejs
heroku/ruby
https://github.com/Lostmyname/heroku-buildpack-post-build-clean
Create .slug-post-clean file in the root directory & add the /node_modules path as the only line in the file (ref: https://johntornow.com/tech/2020/04/heroku-slug-size):
/node_modules
(NOTE: the node_modules path for your project may be different than what's shown here)
Compiled slug size:
Before: 525Mb
After: 258Mb 🤯 (a 50% reduction!!!!)

The resolution for this app was:
Use npm to uninstall modules we did not need or use...
The only way we could make the slug change when deploying the app.
we tried before and didn't work:
-Restoring Heroku cache
-Regenerating the slug in a Linux system
-deleting the vendor folder
-deleting files and no Heroku reduction at all
-contacting support
app details:
Ruby 2.3.3
with web pack + webpacker

Related

rails 4.1 can't deploy via capistrano 3

i have very strange error when deploy my rails app with capistrano. sometimes it's deploy and sometimes not.
for example i add something to css (just one string) e.g. .my_some_class{width:10px}
and after that deployment fails. before i add this - it's deploy ok.
i am sure what nothing else changed cause i make experimental commit which contain only one string of css.
my config:
rails 4.1.0
ruby 2.1.1p76
gem 'capistrano-rails', group: :development
gem 'capistrano-rvm', group: :development
gem 'capistrano-bundler', group: :development
end of deploy log:
Tasks: TOP => deploy:assets:precompile
(See full trace by running task with --trace)
The deploy has failed with an error: #<SSHKit::Command::Failed: rake exit status: 137
rake stdout: Nothing written
rake stderr: SafeYAML Warning
----------------
You appear to have an outdated version of libyaml (0.1.4) installed on your system.
Prior to 0.1.6, libyaml is vulnerable to a heap overflow exploit from malicious YAML payloads.
For more info, see:
https://www.ruby-lang.org/en/news/2014/03/29/heap-overflow-in-yaml-uri-escape-parsing-cve-2014-2525/
The easiest thing to do right now is probably to update Psych to the latest version and enable
the 'bundled-libyaml' option, which will install a vendored libyaml with the vulnerability patched:
gem install psych -- --enable-bundled-libyaml
I, [2014-04-30T09:42:41.121037 #12193] INFO -- : Writing /var/www/default/releases/20140430134522/public/assets/Thumbs-82e32ea0cc1ce375db2805ceadd707ef.db
I, [2014-04-30T09:42:41.123108 #12193] INFO -- : Writing /var/www/default/releases/20140430134522/public/assets/agency_no_logo-a8544e60b8a38abeb431c2eb5089f7c6.png
I, [2014-04-30T09:42:41.461121 #12193] INFO -- : Writing /var/www/default/releases/20140430134522/public/assets/swipebox/img/loader-a66dde050b0b2447862919f2c4c37eda.gif
bash: line 1: 12193 Killed ( RAILS_ENV=production ~/.rvm/bin/rvm default do bundle exec rake assets:precompile )
As was written above, probably, you have not enough RAM.
I solved problem by adding SWAP file on my Ubuntu 14.04 server:
Under the root:
dd if=/dev/zero of=/swapfile bs=1024 count=512k
mkswap /swapfile
swapon /swapfile
Add next line to /etc/fstab:
/swapfile none swap sw 0 0
and:
echo 0 > /proc/sys/vm/swappiness
sudo chown root:root /swapfile
sudo chmod 0600 /swapfile
check SWAP(maybe need reloading):
swapon -s
— How To Add Swap on Ubuntu 14.04 # Digital Ocean Community
if someone have same error - problem was on hosting. server have not enough ram(512mb), and process of compilation css/js was killed every time. than we change rate plan, which has 1gb ram, all deployed successfully. SO if you assets precompile failed - just try to add some resources to server.
P.S. similar problem i found here Capistrano deploy - assets precompile error
Try rebooting the server.Worked for me.
You appear to have an outdated version of libyaml
Update libyaml.

libjpeg with image_optim gem on Heroku Cedar Stack (Ruby buildpack)

I'm trying to get the image_optim gem to work with a Rails 4 app on Heroku's Cedar stack to optimize png/gif/jpeg images.
I threw the required binaries (advpng, gifsicle, jpegoptim, jpegtran, optipng, pngcrush, pngout) into /bin and set ENV['PATH'] = "#{Rails.root}/bin:#{ENV['PATH']}" in an initializer. This works for gifs and pngs, but I'm running into problems with jpegs.
In the rails console I get:
irb(main):001:0> `jpegoptim`
jpegoptim: error while loading shared libraries: libjpeg.so.8: cannot open shared object file: No such file or directory
How should I go about installing libjpeg on Heroku?
The correct way to deploy binaries to Heroku is to use buildpacks:
https://devcenter.heroku.com/articles/buildpacks
Use this one if you want to package up multiple binaries:
https://github.com/ddollar/heroku-buildpack-multi
There are a number of buildpacks already created for various binaries, but none for the ones you have listed.
I have created a gem that allows you to run image_optim on heroku:
https://github.com/mooktakim/image_optim_bin
You're probably looking for this: Heroku Image-Optim Buildpack
Or this: https://github.com/bobbus/image-optim-buildpack

Bundler and hidden gems

I have an interesting error when installing gems directly from github (:git => 'whatever').
Firstly, when I remove all gems and run bundle install command, I get the following:
Installing gem1
Installing gem2
Using gem3 (the one from github)
Then when I want to check what I've got I see the following by using gem list:
gem1 (x.x.x)
gem2 (y.y.y)
No gem3... now, looking closer to the file system, I see the following:
ls -l ~/.rvm/gems/ruby-1.9.3-p125/gems
gem1
gem2
So where is gem 3? Not where I'm expecting it to be:
ls -l ~/.rvm/gems/ruby-1.9.3-p125/bundler/gems
gem3-213213213
So it goes under bundler/gems and is not visible to gem list... and by Capistrano deploy, which gives me following:
git://github.com/author/gem3.git (at master) is not checked out. Please run `bundle install`
I'm more worried about Capistrano unable to deploy... Anyone has any clues?
Bundler gets its gems from various sources on your system. As long as they are the correct version, it will pull them in.
When deploying, it has more strict/conservative behavior.
From bundle help install, in the section about Deployment Mode, which is used when the --deployment flag is specified:
Gems are installed to vendor/bundle not your default system loca-
tion
In development, it's convenient to share the gems used in your
application with other applications and other scripts run on the
system.
In deployment, isolation is a more important default. In addition,
the user deploying the application may not have permission to
install gems to the system, or the web server may not have permis-
sion to read them.
As a result, bundle install --deployment installs gems to the ven-
dor/bundle directory in the application. This may be overridden
using the --path option.

exclude files in gems from heroku slugs (using .slugignore, heroku)

A rails 3.0.x project I am working with uses a gem that contains a large amount of test data that isn't needed for the heroku deployment. I would like to exclude this from the heroku slug, as it adds a few dozen megs to the slug (and has pushed us past the 100mb size limit several times, our slug is large for other reasons.)
I've tried doing this using the .slugignore mechanism, but I can't find a way to have it exclude files in gems as opposed to files in the app. This is a rails 3.0.x application running on the bamboo stack, but I would upgrade to rails 3.1 and/or the cedar stack if there was a workaround / procedure in those versions.
Other suggestions about fixing this that aren't 'make the huge gem smaller' are also great and extremely welcome.
On cedar stack the gems are installed into your vendor folder (the actual path is something like vendor/bundle/ruby/1.9.1/gems/), so I believe you can slugignore the subpaths you need, though I haven't tried it.
In general use case this will not work because .slugignore matched files are being deleted before installing gem files. See the following bit of heroku deployment output:
-----> Deleting 2 files matching .slugignore patterns.
-----> Ruby/Rails app detected
-----> Installing dependencies using Bundler version 1.3.0.pre.5
Running: bundle install --without development:test --path vendor/bundle --binstubs vendor/bundle/bin --deployment
Using rake (10.0.3)
Using Platform (0.4.0)
Using open4 (1.3.0)
...

Using Hunspell With Heroku

I'm building a Rails app that uses Hunspell and the hunspell-ffi gem so that Ruby can interface with it. I'm deploying the app to heroku, but unfortunately it needs Hunspell to be installed on the server in order for the gem to work.
Is there any way for me to install Hunspell on Heroku? Or am I going to have to migrate to EC2?
Thanks in advance :)
You need to build the required Hunspell library and include it in your Heroku project directly.
Heroku runs on 64-bit Ubuntu therefore the binary has to be compiled under that system. The best approach is to simply use Heroku's Vulcan build server to compile on a Heroku instance.
Compiling for Heroku
gem install vulcan
vulcan create vulcan-compile-me last argument is your own app name.
Download Hunspell source
Extract
vulcan build -v -s ./hunspell-1.3.2 Tells Vulcan to build it and downloads the finished product automatically to /tmp/hunspell..
The build server requires the cloudant add-on, this is installed automatically but you have to make sure to have a verified (credit card added) Heroku account. If you get errors in step six of no build output then do heroku addons:add cloudant --app vulcan-compile-me
Adding to Your Project
Extract the Heroku Vulcan build tar from /tmp
Copy the entire lib folder to vendor/hunspell in your project root directory
Tell Heroku where to look for libraries: heroku config:add LD_LIBRARY_PATH=vendor/hunspell/lib.
Install Dictionaries
Download some dictionaries from Open Office and add them to your project. A good location is a folder called dictionaries at root level. This path is then referenced when initializing Hunspell in Ruby.
http://extensions.services.openoffice.org/dictionary
ftp://sunsite.informatik.rwth-aachen.de/pub/mirror/OpenOffice/contrib/
Using
Install your favorite Hunspell gem, I use hunspell-ffi. There is a newer gem for Hunspell but I prefer the previous FFI gem. To use initialize the Hunspell object with your dictionaries folder path and language (language match the dictionary file name).
dict = Hunspell.new("dictionaries", "en_US")
if dict.check('caribean') == false
suggestions = dict.suggest('caribean')
if (suggestions.size)
correction = suggestions.first # returns 'caribbean'
end
end
Vendoring for More Complex Projects
You can also vendor the library into your project by putting the tar built by the Vulcan server in the first step into a public accessible server such as Google Storage and then changing the Heroku build pack to download the tar on each instance startup.
heroku config:set BUILDPACK_URL=https://github.com/peterkeen/heroku-buildpack-vendorbinaries.git
The vendor build pack looks for a .vendor_urls file at the root level with HTTP links to the tar balls to install (needs to end in a new line to work).
http://commondatastorage.googleapis.com/developer.you.com/hunspell-heroku-1.3.tgz
Vendoring unpacks the tar into the root folder so the lib path for the Heroku settings would then just be "lib". heroku config:add LD_LIBRARY_PATH=lib
Unless I am mistaken or something has changed (I cannot find any evidence of this), you cannot install external native libraries on Heroku. If the library is not already installed (this is the case, I think, for ImageMagick, and perhaps others), you will not be able to use the gem.
Checkout this url: http://gems-summary.heroku.com/2011-07-19
It's freaking amazing how much support Heroku has for the gem community. So all you need to to is add the gem to your bundle since Hunspell is on rubygems, bundle install, and then deploy.
Gemfile
source 'http://rubygems.org'
gem 'rails', '3.0.5'
gem 'hunspell'
Then add to git:
git add .
git commit -m 'added hunspell'
Then bundle:
bundle
And deploy:
git push heroku
With Bundler, you should be able to install any gem. According to http://devcenter.heroku.com/articles/how-do-i-install-gems-for-my-app, "Almost any gem - even those with native dependencies - can be installed using Bundler. If there’s a specific gem that won’t install on Heroku, please submit a support ticket."
AFAIK, when your app is spun up, gems in the Gemfile are installed on-the-fly to the server your app is spun up to.
The Aspen stack has pre-installed gems, but you still should be able to add gems not pre-installed.
The bamboo stack has no pre-installed gems, so all gem dependencies must be declared explicitly. I believe that is the same for the Celadon stack.

Resources