I want to start rails server in production mode after installing, migrating and running some scripts in order for this script to be attached as pipeline deploy script.
The problem is that the same script doesn't work as service mode.
ubuntu#ip-x-y-z-w:~/backend.rails.com$ sudo vim /etc/init.d/rails-start-backend
#! /bin/sh
# Start/stop the rails server daemon.
#
### BEGIN INIT INFO
# Provides: rails server start
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Start daemon at boot time
# Description: Enable service provided by daemon.
### END INIT INFO
PATH=/bin:/usr/bin:/sbin:/usr/sbin
DESC="rails daemon"
NAME=rails
DAEMON=/home/ubuntu/backend.rails.com/gitlab-ci.sh
PIDFILE=/var/run/rails.pid
test -f $DAEMON || exit 0
. /lib/lsb/init-functions
case "$1" in
start)
log_daemon_msg "Starting rails"
/home/ubuntu/backend.rails.com/gitlab-ci.sh > /home/ubuntu/backend.rails.com/log/start_script.log
start_daemon -p $PIDFILE $DAEMON $EXTRA_OPTS
log_end_msg $?
;;
stop) log_daemon_msg "Stopping rails" "rails"
sudo kill -9 $(sudo lsof -t -i:3000)
killproc -p $PIDFILE $DAEMON
RETVAL=$?
[ $RETVAL -eq 0 ] && [ -e "$PIDFILE" ] && rm -f $PIDFILE
log_end_msg $RETVAL
;;
restart) log_daemon_msg "Restarting " "rails"
$0 stop
$0 start
;;
reload|force-reload) log_daemon_msg "Reloading rails" "rails"
# rails reloads automatically
log_end_msg 0
;;
*) log_action_msg "Usage: /etc/init.d/rails {start|stop|status|restart|reload|force-reload}"
exit 2
;;
esac
exit 0
and thats my gitlab-ci.sh script
cd /home/ubuntu/backend.rails.com
sudo chmod +x gitlab-ci.sh
rm config/master.key
rm config/credentials.yml.enc
echo "credentials"
RAILS_ENV=production EDITOR="mate --wait" rails credentials:edit
export RAILS_ENV=production
export FRONTEND_BASE_URL=https://www.rails.com
echo "bundle install"
bundle install
echo "rails db:migrate"
bundle exec rails db:migrate
echo "rails rake application:initialize"
bundle exec rake application:initialize
echo "kill"
sudo kill -9 $(sudo lsof -t -i:3000)
echo "start"
rails s &
The problem comes when I restarted the sudo service rails-start-backend restart service. It seems that in that context, all bundle, rails and ruby versions and settings are not the same as when I execute the same script manually in ssh.
The errors I get are:
/usr/bin/env: ‘ruby_executable_hooks2.6’: No such file or directory
bundle: not found
here's my PATH when ssh-logged
/home/ubuntu/.rvm/gems/ruby-2.6.5/bin:/home/ubuntu/.rvm/gems/ruby-2.6.5#global/bin:/usr/share/rvm/rubies/ruby-2.6.5/bin:/usr/share/rvm/bin:/home/ubuntu/bin:/home/ubuntu/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
and then when the script is executed as service
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
It works after setting the right paths to the right variables (using 2.6.5 ruby version)
export PATH=/home/ubuntu/.rvm/gems/ruby-2.6.5/bin:/usr/share/rvm/bin:$PATH
export GEM_PATH=/home/ubuntu/.rvm/gems/ruby-2.6.5:/home/ubuntu/.rvm/gems/ruby-2.6.5#global:$GEM_PATH
source .bash_profile
source ~/.rvm/scripts/rvm
I know this is a very common question but I feel like I need a specific answer to help find out where I am going wrong...
Loaded whenever gem to manage cron jobs - it works fine in development but when I loaded the app into AWS I can't seem to it work...
When I SSH into the instance I can run crontab -l and it lists the whenever tasks but it just doesn't seem to actually execute them. I also cant find any log files to read into why it's not firing.
This is what I pulled from the eb activity log..
+++ GEM_ROOT=/opt/rubies/ruby-2.3.6/lib/ruby/gems/2.3.0
++ (( 0 != 0 ))
+ cd /var/app/current
+ su -c 'bundle exec whenever --update-cron'
[write] crontab file updated
+ su -c 'crontab -l'
# Begin Whenever generated tasks for:
/var/app/current/config/schedule.rb at: 2018-01-10 06:08:24 +0000
0 * * * * /bin/bash -l -c 'cd /var/app/current && bundle exec
bin/rails runner -e production '\''Trendi.home'\'' >>
app/views/pages/cron.html.erb 2>&1'
# End Whenever generated tasks for:
/var/app/current/config/schedule.rb at: 2018-01-10 06:08:24 +0000
[2018-01-10T06:08:24.705Z] INFO [15603] - [Application update app-
a3a0-180109_230627#16/AppDeployStage1/AppDeployPostHook] : Completed
activity. Result:
Successfully execute hooks in directory
/opt/elasticbeanstalk/hooks/appdeploy/post.
This is my config file from ebextensions folder
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/01_cron.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
# Using similar syntax as the appdeploy pre hooks that is
managed by AWS
set -xe
EB_SCRIPT_DIR=$(/opt/elasticbeanstalk/bin/get-config container
-k script_dir)
EB_SUPPORT_DIR=$(/opt/elasticbeanstalk/bin/get-config container
-k support_dir)
EB_DEPLOY_DIR=$(/opt/elasticbeanstalk/bin/get-config container
-k app_deploy_dir)
. $EB_SUPPORT_DIR/envvars
. $EB_SCRIPT_DIR/use-app-ruby.sh
cd $EB_DEPLOY_DIR
su -c "bundle exec whenever --update-cron"
su -c "crontab -l"
My Schedule.rb
ENV['RAILS_ENV'] = "production"
set :output, "app/views/pages/cron.html.erb"
every 1.hour, at: ":00"do # 1.minute 1.day 1.week 1.month 1.year
is also supported
runner "Trendi.home", :environment => 'production'
end
And my task that is stored at /lib/
module Trendi
def self.home
#exectuted task code here
end
You could try a simpler command in your crontab and change it to fire every minute. E.g.
/bin/echo "test" >> /home/username/testcron.log
That way you can quickly rule out if it's the cronjob that's the culprit. Keep in mind to use full paths to each command. So in your case you might want to change the "bundle" command to use the full path. You can find the path with the "which" command.
Also, are you sure you are correctly escaping here?
-e production '\''Trendi.home'\''
Wouldn't this be more adequate?
-e production "Trendi.home"
I have my app with Elastic Beanstalk and I need to create a cronjob that run a task in rails
rake "sitemap:generate"
and I wonder if I can do it with Amazon SQS, anyone knows how to do that?
I tried to do a crontab but it doesn't work in the beanstalk...
files:
"/tmp/cron_job.sh":
mode: "000777"
content: |
#!/usr/bin/env bash
*/2 * * * * cd /var/app/current/ && RACK_ENV=production bundle exec rake sitemap:generate
encoding: plain
container_commands:
01_delete_cron_jobs:
command: "crontab -r -u ec2-user || exit 0"
02_add_cron_jobs:
command: "crontab /tmp/cron_job.sh -u ec2-user"
leader_only: true
Is there another way to do a cronjob in Elastic Beanstalk?
Thank you.
You can run a rake task in a crontab in Elastic Beanstalk as follows.
You may need access to configuration that is set in your profile. If you put the following in a shell script like generate_map.sh, it will include the config that your app is using (this may be overkill for your application). Put this script file in the root of your application.
#!/bin/bash
source /etc/profile
cd /var/app/current
rake sitemap:generate
Put the cronjob you want to run in a text file (like file_with_cron_commands.txt) in your project in the .ebextensions directory. For example, if you want your task to run every minute, that file would include the line below. Make sure to leave a blank line at the end of your text file.
* * * * * root cd /var/app/current ; sh generate_map.sh
In your .config file, use a container command to copy it to the appropriate cron tab directory and set the permissions on it.
container_commands:
01_run_my_gen_map_cron:
command: "cat .ebextensions/file_with_cron_commands.txt > /etc/cron.d/my_genmap_cron && chmod 644 /etc/cron.d/my_genmap_cron"
Looks like an old question, but it pops up in google search.
So here is my attempt
You can create a periodic job (like cron jobs) using SQS with your rails app.
Create an "worker tier" env which uses SQS for queueing
in the home folder of your app create a file "cron.yaml"
sample content like
#cron.yaml
version: 1
cron:
— name: "schedule"
url: "/schedule"
schedule: "0 */12 * * *"
also checkout:
https://medium.com/#joelennon/running-cron-jobs-on-amazon-web-services-aws-elastic-beanstalk-a41d91d1c571
the gem "active-elastic-job"
I'm using Capistrano to deploy my application. Application runs on Passenger standalone. When I redeploy the application the Passenger still uses the Gemfile from the the old release because BUNDLE_GEMFILE environment variable has not been updated.
Where I should put the updated path to Gemfile so that Passenger would pick it up on restart?
The server startup command is in monit and I just call monit scripts from Capistrano tasks except for restart where I just touch the restart.txt.
namespace :deploy do
task :stop do
run("sudo /usr/bin/monit stop my_app_#{rails_env}")
end
task :restart do
run("cd #{current_path} && touch tmp/restart.txt")
end
task :start do
run("sudo /usr/bin/monit start my_app_#{rails_env}")
end
The startup command in monit is:
start program = "/bin/su - app_user -l -c 'cd /home/app_user/current && bundle exec passenger start -d -p 8504 -e production --pid-file=/home/app_user/current/tmp/pids/passenger.8504.pid /home/app_user/current'"
I already tried to add the BUNDLE_GEMFILE into the startup command like this:
start program = "/bin/su - app_user -l -c 'cd /home/app_user/current && BUNDLE_GEMFILE=/home/app_user/current/Gemfile bundle exec passenger start -d -p 8504 -e production --pid-file=/home/app_user/current/tmp/pids/passenger.8504.pid /home/app_user/current'"
But it didn't work since the path /home/app_user/current is a symlink to a release path and that release path was picked up instead.
Simple solution.
Define the Gemfile to be used in the server start command. For example:
BUNDLE_GEMFILE=/home/app_user/current/Gemfile bundle exec passenger start -d -p 9999 -e production --pid-file=/home/app_user/current/tmp/pids/passenger.9999.pid /home/app_user/current
The earlier solution (setting the BUNDLE_GEMFILE env variable in .profile) is not good. When you are deploying a new version of your application and there is a new gem in the bundle the migrations etc. will fail because it will still use the Gemfile defined in the env variable.
I have set up a RoR environement on AWS' elastic beanstalk. I am able to ssh into my EC2 instance.
My home directory is /home/ec2-user, which is effectively empty.
If I move up a directory, there is also a /home/webapp directory that i do not have access to.
Is there a way to run a rake command or rails console on my elastic beanstalk instance?
If I type rails console I get Usage: rails new APP_PATH [options]
If I type RAILS_ENV=production bundle exec rails console, I get "Could not locate Gemfile"
For rails, jump to /var/app/current then as #juanpastas said, run RAILS_ENV=production bundle exec rails c
Don't know why, but since EBS runs everything as root, this worked for me:
sudo su
bundle exec rails c production
None of these solutions mentioned here worked for me, so I cooked up a little script that I put in script/aws-console.
You can run it from the /var/app/current directory as root:
eb ssh
cd /var/app/current
sudo script/aws-console
My script can be found as a Gist here.
None of the other answers worked for me so I went looking - this is working for me now on an elastic beanstalk 64bit amazon linux 2016.03 V2.1.2 ruby 2.2 (puma) stack
cd /var/app/current
sudo su
rake rails:update:bin
bundle exec rails console
that returns me the expected console
Loading production environment (Rails 4.2.6)
irb(main):001:0>
For Ruby 2.7:
if you don't need environment variables:
BUNDLE_PATH=/var/app/current/vendor/bundle/ bundle exec rails c
It looks like environment variables are not loaded automatically anymore, which might prevent rails console from starting.
I solved it by creating this .ebextensions file:
# Simply call `sudo /var/app/scripts/rails_c`
commands:
create_script_dir:
command: "mkdir -p /var/app/scripts"
ignoreErrors: true
files:
"/var/app/scripts/export_envvars":
mode: "000755"
owner: root
group: root
content: |
#!/opt/elasticbeanstalk/.rbenv/shims/ruby
if __FILE__ == $0
require 'json'
env_file = '/var/app/scripts/envvars'
env_vars = env_vars = JSON.parse(`/opt/elasticbeanstalk/bin/get-config environment`)
str = ''
env_vars.each do |key, value|
new_key = key.gsub(/\s/, '_')
str << "export #{new_key}=\"#{value}\"\n"
end
File.open(env_file, 'w') { |f| f.write(str) }
end
"/var/app/scripts/rails_c":
mode: "000755"
owner: root
group: root
content: |
. ~/.bashrc
/var/app/scripts/export_envvars
. /var/app/scripts/envvars
cd /var/app/current
/opt/elasticbeanstalk/.rbenv/shims/bundle exec rails c
Create a .ebextension file named setvars.config and add those lines to it
commands:
setvars:
command: /opt/elasticbeanstalk/bin/get-config environment | jq -r 'to_entries | .[] | "export \(.key)=\"\(.value)\""' > /etc/profile.d/sh.local
packages:
yum:
jq: []
Then deploy your code again it should work.
reference: https://aws.amazon.com/ar/premiumsupport/knowledge-center/elastic-beanstalk-env-variables-shell/
For Ruby 2.7:
As someone said, if you don't need env vars, run the following
BUNDLE_PATH=/var/app/current/vendor/bundle/ bundle exec rails c
However, if you need ENV, I recommend doing this as per AWS doc:
https://aws.amazon.com/premiumsupport/knowledge-center/elastic-beanstalk-env-variables-linux2/
tl;dr
On Amazon Linux 2, all environment properties are centralised into a single file called /opt/elasticbeanstalk/deployment/env. No user can access these outside the app. So, they recommend to add some hook scripts after deploy to basically create a copy.
#!/bin/bash
#Create a copy of the environment variable file.
cp /opt/elasticbeanstalk/deployment/env /opt/elasticbeanstalk/deployment/custom_env_var
#Set permissions to the custom_env_var file so this file can be accessed by any user on the instance. You can restrict permissions as per your requirements.
chmod 644 /opt/elasticbeanstalk/deployment/custom_env_var
#Remove duplicate files upon deployment.
rm -f /opt/elasticbeanstalk/deployment/*.bak
If because of some reason you don't want to run as root, do the following to pass env vars from root into new user environment:
sudo -u <user> -E env "PATH=$PATH" bash -c 'cd /var/app/current/ && <wtv you want to run>'
I like to create an eb_console file at the root of my rails app, then chmod u+x it. It contains the following:
ssh -t ec2-user#YOUR_EC2_STATION.compute.amazonaws.com 'cd /var/app/current && bin/rails c'
This way, I just have to run:
./eb_console
like I would have run heroku run bundle exec rails c.
#!/bin/sh
shell_join () {
ruby -r shellwords -e 'puts Shellwords.join(ARGV)' "$#"
}
command_str () {
printf 'set -e; . /etc/profile.d/eb_envvars.sh; . /etc/profile.d/use-app-ruby.sh; set -x; exec %s\n' "$(shell_join "$#")"
}
exec sudo su webapp -c "$(command_str "$#")"
Put above file somewhere in your source code, deploy, eb ssh into the eb instance, cd /var/app/current, and then execute path/to/above/script bin/rails whatever argumeents you usually use.
Reason why I have written above script is:
When using sudo, it drops some environment variables which might actually be needed for your rails app; so manually load the profiles which the Elastic Beanstalk platform provides.
Current Beanstalk ruby platform assumes you run rails application on user webapp, a non-login-able user, so it would be wise to run your command in this user.
For the latest ruby version, please use the following command:
BUNDLE_PATH=/opt/rubies/ruby-2.6.3/lib/ruby/gems/2.6.0/ bundle exec rails c production
Running it with sudo is not needed.
add an eb extension shortcut:
# .ebextensions/irb.config
files:
"/home/ec2-user/irb":
mode: "000777"
owner: root
group: root
content: |
sudo su - -c 'cd /var/app/current; bundle exec rails c'
then:
$ eb ssh
$ ./irb
irb(main):001:0>
None of these were working for me, including the aws-console script. I finally ended up creating a script directory in /var/app/current and then creating a rails file in that directory as outline by this answer on another SO question.
eb ssh myEnv
cd /var/app/current
sudo mkdir script
sudo vim script/rails
Add this to file and save:
echo #!/usr/bin/env ruby
# This command will automatically be run when you run "rails" with Rails 3 gems installed from the root of your application.
APP_PATH = File.expand_path('../../config/application', __FILE__)
require File.expand_path('../../config/boot', __FILE__)
require 'rails/commands'
Then make it executable and run it:
sudo chmod +x script/rails
sudo script/rails console
And it worked.
You have to find the folder with your Gemfile :p.
To do that, I would take a look in you web server config there should be a config that tells you where your app directory is.
Maybe you know where your app is.
But in case you don't know, I would give a try to:
grep -i your_app_name /etc/apache/*
grep -i your_app_name /etc/apache/sites-enabled/*
To search files containing your_app_name in Apache config.
Or if you are using nginx, replace apache above by nginx.
after you find application folder, cd into it and run RAILS_ENV=production bundle exec rails c.
Making sure that your application is configured to run in production in Apache or nginx configuration.