Right way to deploy Rails + Puma + Postgres app to Elastic beanstalk? - ruby-on-rails

I have an Rails 5 API which I am trying to deploy(correctly) on Elastic Beanstalk.
Here is my initial config/puma.rb file which I use:
threads_count = ENV.fetch("RAILS_MAX_THREADS") { 5 }.to_i
threads threads_count, threads_count
# Specifies the `port` that Puma will listen on to receive requests, default is 3000.
port ENV.fetch("PORT") { 3000 }
# Specifies the `environment` that Puma will run in.
environment ENV.fetch("RAILS_ENV") { "development" }
# Allow puma to be restarted by `rails restart` command.
plugin :tmp_restart
I get the following socket error:
2015/11/24 06:44:12 [crit] 2689#0: *4719 connect() to unix:///var/run/puma/my_app.sock failed (2: No such file or directory) while connecting to upstream
To fix this I tried adding below lines and got it to work:
rails_env = ENV['RAILS_ENV'] || "production"
if rails_env == "production"
bind "unix:///var/run/puma/my_app.sock"
pidfile "/var/run/puma/my_app.sock"
end
My real question is, is this the right way to do it? If anyone has done it before can you point me to it? Is there a way to do this via docker containers?

You can deploy your Rails app as a Rails - Puma app to Elastic Beanstalk or Docker as well. The answer will be more general and rather points where to start than provides complete solution.
Ruby - Puma
This can be a quite tricky: If you create new Elastic Beanstalk Environment for Ruby via Console (in Web Browser), it can set Passenger platform by default, instead of Puma platform. And probably you can't change it in console:
To create new environment with Puma, use eb cli. Nice walkthrough here. However, before you run eb create, you have to do one more thing - select platform:
$ eb platform select
It appears you are using Python. Is this correct?
(y/n): n
Select a platform.
1) Go
2) Node.js
3) PHP
4) Python
5) Ruby
6) Tomcat
7) IIS
8) Docker
9) Multi-container Docker
10) GlassFish
11) Java
(default is 1): 5
Select a platform version.
1) Ruby 2.3 (Puma)
2) Ruby 2.2 (Puma)
3) Ruby 2.1 (Puma)
4) Ruby 2.0 (Puma)
5) Ruby 2.3 (Passenger Standalone)
6) Ruby 2.2 (Passenger Standalone)
7) Ruby 2.1 (Passenger Standalone)
8) Ruby 2.0 (Passenger Standalone)
9) Ruby 1.9.3
(default is 1):
If you want to create Elastic Beanstalk's Worker instead of Web Server, run:
$ eb create -t worker
You can use Console (Web Browser) or eb cli (docs) to set other configuration.
Docker
Following post maybe useful how to setup Rails + Puma + Nginx + Docker:
http://codepany.com/blog/rails-5-and-docker-puma-nginx/
This is multicontainer configuration, where Nginx is binded to port 80 and streams request to puma via socket. In your case it would be: "unix:///var/run/puma/my_app.sock"
To upload Dockers, you can use AWS ECR to store Docker images. You have to create Dockerrun.aws.json file (is quite similar to docker-compose.yml file), which you can than deploy via AWS Console (web browser) to your environment.
EDIT
Here is the puma.rb configuration file:
threads_count = ENV.fetch('RAILS_MAX_THREADS') { 5 }
threads threads_count, threads_count
bind "unix:///var/run/puma.sock?umask=0000"
stdout_redirect "/var/log/puma.stdout.log", "/var/log/puma.stderr.log", true
# Specifies the `environment` that Puma will run in.
#
environment ENV.fetch('RAILS_ENV') { 'development' }
# Allow puma to be restarted by `rails restart` command.
plugin :tmp_restart
Some settings may vary, but the point is that I bind there a puma server to unix socket and it connects with NGINX. The NGINX configuration file:
user root;
error_log /var/log/app-nginx-error.log;
pid /var/run/app-nginx.pid;
events {
worker_connections 8096;
multi_accept on;
use epoll;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/app-nginx-access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 10;
upstream appserver {
server unix:///var/run/puma.sock;
}
server {
listen 80 default_server;
root /var/www/public;
client_max_body_size 16m;
location ^~ /assets/ {
gzip_static on;
expires max;
add_header Cache-Control public;
}
try_files $uri/index.html $uri #appserver;
location #appserver {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
proxy_set_header Client-IP $remote_addr;
proxy_pass http://appserver;
}
access_log /var/log/app-nginx-access.log;
error_log /var/log/app-nginx-error.log debug;
error_page 500 502 503 504 /500.html;
}
}
The most important part in NGINX configuration file is:
upstream appserver {
server unix:///var/run/puma.sock;
}

Related

Steps or steps missing when manually starting Rails server

My Rails application is running fine in my local development enviroment (Mac). I pushed the code to a test server, and trying to start it manually, but getting an error. The details are:
OS: Ubuntu 18.04
Rails 5.2.3
Ruby 2.6.3
Puma
Webpacker
Nginx
my /etc/nginx/sites-enabled/myfqdn.com.conf file:
upstream puma {
server unix:///home/test/shared/tmp/sockets/test-puma.sock;
}
server {
listen 80 default_server deferred;
server_name test.xxxxxx.com;
root /home/amptest/public;
access_log /home/test/log/nginx.access.log;
error_log /home/test/log/nginx.error.log info;
location ^~ /assets/ {
gzip_static on;
expires max;
add_header Cache-Control public;
}
try_files $uri/index.html $uri #puma;
location #puma {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://puma;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 10M;
keepalive_timeout 10;
}
I created the directories:
/home/myapp/shared
/home/myapp/shared/tmp
/home/myapp/shared/tmp/sockets
/home/myapp/shared/tmp/pids
my config/puma.rb, has the following:
threads_count = ENV.fetch("RAILS_MAX_THREADS") { 5 }
threads threads_count, threads_count
port ENV.fetch("PORT") { 3000 }
environment ENV.fetch("RAILS_ENV") { "development" }
plugin :tmp_restart
In my .bashrc, I have:
export RAILS_ENV='test'
In my Gemfile, I have:
gem 'puma'
gem 'webpacker'
I ran:
bundle install
With no errors
And successfully did db migration:
rake db:migrate
and seeded the DB:
rake db:seed
Then I did:
bundle exec rake assets:precompile
Which completed properly:
bundle exec rake assets:precompile
2019-09-10 01:36:10 WARN Selenium [DEPRECATION] Selenium::WebDriver::Chrome#driver_path= is deprecated. Use Selenium::WebDriver::Chrome::Service#driver_path= instead.
yarn install v1.17.3
[1/4] Resolving packages...
success Already up-to-date.
Done in 0.76s.
I, [2019-09-10T01:36:13.413669 #3461] INFO -- : Writing /home/myapp/public/assets/express/lib/application-9232ebdb20ad39572e70fb9e29810e63dbb63b58f5f18617c7c2bc8bd28321b5.js
I, [2019-09-10T01:36:13.413941 #3461] INFO -- : Writing /home/myapp/public/assets/express/lib/application-9232ebdb20ad39572e70fb9e29810e63dbb63b58f5f18617c7c2bc8bd28321b5.js.gz
Compiling…
Compiled all packs in /home/test/public/packs-test
But I try to go to test.xxxxxx.com, I get the error "Something Went Wrong".
The nginx error log says:
2019/09/10 05:38:21 [crit] 1192#1192: *1 connect() to unix:///home/test/shared/tmp/sockets/test-puma.sock failed (2: No such file or directory) while connecting to upstream, client: xx.xxx.xxx.xxxx, server: test.xxxxxx.com, request: "GET / HTTP/1.1", upstream: "http://unix:///home/test/shared/tmp/sockets/test-puma.sock:/", host: "test.xxxxxx.com"
2019/09/10 05:38:47 [info] 1192#1192: *3 client closed connection while waiting for request, client: xx.xxx.xxx.xxxx, server: 0.0.0.0:80
So, I am missing a step or more, including what I need to do to make sure Puma is properly started. Any ideas?
This is because the app's rails server is not running. You never ran rails server command.
If you're using this for test/development purposes, consider this:
Start the rails server at port 3001, rails s -b 0.0.0.0 -p 3001
Then in nginx config use rails server URL to Reverse proxy requests:
proxy_pass http://127.0.0.1:3001
If you're using this for production, then consider using Passenger and you can see passenger installation docs here

rails 5 with puma with nginx systemd

right now i have a droplet in DO and experimenting in rails deployment procedures. I have a rails 5 in ubuntu 16 and have done all necessary procedures to be able to deploy my app except that currently stuck in nginx puma and systemd. I've already seen DO's tutorials but the problem is that they use upstart as the script for init. As a complete beginner in nginx and linux systems, i have no idea how to start my app with nginx. the last procedures i made was
install nginx
test nginx, it is running
put some configs on config/puma.rb*
# Change to match your CPU core count
workers 2
# Min and Max threads per worker
threads 1, 6
app_dir = File.expand_path("../..", __FILE__)
shared_dir = "#{app_dir}/shared"
# Default to production
rails_env = ENV['RAILS_ENV'] || "production"
environment rails_env
# Set up socket location
bind "unix://#{shared_dir}/sockets/puma.sock"
# Logging
stdout_redirect "#{shared_dir}/log/puma.stdout.log", "#{shared_dir}/log/puma.stderr.log", true
# Set master PID and state locations
pidfile "#{shared_dir}/pids/puma.pid"
state_path "#{shared_dir}/pids/puma.state"
activate_control_app
on_worker_boot do
require "active_record"
ActiveRecord::Base.connection.disconnect! rescue ActiveRecord::ConnectionNotEstablished
ActiveRecord::Base.establish_connection(YAML.load_file("# {app_dir}/config/database.yml")[rails_env])
end
edited /etc/nginx/sites-available/default*
upstream app {
# Path to Puma SOCK file, as defined previously
server unix:/home/deploy/appname/shared/sockets/puma.sock fail_timeout=0;
}
server {
listen 80;
server_name localhost;
root /home/deploy/appname/public;
try_files $uri/index.html $uri #app;
location #app {
proxy_pass http://app;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 4G;
keepalive_timeout 10;
}
added /etc/systemd/system/puma.service
*base from https://www.digitalocean.com/community/tutorials/how-to-deploy-a-rails-app-with-puma-and-nginx-on-ubuntu-14-04
this is the code of puma.service
[Unit]
Description=Puma HTTP Server
After=network.target
# Uncomment for socket activation (see below)
# Requires=puma.socket
[Service]
Type=simple
User=(myUser)
WorkingDirectory=/home/(myUser)/apps/(appName)
# ExecStart=<WD>/sbin/puma -b tcp://0.0.0.0:9292 -b ssl://0.0.0.0:9293?key=key.pem&cert=cert.pem
# ExecStart=/usr/local/bin/bundle exec --keep-file-descriptors puma -e production
ExecStart=/usr/local/bin/puma -C /home/(myUser)/apps/(appName)/config/puma.rb
Restart=always
[Install]
WantedBy=multi-user.target
im having exit 203, check some things and found out /usr/local/bin/puma doesnt exist in my linux.
Can someone help my setting up a proper systemd service? i really have no experience in this and im not sure what to put at ExecStart
Install puma if it is not already.
Use which puma to find out where puma is installed, then put that path on ExecStart= line.

Puma not creating socket at defined location when started with `rails server`

I'm deploying my Rails app using nginx, puma, and capistrano. It's deployed by a user called deploy and the deploy location is under the home directory (/home/deploy)
I have Puma configured to create a socket under the shared folder that Capistrano symlinks all it's releases to. Correspondingly, nginx is configured to look at that socket as well (see config files below)
However when I start up the Rails / Puma webserver -
cd /home/deploy/my_app/current
SECRET_KEY_BASE=.... DATABASE_PASSWORD=... rails s -e production
I notice that no socket file is created. When I visit the site in my browser and then look at the Nginx error log, it is also complaining about that socket not existing.
2016/07/17 14:26:19 [crit] 26055#26055: *12 connect() to unix:/home/deploy/my_app/shared/tmp/sockets/puma.sock failed (2: No such file or directory) while connecting to upstream, client: XX.YY.XX.YY, server: localhost, request: "GET http://testp4.pospr.waw.pl/testproxy.php HTTP/1.1", upstream: "http://unix:/home/deploy/my_app/shared/tmp/sockets/puma.sock:/500.html", host: "testp4.pospr.waw.pl"
How do I go about getting puma to create that socket?
Thanks!
Puma Config
# config/puma.rb
...
# `shared_dir` is the symlinked `shared/` directory created
# by Capistrano - `/home/deploy/my_app/shared`
# Set up socket location
bind "unix://#{shared_dir}/tmp/sockets/puma.sock"
# Logging
stdout_redirect "#{shared_dir}/log/puma.stdout.log", "#{shared_dir}/log/puma.stderr.log", true
# Set master PID and state locations
pidfile "#{shared_dir}/tmp/pids/puma.pid"
state_path "#{shared_dir}/tmp/pids/puma.state"
activate_control_app
...
Nginx sites config
# /etc/nginx/sites-available/default
upstream app {
# Path to Puma SOCK file
server unix:/home/deploy/my_app/shared/tmp/sockets/puma.sock fail_timeout=0;
}
server {
listen 80;
server_name localhost;
root /home/deploy/my_app/public;
try_files $uri/index.html $uri #app;
location #app {
proxy_pass http://app;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 4G;
keepalive_timeout 10;
}
Are you sure you are running Puma with that configuration? I don't think rails server is the proper way to start Puma in a production environment.
I would use this instead:
RACK_ENV=production bundle exec puma -C config/puma.rb
Once you get this working manually, then use the --daemon flag to keep the server running in the background.
Also, where is shared_dir defined in your config/puma.rb? Perhaps you omitted the part of the file, but if not, make sure you insert the correct value.
I had a similar issue, the reason was in the incorrect value of shared_dir. You need to update with following if you want to work it on deploy:
set :puma_bind,-> { "unix://#{shared_path}/tmp/sockets/puma.sock" }
set :puma_state, -> { "#{shared_path}/tmp/pids/puma.state" }
set :puma_pid, -> { "#{shared_path}/tmp/pids/puma.pid" }
Notice: after this changes you may have a problem with manual runcap production puma:start/stop/restart and you will need to remove -> {.

Nginx can't connect to localhost

i found myself following this tutorial "How To Deploy a Rails App with Puma and Nginx on Ubuntu 14.04" (https://www.digitalocean.com/community/tutorials/how-to-deploy-a-rails-app-with-puma-and-nginx-on-ubuntu-14-04) in order to use ngnix as load balancer.
i followed all the steps and works fine, i suppose, but at the end, when i configured the nginx config file, i can't access into the localhost (0.0.0.0:3000) it shows "Unable to connect". i checked if the process is running and yes...
Here i let some files
/etc/nginx/sites-available/iaw2015.conf
upstream app {
# Path to Puma SOCK file, as defined previously
server unix:///var/www/iaw2015/shared/sockets/puma.sock fail_timeout=0;
}
server {
listen 80;
server_name localhost;
root /var/www/iaw2015/public;
try_files $uri/index.html $uri #app;
location #app {
proxy_pass http://app;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 4G;
keepalive_timeout 10;
}
i ran this command sudo ln -sf /etc/nginx/sites-available/iaw2015.conf /etc/nginx/sites-enabled/iaw2015.conf to create the link.
config/puma.rb (in my project)
# Change to match your CPU core count
workers 2
# Min and Max threads per worker
threads 1, 6
app_dir = File.expand_path("../..", __FILE__)
shared_dir = "#{app_dir}/shared"
# Default to production
rails_env = ENV['RAILS_ENV'] || "production"
environment rails_env
# Set up socket location
bind "unix://#{shared_dir}/sockets/puma.sock"
# Logging
stdout_redirect "#{shared_dir}/log/puma.stdout.log", "#{shared_dir}/log/puma.stderr.log", true
# Set master PID and state locations
pidfile "#{shared_dir}/pids/puma.pid"
state_path "#{shared_dir}/pids/puma.state"
activate_control_app
on_worker_boot do
require "active_record"
ActiveRecord::Base.connection.disconnect! rescue ActiveRecord::ConnectionNotEstablished
ActiveRecord::Base.establish_connection(YAML.load_file("#{app_dir}/config/database.yml")[rails_env])
end
Based on what you posted, nothing is running on port 3000 that you are trying to connect to. Instead, an app server is running on a socket located at:
/var/www/iaw2015/shared/sockets/puma.sock
First, check that the app is running that you can communicate with it through the socket.
Next, try connecting to http://127.0.0.1/ This will connect to port 80, where Nginx is running. It should hopefully proxy to your app and return a result.

Nginx server configuration for Thin, Faye & Redis

/etc/nginx/nginx.conf looks like:
user deploy;
worker_processes 5;
error_log logs/error.log;
events {
worker_connections 1024;
use epoll;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
upstream foreman4000 {
server x.x.x.x:4000;
server x.x.x.x:4001;
server x.x.x.x:4002;
server x.x.x.x:4003;
server x.x.x.x:4004;
}
server {
listen 80;
server_name x.x.x.x; #server IP
access_log /opt/nginx/foreman4000.access.log;
location / {
proxy_pass http://foreman4000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
}
}
}
Here I use gem foreman, which uses upstart to manage all process and start all servers with one command
I created Procfile in the main directory of the project which contains:
redis: redis-server
thin: bundle exec thin start -p $PORT
faye: rackup faye.ru -E production -s thin
Added to Gemfile:
gem 'foreman'
gem 'thin'
gem "foreman-export-daemontools", "~> 0.0.1"
Ran bundle install locally to edit Gemfile.lock
Deployed project on the server.
Started Nginx
deploy#dcards101:/opt/nginx/conf$ sudo /etc/init.d/nginx stop [ OK ]
deploy#dcards101:/opt/nginx/conf$ sudo /etc/init.d/nginx srart [ OK ]
Exported data from Procfile to Upstart
deploy#dcards101:/var/www/cards/current$ rvmsudo foreman export upstart -a cards -u root
Started application
deploy#dcards101:/var/www/cards/current$ rvmsudo start cards
Now everything had to be good but what i see on the server is only
502 Bad Gateway
nginx/1.0.15
Logs say:
2012/07/17 17:22:30 [error] 11593#0: *148 no live upstreams while connecting to upstream, client: x.x.x.x, server: x.x.x.x, request: "GET / HTTP/1.1", upstream: "http://foreman4000/", host: "x.x.x.x"
Please help with anything you can. Server -- Ubuntu 10 LTS.
got the same error solved it this way:
first install nginx_tcp_proxy_module
( I followed this tutorial but changed it to use passenger and thin with nginx)
than add the tcp part to your nginx.conf:
tcp {
upstream websockets {
## node processes
server 12.34.56.78:9292;
check interval=300 rise=2 fall=5 timeout=1000;
}
server {
listen 9200;
server_name domain.org;
tcp_nodelay on;
proxy_pass websockets;
}
}
doesn´t work on port 80 for me
after that I still get empty responses from faye/privat_pub but there was an extremly trivial solution:
RAILS_ENV=production bundle exec rackup private_pub.ru -s thin -E production
look private_pub - Issue #29
Now everything works except chrome how fires 2 times
(and I need an deamon-process for the rackup)
hope it helps you too
I think your problem is that you put your app-server and the faye server in the same upstream!
If I get the method of upstream and foreman right, your first visitor get the app the second faye and so on. ( maybe I`m wrong because I don´t know foreman .. but if foreman shares all available servers to all services, that might be your problem )
I wood say try capistrano instead of foreman .. so you have full control which server starts where .. because at my my host http don`t work for private_pub (because of nginx) so I had to install the nginx_tcp_proxy_module to get the tcp block working in my nginx.conf
or just try server by server via ssh to find the error

Resources