I've deployed my RoR 4 app using Capistrano 2, Unicorn, Nginx.
The problem is that I get 404 on assets(stylesheets, javascripts).
Here's Nginx access log:
89.0.40.233 - - [16/Mar/2014:08:24:26 +0000] "GET /stylesheets/application.css HTTP/1.1" 404 650 "http://host.cloudapp.net/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.149 Safari/537.36"
89.0.40.233 - - [16/Mar/2014:08:24:26 +0000] "GET /javascripts/application.js HTTP/1.1" 404 650 "http://host.cloudapp.net/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.149 Safari/537.36"
My assets are in the app folder:
azureuser#host:~/apps/testify/current/public$ ls -a assets
. application-d65a0eaefe6ca2eef9400045f94ab52b.js
.. application-d65a0eaefe6ca2eef9400045f94ab52b.js.gz
application-71e2591e9586afebf3fb4ff70aaae199.css manifest-a348973e84698f7d898e8021bd6e5388.json
application-71e2591e9586afebf3fb4ff70aaae199.css.gz
My Nginx config:
upstream unicorn {
server unix:/tmp/unicorn.testify.sock fail_timeout=0;
}
server {
listen 80 default deferred;
root /home/azureuser/apps/testify/current/public;
location ^~ /assets/ {
expires max;
add_header Cache-Control public;
}
try_files $uri/index.html $uri #unicorn;
location #unicorn {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://unicorn;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 4G;
keepalive_timeout 10;
}
Where do I start looking?
As it seems from access.log you just hardcoded application.css/.js in your layout. There is no such files in production public folder because of fingerprint name that asset pipeline gives them (look at your example ls output).
You may read about this here.
Fixing your problem really simple. Replace hardcoded links for application.css/.js with this code:
<%= javascript_include_tag "application" %>
<%= stylesheet_link_tag "application" %>
Adding to the answer of Sergey Moiseev
Try fetching the assets directly with the url
/assets/javascripts/application-d65a0eaefe6ca2eef9400045f94ab52b.js.If it doesnt work the your issue is in nginx rather than rails.
Also please check whether you are fetching the file with right fingerprint. In this case check whether its application-d65a0eaefe6ca2eef9400045f94ab52b.js or someother application.js with a different fingerprint. I got a similar issue with multiple servers.
For me, the root path was wrong in my nginx config, and I was experiencing this exact issue. Reference: https://stackoverflow.com/a/25341195/2544629
I would guess Capistrano is not using "production" as the environment name for Unicorn (-E option). "ps aux|grep unicorn" would probably tell you which environment it's using.
Related
I need send POST request to my nginx frontend server which should redirect it to upstream servers.
In details:
send request to http://192.168.0.10/foo/bar/blah and URL in this request should be changed to http://192.168.0.21[22,23]:8080/foo/blah
upstream myapp {
server 192.168.0.21:8080;
server 192.168.0.22:8080;
server 192.168.0.23:8080;
}
server {
listen 80;
server_name localhost;
location /foo/bar/blah/ {
rewrite ^/foo/blah^/ /$1 break;
proxy_pass http://myapp;
}
but in nginx error log I see that my request changed from POST to GET and also seems didn't change URL:
"POST /foo/bar/blah HTTP/1.1" 301 185 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64)
"GET /foo/bar/blah/ HTTP/1.1" 404 117 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64)
How can I keep my POST request and change the URL?
Also about my rewrite rule
rewrite ^/foo/blah^/ /$1 break;
I found a lot of examples for changing URL and all of them looks the same.
And it is really strange for me, how this rewrite rule can change URL from /foo/bar/blah/ to /foo/blah/:
in documentaion says:
http://nginx.org/en/docs/http/ngx_http_rewrite_module.html#rewrite
If the specified regular expression matches a request URI, URI is
changed as specified in the replacement string.
but in my case request URI is /foo/bar/blah/ so regular expression /foo/blah^/ doesn't matches URI, so this rule shouldn't work.
Am I right?
Would be helpful any advises.
UPD:
fixed:
location = /foo/bar/blah {
proxy_pass http://myapp/foo/blah;
}
My guess, you don't need rewrite.
location /foo/bar/ {
proxy_pass http://myapp/;
}
This should remove /foo/bar part from proxied URL.
The asset files for a couple of gems I'm using aren't loading after using Nginx in production mode. I'm pretty certain it has to do with the location blocks in my Nginx config, but I'm not sure what to add so that Nginx will point to where the files are located.
The gems in question are sidekiq and rack-mini-profiler
upstream cable {
server unix:///tmp/cable.sock;
}
server {
listen 80;
server_name 66.207.0.133;
root /home/john/rails/cable/public/assets;
location / {
proxy_pass http://cable;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location ~* \.(css|js|otf|woff|ttf|svg|eot)$ {
root /home/john/rails/cable/public/;
}
}
The error in the Nginx log is:
2016/11/07 21:04:36 [error] 22745#22745: *51175 open() "/home/john/rails/cable/public/sidekiq/javascripts/dashboard.js" failed (2: No such file or directory), client: 69.49.80.136, server: 66.207.0.133, request: "GET /sidekiq/javascripts/dashboard.js HTTP/1.1", host: "66.207.0.133", referrer: "http://66.207.0.133/sidekiq"
Obviously the second location block is redirecting all requests for the needed .js and .css files to the wrong location, but how and to where can I redirect requests to /sidekiq/*.js to the correct files?
You first need to find where the correct files are in your filesystem.
find / -path "*/javascripts/dashboard.js"
A web search reveals that it might be /home/site/homepage_production/shared/bundle/ruby/1.9.1/gems/sidekiq-2.16.1/web/assets/javascripts/dashboard.js.
So, if you gotta serve that from /sidekiq/javascripts/dashboard.js on the web, and provided that all requests within /sidekiq/ are for static assets, then the following should be used:
location ^~ /sidekiq/ {
alias /home/site/homepage_production/shared/bundle/ruby/1.9.1/gems/sidekiq-2.16.1/web/assets/;
}
For more details, see:
http://nginx.org/r/location
http://nginx.org/r/alias
I've deployed a rails app that allows a user to upload a photo and have it display on another page, fairly simple. I tested it in development, the image uploads to the public folder and displays properly. In production and deployed the image uploads to the server but isn't being rendered on the page.
404 errors for only the images I've uploaded, path looks like this:
http://IP-OF-APP/uploads/blog/name-of-image.jpg
I read in another S.O. article mentioning setting a production config serve_static_files as true. This didn't fix the problem.
I thought it might be a server configuration with NGINX not picking up the uploads path here is my /sites-default/nginx.conf file.
upstream app {
server unix: /home/deploy/MYAPPNAME/shared/tmp/sockets/puma.sock fail_timeout=0;
}
server {
listen 80;
server_name IP_ADDRESS_OF_SERVER;
root /home/deploy/MYAPPNAME/public;
location #app {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_pass http://app;
proxy_redirect off;
}
location ~ ^/(assets)/ {
root /home/deploy/MYAPPNAME/shared/public
gzip_static on;
expires max;
add_header Cache-Control public;
}
location ~ ^/uploads/ {
root /home/deploy/MYAPPNAME/shared/public;
expires 24h;
add_header Cache-Control public;
break;
}
location ~ ^/(fonts|system)/favicon.ico/robots.txt {
gzip_static on;
expires max;
add_header Cache-Control public;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 4G;
keepalive_timeout 10;
}
I also thought it might be that the image isn't uploading to the server correctly so I SSH'd into the server and found the image uploaded living in:
home/deploy/MYAPPNAME/shared/public/uploads/blog/name-of-image.jpg
Also found what I assume to be the same image in:
home/deploy/MYAPPNAME/current/public/uploads/blog/name-of-image.jpg
My uploader looks like this:
class BlogUploader < CarrierWave::Uploader::Base
include CarrierWave::MiniMagick
storage :file
def store_dir
'uploads/blog'
end
def extension_whitelist
%w(jpg jpeg gif png)
end
process :resize_to_fit => [825, 825]
#titles are validated to be unique
def filename
"#{model.title}"+".#{file.extension}" if original_filename.present?
end
end
using
Rails 4.2.6
Ruby 2.3.0
Carrierwave 0.11.0
EDIT
All of the other static images, CSS and JS are displaying and rendering properly.
The NGINX error when rending the upload image produces this:
2016/04/29 17:32:34 [error] 4993#0: *23 open() "/home/deploy/MYAPPNAME/shared/public/assets/uploads/blog/name-of-image.jpg" fails (2: no such file or directory), client: *******, server: SERVER_IP, request: "GET /assets/uploads/blog/name-of-image.jpg HTTP/1.1", host: "SERVER_IP"
From your log:
User is requesting:
/assets/uploads/blog/name-of-image.jpg
Nginx is looking for the image in:
/home/deploy/MYAPPNAME/shared/public/assets/uploads/blog/name-of-image.jpg
You confirmed that the image is in:
home/deploy/MYAPPNAME/shared/public/uploads/blog/name-of-image.jpg
Nginx is looking in public/assets/uploads, your file is in public/uploads.
I can't for the life of me stream a mp4 to Chrome with a html5 <video> tag. If I drop the file in public then everything is gravy and works as expected. But if I try to serve it using send_file, pretty much everything imaginable goes wrong. I am using a rails app that is proxied by nginx, with a Video model that has a location attribute that is an absolute path on disk.
At first I tried:
def show
send_file Video.find(params[:id]).location
end
And I was sure I would be basking in the glory that is modern web development. Ha. This plays in both Chrome and Firefox, but neither seek and neither have any idea how long the video is. I poked at the response headers and realized that Content-Type is being sent as application/octet-stream and there is no Content-Length set. Umm... wth?
Okay, I guess I can set those in rails:
def show
video = Video.find(params[:id])
response.headers['Content-Length'] = File.stat(video.location).size
send_file(video.location, type: 'video/mp4')
end
At this point everything works pretty much as expected in Firefox. It knows how long the video is and seeking works as expected. Chrome appears to know how long the video is (doesn't show timestamps, but seek bar looks appropriate) but seeking doesn't work.
Apparently Chrome is pickier than Firefox. It requires that the server respond with a Accept-Ranges header with value bytes and respond to subsequent requests (that happen when the users seeks) with 206 and the appropriate portion of the file.
Okay, so I borrowed some code from here and then I had this:
video = Video.find(params[:id])
file_begin = 0
file_size = File.stat(video.location).size
file_end = file_size - 1
if !request.headers["Range"]
status_code = :ok
else
status_code = :partial_content
match = request.headers['Range'].match(/bytes=(\d+)-(\d*)/)
if match
file_begin = match[1]
file_end = match[2] if match[2] && !match[2].empty?
end
response.header["Content-Range"] = "bytes " + file_begin.to_s + "-" + file_end.to_s + "/" + file_size.to_s
end
response.header["Content-Length"] = (file_end.to_i - file_begin.to_i + 1).to_s
response.header["Accept-Ranges"]= "bytes"
response.header["Content-Transfer-Encoding"] = "binary"
send_file(video.location,
:filename => File.basename(video.location),
:type => 'video/mp4',
:disposition => "inline",
:status => status_code,
:stream => 'true',
:buffer_size => 4096)
Now Chrome attempts to seek, but when you do the video stops playing and never works again until the page reloads. Argh. So I decided to play around with curl to see what was happening and I discovered this:
$ curl --header "Range: bytes=200-400" http://localhost:8080/videos/1/001.mp4
ftypisomisomiso2avc1mp41 �moovlmvhd��#��trak\tkh��
$ curl --header "Range: bytes=1200-1400" http://localhost:8080/videos/1/001.mp4
ftypisomisomiso2avc1mp41 �moovlmvhd��#��trak\tkh��
No matter the byte range request, the data always starts from the beginning of the file. The appropriate amount of bytes is returned (201 bytes in this case), but it's always from the beginning of the file. Apparently nginx respects the Content-Length header but ignores the Content-Range header.
My nginx.conf is untouched default:
user www-data;
worker_processes 4;
pid /run/nginx.pid;
events {
worker_connections 768;
}
http {
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
gzip on;
gzip_disable "msie6";
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
and my app.conf is pretty basic:
upstream unicorn {
server unix:/tmp/unicorn.app.sock fail_timeout=0;
}
server {
listen 80 default deferred;
root /vagrant/public;
try_files $uri/index.html $uri #unicorn;
location #unicorn {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header HOST $http_host;
proxy_redirect off;
proxy_pass http://unicorn;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 4G;
keepalive_timeout 5;
}
First I tried the nginx 1.4.x that comes with Ubuntu 14.04, then tried 1.7.x from a ppa - same results. I even tried apache2 and had exactly the same results.
I would like to reiterate that the video file is not the problem. If I drop it in public then nginx serves it with the appropriate mime types, headers and everything needed for Chrome to work properly.
So my question is a two-parter:
Why doesn't nginx/apache handle all this stuff automagically with send_file (X-Accel-Redirect/X-Sendfile) like it does when the file is served statically from public? Handling this stuff in rails is so backwards.
How the heck can I actually use send_file with nginx (or apache) so that Chrome will be happy and allow seeking?
Update 1
Okay, so I thought I'd try to take the complication of rails out of the picture and just see if I could get nginx to proxy the file correctly. So I spun up a dead-simple nodjs server:
var http = require('http');
http.createServer(function (req, res) {
res.writeHead(200, {
'X-Accel-Redirect': '/path/to/file.mp4'
});
res.end();
}).listen(3000, '127.0.0.1');
console.log('Server running at http://127.0.0.1:3000/');
And chrome is happy as a clam. =/ curl -I even shows that Accept-Ranges: bytes and Content-Type: video/mp4 is being inserted by nginx automagically - as it should be. What could rails be doing that's preventing nginx from doing this?
Update 2
I might be getting closer...
If I have:
def show
video = Video.find(params[:id])
send_file video.location
end
Then I get:
$ curl -I localhost:8080/videos/1/001.mp4
HTTP/1.1 200 OK
Server: nginx/1.7.9
Date: Sun, 18 Jan 2015 12:06:38 GMT
Content-Type: application/octet-stream
Connection: keep-alive
Status: 200 OK
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
Content-Disposition: attachment; filename="001.mp4"
Content-Transfer-Encoding: binary
Cache-Control: private
Set-Cookie: request_method=HEAD; path=/
X-Meta-Request-Version: 0.3.4
X-Request-Id: cd80b6e8-2eaa-4575-8241-d86067527094
X-Runtime: 0.041953
And I have all the problems described above.
But if I have:
def show
video = Video.find(params[:id])
response.headers['X-Accel-Redirect'] = video.location
head :ok
end
Then I get:
$ curl -I localhost:8080/videos/1/001.mp4
HTTP/1.1 200 OK
Server: nginx/1.7.9
Date: Sun, 18 Jan 2015 12:06:02 GMT
Content-Type: text/html
Content-Length: 186884698
Last-Modified: Sun, 18 Jan 2015 03:49:30 GMT
Connection: keep-alive
Cache-Control: max-age=0, private, must-revalidate
Set-Cookie: request_method=HEAD; path=/
ETag: "54bb2d4a-b23a25a"
Accept-Ranges: bytes
And everything works perfectly.
But why? Those should do exactly the same thing. And why doesn't nginx set Content-Type automagically here like it does for the simple nodejs example? I have config.action_dispatch.x_sendfile_header = 'X-Accel-Redirect' set. I have moved it back and forth between application.rb and development.rb with the same results. I guess I never mentioned... this is rails 4.2.0.
Update 3
Now I've changed my unicorn server to listen on port 3000 (since I already changed nginx to listen on 3000 for the nodejs example). Now I can make requests directly to unicorn (since it's listening on a port and not a socket) so I have found that curl -I directly to unicorn shows that no X-Accel-Redirect header is sent and just curling unicorn directly actually sends the file. It's like send_file isn't doing what it's supposed to.
I finally have the answers to my original questions. I didn't think I'd ever get here. All my research had lead to dead-ends, hacky non-solutions and "it just works out of the box" (well, not for me).
Why doesn't nginx/apache handle all this stuff automagically with send_file (X-Accel-Redirect/X-Sendfile) like it does when the file is served statically from public? Handling this stuff in rails is so backwards.
They do, but they have to be configured properly to please Rack::Sendfile (see below). Trying to handle this in rails is a hacky non-solution.
How the heck can I actually use send_file with nginx (or apache) so that Chrome will be happy and allow seeking?
I got desperate enough to start poking around rack source code and that's where I found my answer, in the comments of Rack::Sendfile. They are structured as documentation that you can find at rubydoc.
For whatever reason, Rack::Sendfile requires the front end proxy to send a X-Sendfile-Type header. In the case of nginx it also requires a X-Accel-Mapping header. The documentation also has examples for apache and lighttpd as well.
One would think the rails documentation could link to the Rack::Sendfile documentation since send_file does not work out of the box without additional configuration. Perhaps I'll submit a pull request.
In the end I only needed to add a couple lines to my app.conf:
upstream unicorn {
server unix:/tmp/unicorn.app.sock fail_timeout=0;
}
server {
listen 80 default deferred;
root /vagrant/public;
try_files $uri/index.html $uri #unicorn;
location #unicorn {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header HOST $http_host;
proxy_set_header X-Sendfile-Type X-Accel-Redirect; # ADDITION
proxy_set_header X-Accel-Mapping /=/; # ADDITION
proxy_redirect off;
proxy_pass http://localhost:3000;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 4G;
keepalive_timeout 5;
}
Now my original code works as expected:
def show
send_file(Video.find(params[:id]).location)
end
Edit:
Although this worked initially, it stopped working after I restarted my vagrant box and I had to make further changes:
upstream unicorn {
server unix:/tmp/unicorn.app.sock fail_timeout=0;
}
server {
listen 80 default deferred;
root /vagrant/public;
try_files $uri/index.html $uri #unicorn;
location ~ /files(.*) { # NEW
internal; # NEW
alias $1; # NEW
} # NEW
location #unicorn {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header HOST $http_host;
proxy_set_header X-Sendfile-Type X-Accel-Redirect;
proxy_set_header X-Accel-Mapping /=/files/; # CHANGED
proxy_redirect off;
proxy_pass http://localhost:3000;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 4G;
keepalive_timeout 5;
}
I find this whole thing of mapping one URI to another and then mapping that URI to a location on disk to be totally unnecessary. It's useless for my use case and I'm just mapping one to another and back again. Apache and lighttpd don't require it. But at least it works.
I also added Mime::Type.register('video/mp4', :mp4) to config/initializers/mime_types.rb so the file is served with the correct mime type.
I'm trying to deploy a rails4 (ruby-2.0.0) app to my server. Almost all of my assets are precompiled, and served by nginx.
One js.erb, generates a dynamic html-list, by getting models from my database. This asset can't be precompiled, because it must remain dynamic.
I'm excluding this asset from asset.precompile, and turned on
config.assets.compile = true
to fall back to the asset pipeline, for this one asset.
In my local production env, everthing is working, but on my server (nginx, unicorn) the asset pipeline fall back won't work. I get a simple 404 Error
nginx error log:
2013/09/13 08:54:54 [error] 27442#0: *58 open() "/XXX/current/public/assets/rails_admin/rails_admin_switchable-051203ae1d7aca2c08092e5c92bcdf15.js" failed (2: No such file or directory), client: XXX, server: , request: "GET /assets/rails_admin/rails_admin_switchable-051203ae1d7aca2c08092e5c92bcdf15.js HTTP/1.1", host: "XXX", referrer: "http://XXX/admin"
unicorn and rails don't show any errors.
Any ideas, how I can solve this?
best,
Franz
It looks like your nginx server definition isn't properly integrated with your app server. It should be configured to pass a request that doesn't match a physical file on to the app server.
Here is a standard configuration for a rails app living in /app with nginx via a unicorn/UNIX-socket integration:
upstream app_server {
server unix:/tmp/nginx.socket fail_timeout=0;
}
server {
listen <%= ENV["PORT"] %>;
server_name _;
keepalive_timeout 5;
# path for static files
root /app/public;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://app_server;
}
# Rails error pages
error_page 500 502 503 504 /500.html;
location = /500.html {
root /app/public;
}
}
If your asset pipeline compiles to /app/public/assets you should be good to go.