I've a rails app that serves large static files to registered users. I was able to implement it by following the excellent guide here: Protected downloads with nginx, Rails 3.0, and #send_file. The downloads and everything else is working great, but there is just this problem - The Content-Length header isn't being sent.
It's okay for small files, but it gets really frustrating when large files are downloaded, since download managers and browsers don't show any progress. How can I fix this? Do I have to add something to my nginx configuration or do I have to pass along some other option to the send_file method in my rails controller? I have been searching online for quite some time but have been unsuccessful. Please Help! Thanks!
Here's my nginx.conf:
upstream unicorn {
server unix:/tmp/unicorn.awesomeapp.sock fail_timeout=0;
}
server {
listen 80 default_server deferred;
# server_name example.com;
root /home/deploy/apps/awesomeapp/current/public;
location ~ /downloads/(.*) {
internal;
alias /home/deploy/uploads/$1;
}
location ^~ /assets/ {
gzip_static on;
expires max;
add_header Cache-Control public;
}
try_files $uri/index.html $uri #unicorn;
location #unicorn {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_set_header X-Sendfile-Type X-Accel-Redirect;
proxy_set_header X-Accel-Mapping /downloads/=/home/deploy/uploads/;
proxy_pass http://unicorn;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 20M;
keepalive_timeout 10;
}
Okay, here's something. I don't know if it's the right way or not but I was able to fix the issue by manually sending the Content-Length Header from my Rails Controller. Here's what I'm doing:
def download
#file = Attachment.find(params[:id])
response.headers['Content-Length'] = #file.size.to_s
send_file(#file.path, x_sendfile: true)
end
nginx should be automatically able to set the header. There must be something that I'm missing; but until I find a 'proper' solution, I guess this will have to do.
P.S: The Header needs to be a string to work properly with some webservers, hence the .to_s
Related
Been having a problem setting up sendfile for nginx+rails lately. We have one kind of file download that's already handled by nginx, and we wanted to add a second rule to deal with another kind of files at another location, but we have no success so far.
Ruby controller :
def download_file
send_file("/srv/www/myapp/shared/tmp/directory/file.zip")
end
Environment configuration file:
Rails.application.configure do
# ..
config.action_dispatch.x_sendfile_header = 'X-Accel-Redirect' # for nginx
# ..
end
Nginx configuration :
upstream myapp {
server 127.0.0.1:9292;
}
server {
listen 80;
server_name myapp.tld;
client_max_body_size 10M;
root /srv/www/myapp/current/public;
# This first block works perfectly
location /__working_file {
internal;
alias /var/lib/myapp;
}
# This second block does not work at all
location /__new_files {
internal;
alias /srv/www/myapp/shared/tmp/directory;
}
location / {
root /srv/www/myapp/current/public;
try_files $uri #app;
}
location #app {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_set_header X-Sendfile-Type X-Accel-Redirect;
# This first rule works perfectly
proxy_set_header X-Accel-Mapping /var/lib/myapp/=/__working_file/;
# This second rule doesn't
proxy_set_header X-Accel-Mapping /srv/www/myapp/shared/tmp/directory/=/__new_files/;
proxy_pass_header Server;
proxy_read_timeout 300;
proxy_pass http://myapp;
}
}
Result
When accessing the controller action, the send_file command is triggered, then we get a "File not found" in the browser, nothing gets downloaded, and the rails log shows this:
Sent file /srv/www/myapp/shared/tmp/directory/file.zip (0.4ms)
Completed 200 OK in 169ms (ActiveRecord: 37.5ms)
Started GET "/srv/www/myapp/shared/tmp/directory/file.zip" for 109.190.197.126 at 2018-11-20 11:34:44 +0100
ActionController::RoutingError (No route matches [GET] "/srv/www/myapp/shared/tmp/directory/file.zip"):
The file does exist and is readable but nginx can't seem to access it. Any idea?
The problem comes from the way the two X-Accel-Mapping are set. Rack is indeed able to deal with several mappings since the merge of this PR #1187.
However, as of today, this PR has been merged on master but not released yet (2.0.6 is currently the latest release).
The only thing is, the correct way of setting several mappings is by using one single proxy_set_header rule, and separate each mapping with a comma, like this:
proxy_set_header X-Accel-Mapping, /var/lib/myapp/=/__working_file/,/srv/www/myapp/shared/tmp/directory/=/__new_files/
while routing it is not getting . you have to provide proper route to work.
I'm serving two Rails sites using nginx. Rails1 does not use the asset pipeline but Rails2 does. Rails2 also uses a prefix to differentiate it from Rails1. For example:
http://myhost -> Rails1
http://myhost/abc -> Rails2
Both sites are running, however any reference to assets on the Rails2 site are not found.
Here's what my pseudo nginx.conf looks like:
http {
upstream rails1 {
server 127.0.0.1:3000;
}
upstream rails2 {
server 127.0.0.1:3030;
}
server {
location ~ ^/assets/ {
expires max;
add_header Cache-Control public;
access_log off;
}
location /abc {
proxy_pass http://rails2;
}
location / {
proxy_pass http://rails1;
}
}
}
Also, the routes.rb in my Rails 2 app:
Rails2App::Application.routes.draw do
scope '/abc' do
resources :projects
root :to => 'home#index'
end
end
Browsing to http://myhost/abc/ for the Rails2 app, brings up the page with no css, and the following error:
GET http://myhost/assets/application-asdasd.css 404 (Not Found)
I've tried using config.assets.prefix = '/abc' in the production.rb file but it didn't work. I've also tried different variations in the ngnix.conf file to no avail either.
Anyone know what I'm doing wrong, or missing?
UPDATE
I'm not quite sure why, but I was able to get this to (incorrectly) work using an #location instead of an upstream. But I had to move the assets folder from the Rails2 app to the Rails1 app. Not exactly ideal.
Changes to the server section:
location ~ ^/(assets)/ {
expires max;
add_header Cache-Control public;
access_log off;
}
location ~ ^/(abc)/ {
root /rails2/public;
try_files $uri/index.html $uri.html $uri #rails2;
error_page 404 /404.html;
error_page 422 /422.html;
error_page 500 502 503 504 /500.html;
error_page 403 /403.html;
}
location / {
root /rails1/public;
try_files $uri/index.html $uri.html $uri #rails1;
error_page 404 /404.html;
error_page 422 /422.html;
error_page 500 502 503 504 /500.html;
error_page 403 /403.html;
}
location #rails1 {
proxy_pass http://127.0.0.1:3000;
}
location #rails2 {
proxy_pass http://127.0.0.1:3030;
}
From a similar question: https://stackoverflow.com/a/3355589/417872
Have you tried this?
config.action_controller.relative_url_root = '/rails_app'
I would recommend googleing for how to correctly serve a rails app from a subdirectory. That's the real issue you're dealing with, and a quick search returned several pages of useful looking links.
I had a similar problem running Rails 4 on Nginx in production environment. The solution I found was to specify the root path for the asset location in nginx.conf:
location ^~ /assets/ {
root /home/rails/myapp/public;
gzip_static on;
expires max;
add_header Cache-Control public;
}
Hope this helps
I am trying to use the HTTP_IF_MODIFIED_SINCE header in my app to determine if resources are stale/fresh and render 200/304 in those cases.
In my dev environment everything works fine but I can't for the life of me get it to work in production.
I am using Passenger 3.0.11 and Nginx 1.0.13.
As you see below, I tried proxy_pass_header, proxy_set_header, passenger_pass_header and passenger_set_cgi_param. The last one actually sets a HTTP_IF_MODIFIED_SINCE header but it is empty...
Any help/ideas would be greatly appreciated!
My config:
server {
listen 80 default_server;
root /home/rails/myapp/current/public;
passenger_enabled on;
charset utf-8;
proxy_pass_header If-Modified-Since;
proxy_set_header If-Modified-Since $http_if_modified_since;
passenger_pass_header If-Modified-Since;
passenger_set_cgi_param HTTP_IF_MODIFIED_SINCE $http_if_modified_since;
if (-f $document_root/system/maintenance.html) {
rewrite ^(.*)$ /system/maintenance.html break;
}
location ~ \.(aspx|jsp|cgi)$ {
return 410;
}
location ~ ^/(assets)/ {
# http://guides.rubyonrails.org/asset_pipeline.html#server-configuration
# gzip_static on;
expires 1y;
add_header Cache-Control public;
add_header Last-Modified "";
add_header ETag "";
break;
}
}
to get it working with non-standard headers, containing underscores, do this inside your http or server block in the nginx.conf file:
underscores_in_headers on;
ignore_invalid_headers off;
and in the server block:
proxy_pass_header HTTP_IF_MODIFIED_SINCE
This can be useful if you have legacy HTTP headers which you need to deal with, and which contain underscores.
This was a user error after all. I sent the header to the app in the wrong format (IF_MODIFIED_SINCE instead of If-Modified-Since). After fixing that, it worked without any of the extra directives.
This is soo odd, I've been receiving:
ActionController::RoutingError (No route matches "/favicon.ico")
but I have the favicon.ico in my public directory... any ideas how to solve this? Nginx doesn't throw an error at all.
Run
rake assets:precompile
then set
config.serve_static_assets = true
in config\environments\production.rb file.
Then restart your server.
But I think rake assets:precompile is not required.
It seems that nginx doesn't handle your static assets (since this request for static file goes to the ActionController). Check public root in nginx config file nginx.conf.
Here is an example with Capistrano deployments:
server {
listen 80;
root /var/www/my_project/current/public;
}
And do you use a favicon_link_tag helper in your head :) ?
If you want to keep config.serve_static_assets = false, which is recommended if you have nginx or apache, you can tell nginx to statically serve the files directly. This is especially important for performance reasons, as you don't want rails serving these assets.
Below is a sample which also correctly has nginx statically serve the assets directory:
server {
listen 80;
root /var/www/my_project/current/public;
location / {
proxy_pass http://mysite;
proxy_redirect off;
proxy_set_header X_FORWARDED_PROTO https;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size 10m;
client_body_buffer_size 128k;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffer_size 4k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;
}
# static resource routing - both assets folder and favicon.ico
location ~* ^/assets/|favicon.ico {
# Per RFC2616 - 1 year maximum expiry
# http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
expires 1y;
add_header Cache-Control public;
# Some browsers still send conditional-GET requests if there's a
# Last-Modified header or an ETag header even if they haven't
# reached the expiry date sent in the Expires header.
add_header Last-Modified "";
add_header ETag "";
break;
}
}
Make sure that the favicon.ico file isn't empty (byte size > 0). For some reason I had an empty favicon.ico file which was triggering the same error, even though the file did exist.
delete slash sign before favicon.ico and try to use something like:
<link rel="shortcut icon" type="image/png" href="favicon.ico" />
I was facing the same problem when I first clone code from git repository and run with RAILS_ENV=production. Since there was no assets directory in my git repository, I needed to run rake assets:precompile.
Also I was run with rails s, so config.serve_static_assets = true worked. Thanks #Jiemurat
I'm using Nginx as a reverse proxy for Thin instances.
My goal is to set up a Rails (3) app to upload large files and do something with them.
For that, I came across the Nginx Upload and Upload Progress modules.
I was reading, for the most part, this post, but that's specifically wrote thinking in Passenger.
If possible, I'm looking for two possible answers:
1) Information an examples of implementing this stack (with Thin instead of Passenger)
2) Specific Information of how could I rewrite this:
location ^~ /progress {
# report uploads tracked in the 'proxied' zone
upload_progress_json_output;
report_uploads proxied;
}
location #fast_upload_endpoint {
passenger_enabled on;
rails_env development;
}
location / {
rails_env development;
passenger_enabled on;
}
I don't know what is Passenger exclusive, and how to write it for a typical 4 workers / 3 thin instances conf.
Thanks.
First, you should install nginx with the upload module. The nginx config for site:
upstream uploader_cluster {
server unix:/tmp/thin.uploader.0.sock;
server unix:/tmp/thin.uploader.1.sock;
server unix:/tmp/thin.uploader.2.sock;
server unix:/tmp/thin.uploader.3.sock;
server unix:/tmp/thin.uploader.4.sock;
}
server {
listen 80;
server_name ***.com;
charset off;
client_max_body_size 1000m;
access_log /var/www/uploader/log/access.log;
error_log /var/www/uploader/log/error.log;
root /var/www/uploader/public;
index index.html;
location / {
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
if (-f $request_filename/index.html) {
rewrite (.*) $1/index.html break;
}
if (-f $request_filename.html) {
rewrite (.*) $1.html break;
}
if (!-f $request_filename) {
proxy_pass http://uploader_cluster;
break;
}
}
location ~*uploads$ {
if ($request_method = GET) {
proxy_pass http://uploader_cluster;
break;
}
# pass request body to here
upload_pass #upload_photos;
# Store files to this directory
# The directory is hashed, subdirectories 0 1 2 3 4 5 6 7 8 9 should exist
# i.e. make sure to create /vol/uploads/0 /vol/uploads/1 etc.
upload_store /vol/uploads 1;
# set permissions on the uploaded files
upload_store_access user:rw group:rw all:r;
# Set specified fields in request body
# this puts the original filename, new path+filename and content type in the requests params
upload_set_form_field upload[file_name] "$upload_file_name";
upload_set_form_field upload[file_content_type] "$upload_content_type";
upload_set_form_field upload[file_path] "$upload_tmp_path";
upload_aggregate_form_field upload[file_size] "$upload_file_size";
upload_pass_form_field "^fb_sig_user$|^aid$|^batch_id$|^album_name$|^album_visible$|^caption$|^tags$|^complete$";
upload_cleanup 400 404 499 500-505;
}
location #upload_photos {
proxy_pass http://uploader_cluster;
}
}