I am trying to use the HTTP_IF_MODIFIED_SINCE header in my app to determine if resources are stale/fresh and render 200/304 in those cases.
In my dev environment everything works fine but I can't for the life of me get it to work in production.
I am using Passenger 3.0.11 and Nginx 1.0.13.
As you see below, I tried proxy_pass_header, proxy_set_header, passenger_pass_header and passenger_set_cgi_param. The last one actually sets a HTTP_IF_MODIFIED_SINCE header but it is empty...
Any help/ideas would be greatly appreciated!
My config:
server {
listen 80 default_server;
root /home/rails/myapp/current/public;
passenger_enabled on;
charset utf-8;
proxy_pass_header If-Modified-Since;
proxy_set_header If-Modified-Since $http_if_modified_since;
passenger_pass_header If-Modified-Since;
passenger_set_cgi_param HTTP_IF_MODIFIED_SINCE $http_if_modified_since;
if (-f $document_root/system/maintenance.html) {
rewrite ^(.*)$ /system/maintenance.html break;
}
location ~ \.(aspx|jsp|cgi)$ {
return 410;
}
location ~ ^/(assets)/ {
# http://guides.rubyonrails.org/asset_pipeline.html#server-configuration
# gzip_static on;
expires 1y;
add_header Cache-Control public;
add_header Last-Modified "";
add_header ETag "";
break;
}
}
to get it working with non-standard headers, containing underscores, do this inside your http or server block in the nginx.conf file:
underscores_in_headers on;
ignore_invalid_headers off;
and in the server block:
proxy_pass_header HTTP_IF_MODIFIED_SINCE
This can be useful if you have legacy HTTP headers which you need to deal with, and which contain underscores.
This was a user error after all. I sent the header to the app in the wrong format (IF_MODIFIED_SINCE instead of If-Modified-Since). After fixing that, it worked without any of the extra directives.
Related
I use nginx, with passenger, to serve a Rails API. In preparation for our 1.0 launch, I would like to version this API using the method described in this answer.
I have a version number that is pulled from a specification, along with capistrano recipes set up to create the following directory structure:
/var/www/foamfactory.io/dev.api.foamfactory.io
- v0
- current (points to v0.0)
- v0.0
- current (points to v0.0.15)
- v0.0.15 (this is the capistrano directory)
- current (points to latest release under releases/)
- releases
- 20210308162204 (rails directory)
Now, my nginx setup was as follows, in order to rewrite, for example, dev.api.foamfactory.io/version to dev.api.foamfactory.io/current/current/current/current/current/public (I'm working on getting this basic example working before I struggle with, e.g dev.api.foamfactory.io/v0/v0.0/version.
server {
rewrite_log on;
server_name dev.api.foamfactory.io;
access_log /var/log/nginx/dev.api.foamfactory.io-access.log;
error_log /var/log/nginx/dev.api.foamfactory.io-error.log notice;
passenger_enabled on;
passenger_ruby /home/www-data/.rvm/rubies/default/bin/ruby;
passenger_sticky_sessions on;
passenger_app_env development;
passenger_env_var SECRET_KEY_BASE redacted;
location / {
if ($request_method = 'GET') {
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range';
add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range';
}
}
rewrite ^/(.*)$ /current/current/current/current/public/$1 last;
rewrite ^/v([0-9]+)/(.*)$ /v$1/current/current/current/public/$2 last;
rewrite ^/v([0-9]+).([0-9]+)/(.*)$ /v$1/v$1.$2/current/current/public/$3 last;
rewrite ^/v([0-9]+).([0-9]+).([0-9]+)/(.*)$ /v$1/v$1.$2/v$1.$2.$3/current/public/$4 last;
}
I modified this first rewrite statement to be a location/ statement:
server {
rewrite_log on;
server_name dev.api.foamfactory.io;
access_log /var/log/nginx/dev.api.foamfactory.io-access.log;
error_log /var/log/nginx/dev.api.foamfactory.io-error.log notice;
passenger_enabled on;
passenger_ruby /home/www-data/.rvm/rubies/default/bin/ruby;
passenger_sticky_sessions on;
passenger_app_env development;
passenger_env_var SECRET_KEY_BASE redacted;
location / {
if ($request_method = 'GET') {
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range';
add_header 'Access-Control-Expose-Headers' 'Content-Length,Content-Range';
}
}
location ~ ^/(.*)$ {
rewrite ^/(.*)$ /current/current/current/current/public/$1 break;
root /var/www/foamfactory.io/dev.api.foamfactory.io/current/current/current/current/public;
}
}
I can now access the Rails app, BUT, when I try to access an endpoint (e.g. /version, as in dev.api.foamfactory.io/version, Rails reports the following error:
ActionController::RoutingError: No route matches [GET] \"/current/current/current/current/public/version\"
My suspicion is that it has something to do with the relative_url_root configuration variable inside of the Rails configuration, so inside of environments/development.rb, I modified the configuration to include:
config.relative_url_root = '/current/current/current/current/public'
However, this did not alleviate the error.
I'm wondering if someone might be able to help me with what I need to do to get this working. I feel like I'm not even sure if it's an nginx configuration problem or a Rails configuration problem at this point.
I have an angularJS web app running in a Nginx server that sends request to a Rails API running in a Puma server. I have integrated Sidekiq 5.2.8 and everything works great but the Sidekiq web interface.
In my Nginx config file, I have a rule to pass request to the API. Please find the whole nginx.conf document:
events {
worker_connections 1024;
}
http {
upstream api.development {
# Path to Puma SOCK file, as defined previously
server unix:/tmp/puma.sock fail_timeout=0;
}
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
# set client body size to 10M #
client_max_body_size 10M;
gzip on;
server {
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;
#charset koi8-r;
#access_log logs/host.access.log main;
root /Users/Rober/Projects/domain/dev/domain/app;
index index.html index.htm;
try_files $uri $uri/ /index.html =404;
# Proxy requests to the backoffice Rails API
location /api {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
#proxy_set_header X-Forwarded-Proto https;
proxy_redirect off;
rewrite ^/api(.*) /$1 break;
proxy_pass http://api.development;
}
# Rule to proxy the sidekiq web UI
location /sidekiq {
proxy_pass http://api.development;
}
# Expire rules for static content
# RCM: WPO
# Images
location ~* \.(?:jpg|jpeg|gif|png|ico|cur|gz|svg|svgz|mp4|ogg|ogv|webm|htc)$ {
root /Users/Rober/Projects/domain/dev/domain/app;
expires 1w;
add_header Cache-Control "public";
}
# This rule is the root cause of the problems with the sidekiq css
# I have commented it for testing purposes
# CSS and Javascript
#location ~* \.(?:css|js)$ {
# root /Users/Rober/Projects/domain/dev/domain/app;
# expires 1w;
# add_header Cache-Control "public";
#}
# I have replaced the previous location above for this as suggested by #Beena Shetty.
location ~* \.(?:css|js)$ {
add_header X-debug-message "Into the location css" always;
if ($uri !~* "^/sidekiq/\w*(.*)+$") {
add_header X-debug-message "Into the location css if" always;
root /Users/Rober/Projects/domain/dev/domain/app;
expires 1w;
add_header Cache-Control "public";
}
}
# cache.appcache, your document html and data
location ~* \.(?:manifest|appcache|html?|xml|json)$ {
root /Users/Rober/Projects/domain/dev/domain/app;
expires -1;
}
}
include servers/*;
}
In Rails:
routes:
require 'sidekiq/web'
mount Sidekiq::Web => '/sidekiq'
I have included next rule in Nginx config file and now when I request http://localhost/sidekiq I can see the web interface and navigate, but still cannot see the styles.
location /sidekiq {
proxy_pass http://api.development;
}
See screenshot.
The dev tools shows that when I load sideqik is trying to get bootstrap.css and some other css and javascript in http://localhost/sidekiq/stylesheets/bootstrap.css
What am I missing?
UPDATE:
I have found out the root cause of the problem in my nginx.conf. I have next rule that set a cache expiration time for performance purposes. If I comment this code, everything works. But how can I have both things living together?
CSS and Javascript
location ~* \.(?:css|js)$ {
root /Users/Rober/Projects/domain/dev/domain/app;
expires 1w;
add_header Cache-Control "public";
}
UPDATE 2: Just in case the problem comes from another point, I have included my whole nginx conf.
Now, with the provided config, the expiration rules in my web app are still working, but the css in the sidekiq webapp do not.
I have included two headers as debug. One when the server is accessing the location rule and the second when the server is accessing inside the if condition. When I request my home page with localhost and I check the request for my own css, such as app.css, I can see the header X-debug-message: Into the location css if, which is right.
If I request sidekiq with localhost/sidekiq I still get 404 error for css, let´s say http://localhost/sidekiq/stylesheets/bootstrap.css and I can see the header X-debug-message: Into the location css.
Current conclusions:
As soon as I include the location ~* .(?:css|js)$ rule, sidekiq css stops working. Even if the rule is empty, like:
location ~* \.(?:css|js)$ {
}
As soon as I delete or comment the whole rule, the sidekiq css works perfectly, but unfortunately this is not compatible with the expires rules that we need to include for performance purposes.
Try this:
location ~* \.(?:css|js)$ {
if ($uri !~* "^/sidekiq/\w*(.*)+$"){
root /Users/Rober/Projects/domain/dev/yanpy/app;
expires 1w;
add_header Cache-Control "public";
}
}
I wasn't able to find a fix for this, so I hacked it with the following method: I copied sidekiq's assets to the public folder and it started working ( since they're referenced by the UI ).
|- images
|-- favicon.io
|-- logo.png
|-- status.png
|- javascripts
|-- application.js
|-- dashboard.js
|- stylesheets
|-- application.css
|-- application-rtl.css
|-- application-dark.css
|-- application-rtl.min.css
|-- bootstrap.css
Mainly, these files: https://github.com/mperham/sidekiq/tree/master/web/assets
I have a couple of questions regarding my nginx configuration as it pertains to serving webp files as well as using named locations with try_files.
Current config:
server {
listen 80;
server_name assets.manager manager;
passenger_app_env production;
passenger_ruby /home/web-server/.rvm/gems/ruby-2.2.1#manager/wrappers/ruby;
passenger_enabled on;
error_log /home/web-server/web-applications/manager/current/log/nginx-error.log;
root /home/web-server/web-applications/manager/current/public;
satisfy any;
allow 127.0.0.1;
allow 192.168.0.0/24;
deny all;
location ~ ^/assets/ {
gzip_static on;
expires max;
add_header Cache-Control public;
add_header Last-Modified "";
add_header ETag "";
}
location ~* .+\.(?:png|jpe?g|gif)$ {
if ($webp_suffix != "") {
add_header Vary Accept;
}
try_files $uri$webp_suffix $uri =404;
}
}
As it stands, nginx is not serving the webp files. If I were to place add_header X-Webp-Uri "$uri$webp_suffix"; inside the first location block, the header is added. If I were to put that in the second png/jpeg matching location block the header doesn't get set. It was my understanding that regular expression location blocks are sequential (i.e., it doesn't stop matching at the 1st match).
1) I want to serve webp images if present (my attempt at this is the 2nd location block). Would having a nested location block help in this circumstance? I want to set gzip_static, expires, etc. for ALL files in /assets/, but I only want to serve webp version if they exist for certain file extensions within /assets/.
2) On another topic, I want to serve static .html files if present. To do this (after looking up tutorials on the web) I need a combination of try_files and a named location block that points to an upstream application (Rails). However I can't seem to find out how to declare the upstream block if I'm using Passenger (installed using passenger-install-nginx-module). The only configurations I can find for a Passenger/Nginx setup is to use passenger_enabled on;
EDIT: I found a sample configuration; here's an example (this ignores problem #1):
server {
listen 80;
server_name assets.staging.pos staging.pos;
passenger_app_env staging;
passenger_ruby /home/vagrant/.rvm/gems/ruby-2.2.1#pos/wrappers/ruby;
passenger_enabled on;
error_log /home/vagrant/rails/staging.pos/log/nginx-error.log;
root /home/vagrant/rails/staging.pos/public;
try_files $uri /cache/$uri /cache/$uri.html #app;
location #app {
proxy_set_header X-Forwarded-Proto http;
}
location ~* ^/images/.+\.(png|jpe?g)$ {
if ($webp_suffix != "") {
add_header Vary Accept;
}
try_files $uri$webp_suffix $uri =404;
}
}
EDIT 2: I've discovered that nginx completely wipes out any instruction outside of the if block!
This will result in only the Vary Accept header being set:
server {
location ~* ^/images/.+\.(png|jpe?g)$ {
add_header X-Whatever "Yo";
if ($webp_suffix != "") {
add_header Vary Accept;
}
}
}
This will result in both headers being set:
server {
location ~* ^/images/.+\.(png|jpe?g)$ {
if ($webp_suffix != "") {
add_header Vary Accept;
add_header X-Whatever "Yo";
}
}
}
EDIT 3: So now it's even more befuddling. It's like any prior add_header that isn't within the last code block (location or if statement) is completely ignored. i.e., with the following, only the X-Whatever header is set:
location ~ ^/assets/ {
gzip_static on;
expires max;
add_header Cache-Control public; # Ignored?!
add_header Last-Modified ""; # Ignored?!
add_header ETag ""; # Ignored?!
location ~* ^/assets/.+\.(?:png|gif|jpe?g)$ {
add_header X-Something "$uri$webp_suffix"; # Not ignored!
}
}
I had to remove the Vary Accept header condition and simply always apply it. I don't know if that's good or bad but I don't have any other choice considering it just removes every other header I've applied! I also had to move the webp location block above the assets block and duplicate code which sucks.
Working configuration:
server {
listen 80;
server_name assets.staging.pos staging.pos;
passenger_app_env staging;
passenger_ruby /home/vagrant/.rvm/gems/ruby-2.2.1#pos/wrappers/ruby;
passenger_enabled on;
error_log /home/vagrant/rails/staging.pos/log/nginx-error.log;
root /home/vagrant/rails/staging.pos/public;
# Try to return cached responses without hitting the app
try_files $uri /cache/$uri /cache/$uri.html #app;
location #app {
proxy_set_header X-Forwarded-Proto http;
}
# Return webp images when possible
location ~* ^/assets/.+\.(?:png|gif|jpe?g)$ {
expires max;
add_header Cache-Control public;
add_header Vary Accept;
add_header Last-Modified "";
add_header ETag "";
try_files $uri$webp_suffix $uri =404;
}
# Regular asset headers
location ~ ^/assets/ {
gzip_static on;
expires max;
add_header Cache-Control public;
add_header Last-Modified "";
add_header ETag "";
}
}
I've a rails app that serves large static files to registered users. I was able to implement it by following the excellent guide here: Protected downloads with nginx, Rails 3.0, and #send_file. The downloads and everything else is working great, but there is just this problem - The Content-Length header isn't being sent.
It's okay for small files, but it gets really frustrating when large files are downloaded, since download managers and browsers don't show any progress. How can I fix this? Do I have to add something to my nginx configuration or do I have to pass along some other option to the send_file method in my rails controller? I have been searching online for quite some time but have been unsuccessful. Please Help! Thanks!
Here's my nginx.conf:
upstream unicorn {
server unix:/tmp/unicorn.awesomeapp.sock fail_timeout=0;
}
server {
listen 80 default_server deferred;
# server_name example.com;
root /home/deploy/apps/awesomeapp/current/public;
location ~ /downloads/(.*) {
internal;
alias /home/deploy/uploads/$1;
}
location ^~ /assets/ {
gzip_static on;
expires max;
add_header Cache-Control public;
}
try_files $uri/index.html $uri #unicorn;
location #unicorn {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_set_header X-Sendfile-Type X-Accel-Redirect;
proxy_set_header X-Accel-Mapping /downloads/=/home/deploy/uploads/;
proxy_pass http://unicorn;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 20M;
keepalive_timeout 10;
}
Okay, here's something. I don't know if it's the right way or not but I was able to fix the issue by manually sending the Content-Length Header from my Rails Controller. Here's what I'm doing:
def download
#file = Attachment.find(params[:id])
response.headers['Content-Length'] = #file.size.to_s
send_file(#file.path, x_sendfile: true)
end
nginx should be automatically able to set the header. There must be something that I'm missing; but until I find a 'proper' solution, I guess this will have to do.
P.S: The Header needs to be a string to work properly with some webservers, hence the .to_s
I upload images to my App using Ajax and an Iframe. In Development everything works like a charm. But in production Nginx suddenly raises a 404 error. When I look into the log, the request never hits the Rails app. So I guess it has something to do with my Nginx configuration (maybe gzip compression).
The failing requests is send to "/images.js".
Any ideas on how to solve this? Google couldn't help me...
My Nginx config:
server {
listen 80;
server_name www.myapp.de;
root /var/www/apps/myapp/current/public; # <--- be sure to point to 'public'!
passenger_enabled on;
rails_env production;
# set the rails expires headers: http://craigjolicoeur.com/blog/setting-static-asset-expires-headers-with-nginx-and-passenger
location ~* \.(ico|css|js|gif|jp?g|png)(\?[0-9]+)?$ {
expires max;
break;
}
gzip on;
gzip_http_version 1.0;
gzip_vary on;
gzip_comp_level 6;
gzip_proxied any;
gzip_types text/plain text/html text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
# make sure gzip does not lose large gzipped js or css files
# see http://blog.leetsoft.com/2007/7/25/nginx-gzip-ssl
gzip_buffers 16 8k;
# this rewrites all the requests to the maintenance.html
# page if it exists in the doc root. This is for capistrano?^?^?s
# disable web task
if (-f $document_root/system/maintenance.html) {
rewrite ^(.*)$ /system/maintenance.html last;
break;
}
# Set the max size for file uploads to 10Mb
client_max_body_size 10M;
error_page 500 502 503 504 /500.html;
location = /500.html {
root /var/www/apps/myapp/current/public;
}
}
nginx will serve the request for /images.js from your root /var/www/apps/myapp/current/public since it matches
location ~* \.(ico|css|js|gif|jp?g|png)(\?[0-9]+)?$ {
expires max;
break;
}
(the break directive only applies to rewrite rules afaik so it can be removed)
If you want to serve /images.js from rails you need to enable rails for that location.
It's worth noting that any .json requests will get caught if you use the regex above. Add a $ to avoid that.
location ~* \.(ico|css|js$|gif|jp?g|png)(\?[0-9]+)?$ {
expires max;
break;
}
For this purpose I use this location directives:
location ~* ^.+\.(jpg|jpeg|gif|png|css|swf|ico|mov|flv|fla|pdf|zip|rar|doc|xls)$
{
expires 12h;
add_header Cache-Control private;
}
location ~* ^/javascripts.+\.js$
{
expires 12h;
add_header Cache-Control private;
}
I ran into a very similar issue, and put format.json instead of format.js - this made a URL that had a .json extension, but allowed me to not modify my Nginx config.