Nginx 404 (Not Found) only for rails images on AWS - ruby-on-rails

Im testing my app in production mode and running into a weird problem. The image assets aren't loading on the webpage. All the css and javascript are loading fine but any image gets a 404 (Not Found) from nginx. Im pushing my app with docker to AWS services. When I precompile the assets the images are included in the output so they should be there:
I, [2017-08-30T12:10:50.092655 #7] INFO -- : Writing /livingrecipe/public/assets/default-user-d39e051d757a0f45602b6065147cec0a307c6a2a070834f6fe6fc2b0fc914207.jpg
I, [2017-08-30T12:10:50.104846 #7] INFO -- : Writing /livingrecipe/public/assets/default_recipe-be8d509a8e34dfec514c097b119753d4aaadbfd1dd6747e3c868ed454b8a36b3.jpg
I, [2017-08-30T12:10:50.108486 #7] INFO -- : Writing /livingrecipe/public/assets/edit-4e381318428d71d55ed0cbdc32b4ce99db6622140c6e7c751ae0f1bfb40d3963.png
I, [2017-08-30T12:10:50.111606 #7] INFO -- : Writing /livingrecipe/public/assets/favorite-91432751e505915d80cf4e24d7a8d0abd3493d05d2d447c1cebb055d99757ac8.png
I, [2017-08-30T12:10:50.115720 #7] INFO -- : Writing /livingrecipe/public/assets/icon_email-203dcd7bdcf7559d742f76a9d8c21e98694d45b9556b29ab9ce286a7dd37af11.png
I, [2017-08-30T12:10:50.118457 #7] INFO -- : Writing /livingrecipe/public/assets/icon_facebook-0140437ec6cb29bc6ac6f8505f05f11603a430298e48c1ea483f7de390ba34a9.png
I, [2017-08-30T12:10:50.121393 #7] INFO -- : Writing /livingrecipe/public/assets/icon_google-2a98026430dddc53539d283c04a2ad4b50536c93d0becadb03f3f61443e52c9c.png
I, [2017-08-30T12:10:50.125228 #7] INFO -- : Writing /livingrecipe/public/assets/star-half-db15fb9b3561d5c741d8aea9ef4f0957bd9bc51aa1caa6d7a5c316e083c1abd5.png
I, [2017-08-30T12:10:50.127714 #7] INFO -- : Writing /livingrecipe/public/assets/star-off-6aaeebdaab93d594c005d366ce0d94fba02e7a07fd03557dbee8482f04a91c22.png
I, [2017-08-30T12:10:50.130083 #7] INFO -- : Writing /livingrecipe/public/assets/star-on-fd26bf0ea0990cfd808f7540f958eed324b86fc609bf56ec2b3a5612cdfde5f5.png
I, [2017-08-30T12:10:50.132442 #7] INFO -- : Writing /livingrecipe/public/assets/unfavorite-7000a53c81f55cf94c88031a8529f847e5a6abd7460e0e54ae0b581b0cde85a3.png
And here is my default.conf file for the nginx:
upstream PLACEHOLDER_BACKEND_NAME {
# The web app.
server PLACEHOLDER_BACKEND_NAME:PLACEHOLDER_BACKEND_PORT;
}
# Redirect 'www' addresses to the non-www version, and also take care of
# redirects to HTTPS at the same time.
server {
listen 80;
server_name www.PLACEHOLDER_VHOST;
return 301 https://$host$request_uri;
}
server {
# "deferred" reduces the number of formalities between the server and client.
listen 80 default deferred;
server_name PLACEHOLDER_VHOST;
# Static asset path, which is read from the PLACEHOLDER_BACKEND_NAME
# container's VOLUME.
root /PLACEHOLDER_BACKEND_NAME/public;
# Serve static assets.
#
# gzip_static is enabled because the assets are already gzipped to a maximum
# level due to Rail's asset pipeline.
#
# Add headers to set the maximum amount of cache time.
#
# We can do this because the Ruby on Rails asset pipeline md5 hashes all of
# the file names for us. When a file changes, its md5 will change, and the
# cache will be automatically busted. Other frameworks can do this as well.
location ~ ^/assets/ {
gzip_static on;
# Set a maximum cache time period and null out a few headers to address
# certain browsers from occasionally requesting cached content.
expires max;
add_header Cache-Control public;
add_header Last-Modified "";
add_header ETag "";
}
# Ensure timeouts are equal across browsers.
keepalive_timeout 60;
# Disallow access to hidden files and directories.
location ~ /\. {
return 404;
access_log off;
log_not_found off;
}
# Allow optionally writing an index.html file to take precedence over the upstream.
try_files $uri $uri/index.html $uri.html #PLACEHOLDER_BACKEND_NAME;
# Attempt to load the favicon or fall back to status code 204.
location = /favicon.ico {
try_files /favicon.ico = 204;
access_log off;
log_not_found off;
}
# Force SSL connections on agents (browsers) who support this header.
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains;";
# Load the web app back end with proper headers.
location #PLACEHOLDER_BACKEND_NAME {
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Ssl on;
proxy_redirect off;
if ($http_x_forwarded_proto = "http") {
return 301 https://$host$request_uri;
}
proxy_pass http://PLACEHOLDER_BACKEND_NAME;
}
}

I am going to answer from experience, although I had the same problem with Nginx + Django. What saved my situation is giving a user that runs the server (might be www-data, in my case was me) privilege to access images. Yes, it sounds stupid since it has permissions on other files, but that helped. Try
sudo chmod -R 750 /root/of/website
This will recursively give permission 750 to all files in that directory. Hope it helps!
Note: It might not be the problem so please do not downvote if it doesn't help.

So it was a simple mistake of wrong way to reference the assets. In rails I was using this:
<%= asset_path('/assets/image.png') %>
which in development mode still showed the assets but in production it couldnt find it cause it already adds the assets folder so was looking in assets/assets/image. Changed everything to:
<%= asset_path('image.png') %>
Like it should be and it worked just fine.

Related

Nginx + Puma + Sidekiq web interface not showing css styles

I have an angularJS web app running in a Nginx server that sends request to a Rails API running in a Puma server. I have integrated Sidekiq 5.2.8 and everything works great but the Sidekiq web interface.
In my Nginx config file, I have a rule to pass request to the API. Please find the whole nginx.conf document:
events {
worker_connections 1024;
}
http {
upstream api.development {
# Path to Puma SOCK file, as defined previously
server unix:/tmp/puma.sock fail_timeout=0;
}
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
# set client body size to 10M #
client_max_body_size 10M;
gzip on;
server {
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;
#charset koi8-r;
#access_log logs/host.access.log main;
root /Users/Rober/Projects/domain/dev/domain/app;
index index.html index.htm;
try_files $uri $uri/ /index.html =404;
# Proxy requests to the backoffice Rails API
location /api {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
#proxy_set_header X-Forwarded-Proto https;
proxy_redirect off;
rewrite ^/api(.*) /$1 break;
proxy_pass http://api.development;
}
# Rule to proxy the sidekiq web UI
location /sidekiq {
proxy_pass http://api.development;
}
# Expire rules for static content
# RCM: WPO
# Images
location ~* \.(?:jpg|jpeg|gif|png|ico|cur|gz|svg|svgz|mp4|ogg|ogv|webm|htc)$ {
root /Users/Rober/Projects/domain/dev/domain/app;
expires 1w;
add_header Cache-Control "public";
}
# This rule is the root cause of the problems with the sidekiq css
# I have commented it for testing purposes
# CSS and Javascript
#location ~* \.(?:css|js)$ {
# root /Users/Rober/Projects/domain/dev/domain/app;
# expires 1w;
# add_header Cache-Control "public";
#}
# I have replaced the previous location above for this as suggested by #Beena Shetty.
location ~* \.(?:css|js)$ {
add_header X-debug-message "Into the location css" always;
if ($uri !~* "^/sidekiq/\w*(.*)+$") {
add_header X-debug-message "Into the location css if" always;
root /Users/Rober/Projects/domain/dev/domain/app;
expires 1w;
add_header Cache-Control "public";
}
}
# cache.appcache, your document html and data
location ~* \.(?:manifest|appcache|html?|xml|json)$ {
root /Users/Rober/Projects/domain/dev/domain/app;
expires -1;
}
}
include servers/*;
}
In Rails:
routes:
require 'sidekiq/web'
mount Sidekiq::Web => '/sidekiq'
I have included next rule in Nginx config file and now when I request http://localhost/sidekiq I can see the web interface and navigate, but still cannot see the styles.
location /sidekiq {
proxy_pass http://api.development;
}
See screenshot.
The dev tools shows that when I load sideqik is trying to get bootstrap.css and some other css and javascript in http://localhost/sidekiq/stylesheets/bootstrap.css
What am I missing?
UPDATE:
I have found out the root cause of the problem in my nginx.conf. I have next rule that set a cache expiration time for performance purposes. If I comment this code, everything works. But how can I have both things living together?
CSS and Javascript
location ~* \.(?:css|js)$ {
root /Users/Rober/Projects/domain/dev/domain/app;
expires 1w;
add_header Cache-Control "public";
}
UPDATE 2: Just in case the problem comes from another point, I have included my whole nginx conf.
Now, with the provided config, the expiration rules in my web app are still working, but the css in the sidekiq webapp do not.
I have included two headers as debug. One when the server is accessing the location rule and the second when the server is accessing inside the if condition. When I request my home page with localhost and I check the request for my own css, such as app.css, I can see the header X-debug-message: Into the location css if, which is right.
If I request sidekiq with localhost/sidekiq I still get 404 error for css, let´s say http://localhost/sidekiq/stylesheets/bootstrap.css and I can see the header X-debug-message: Into the location css.
Current conclusions:
As soon as I include the location ~* .(?:css|js)$ rule, sidekiq css stops working. Even if the rule is empty, like:
location ~* \.(?:css|js)$ {
}
As soon as I delete or comment the whole rule, the sidekiq css works perfectly, but unfortunately this is not compatible with the expires rules that we need to include for performance purposes.
Try this:
location ~* \.(?:css|js)$ {
if ($uri !~* "^/sidekiq/\w*(.*)+$"){
root /Users/Rober/Projects/domain/dev/yanpy/app;
expires 1w;
add_header Cache-Control "public";
}
}
I wasn't able to find a fix for this, so I hacked it with the following method: I copied sidekiq's assets to the public folder and it started working ( since they're referenced by the UI ).
|- images
|-- favicon.io
|-- logo.png
|-- status.png
|- javascripts
|-- application.js
|-- dashboard.js
|- stylesheets
|-- application.css
|-- application-rtl.css
|-- application-dark.css
|-- application-rtl.min.css
|-- bootstrap.css
Mainly, these files: https://github.com/mperham/sidekiq/tree/master/web/assets

Nginx + Rails + sendfile : file not found

Been having a problem setting up sendfile for nginx+rails lately. We have one kind of file download that's already handled by nginx, and we wanted to add a second rule to deal with another kind of files at another location, but we have no success so far.
Ruby controller :
def download_file
send_file("/srv/www/myapp/shared/tmp/directory/file.zip")
end
Environment configuration file:
Rails.application.configure do
# ..
config.action_dispatch.x_sendfile_header = 'X-Accel-Redirect' # for nginx
# ..
end
Nginx configuration :
upstream myapp {
server 127.0.0.1:9292;
}
server {
listen 80;
server_name myapp.tld;
client_max_body_size 10M;
root /srv/www/myapp/current/public;
# This first block works perfectly
location /__working_file {
internal;
alias /var/lib/myapp;
}
# This second block does not work at all
location /__new_files {
internal;
alias /srv/www/myapp/shared/tmp/directory;
}
location / {
root /srv/www/myapp/current/public;
try_files $uri #app;
}
location #app {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_set_header X-Sendfile-Type X-Accel-Redirect;
# This first rule works perfectly
proxy_set_header X-Accel-Mapping /var/lib/myapp/=/__working_file/;
# This second rule doesn't
proxy_set_header X-Accel-Mapping /srv/www/myapp/shared/tmp/directory/=/__new_files/;
proxy_pass_header Server;
proxy_read_timeout 300;
proxy_pass http://myapp;
}
}
Result
When accessing the controller action, the send_file command is triggered, then we get a "File not found" in the browser, nothing gets downloaded, and the rails log shows this:
Sent file /srv/www/myapp/shared/tmp/directory/file.zip (0.4ms)
Completed 200 OK in 169ms (ActiveRecord: 37.5ms)
Started GET "/srv/www/myapp/shared/tmp/directory/file.zip" for 109.190.197.126 at 2018-11-20 11:34:44 +0100
ActionController::RoutingError (No route matches [GET] "/srv/www/myapp/shared/tmp/directory/file.zip"):
The file does exist and is readable but nginx can't seem to access it. Any idea?
The problem comes from the way the two X-Accel-Mapping are set. Rack is indeed able to deal with several mappings since the merge of this PR #1187.
However, as of today, this PR has been merged on master but not released yet (2.0.6 is currently the latest release).
The only thing is, the correct way of setting several mappings is by using one single proxy_set_header rule, and separate each mapping with a comma, like this:
proxy_set_header X-Accel-Mapping, /var/lib/myapp/=/__working_file/,/srv/www/myapp/shared/tmp/directory/=/__new_files/
while routing it is not getting . you have to provide proper route to work.

Carrierwave + NGINX rails production images not displaying

I've deployed a rails app that allows a user to upload a photo and have it display on another page, fairly simple. I tested it in development, the image uploads to the public folder and displays properly. In production and deployed the image uploads to the server but isn't being rendered on the page.
404 errors for only the images I've uploaded, path looks like this:
http://IP-OF-APP/uploads/blog/name-of-image.jpg
I read in another S.O. article mentioning setting a production config serve_static_files as true. This didn't fix the problem.
I thought it might be a server configuration with NGINX not picking up the uploads path here is my /sites-default/nginx.conf file.
upstream app {
server unix: /home/deploy/MYAPPNAME/shared/tmp/sockets/puma.sock fail_timeout=0;
}
server {
listen 80;
server_name IP_ADDRESS_OF_SERVER;
root /home/deploy/MYAPPNAME/public;
location #app {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_pass http://app;
proxy_redirect off;
}
location ~ ^/(assets)/ {
root /home/deploy/MYAPPNAME/shared/public
gzip_static on;
expires max;
add_header Cache-Control public;
}
location ~ ^/uploads/ {
root /home/deploy/MYAPPNAME/shared/public;
expires 24h;
add_header Cache-Control public;
break;
}
location ~ ^/(fonts|system)/favicon.ico/robots.txt {
gzip_static on;
expires max;
add_header Cache-Control public;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 4G;
keepalive_timeout 10;
}
I also thought it might be that the image isn't uploading to the server correctly so I SSH'd into the server and found the image uploaded living in:
home/deploy/MYAPPNAME/shared/public/uploads/blog/name-of-image.jpg
Also found what I assume to be the same image in:
home/deploy/MYAPPNAME/current/public/uploads/blog/name-of-image.jpg
My uploader looks like this:
class BlogUploader < CarrierWave::Uploader::Base
include CarrierWave::MiniMagick
storage :file
def store_dir
'uploads/blog'
end
def extension_whitelist
%w(jpg jpeg gif png)
end
process :resize_to_fit => [825, 825]
#titles are validated to be unique
def filename
"#{model.title}"+".#{file.extension}" if original_filename.present?
end
end
using
Rails 4.2.6
Ruby 2.3.0
Carrierwave 0.11.0
EDIT
All of the other static images, CSS and JS are displaying and rendering properly.
The NGINX error when rending the upload image produces this:
2016/04/29 17:32:34 [error] 4993#0: *23 open() "/home/deploy/MYAPPNAME/shared/public/assets/uploads/blog/name-of-image.jpg" fails (2: no such file or directory), client: *******, server: SERVER_IP, request: "GET /assets/uploads/blog/name-of-image.jpg HTTP/1.1", host: "SERVER_IP"
From your log:
User is requesting:
/assets/uploads/blog/name-of-image.jpg
Nginx is looking for the image in:
/home/deploy/MYAPPNAME/shared/public/assets/uploads/blog/name-of-image.jpg
You confirmed that the image is in:
home/deploy/MYAPPNAME/shared/public/uploads/blog/name-of-image.jpg
Nginx is looking in public/assets/uploads, your file is in public/uploads.

Content-Length Header missing from Nginx-backed Rails app

I've a rails app that serves large static files to registered users. I was able to implement it by following the excellent guide here: Protected downloads with nginx, Rails 3.0, and #send_file. The downloads and everything else is working great, but there is just this problem - The Content-Length header isn't being sent.
It's okay for small files, but it gets really frustrating when large files are downloaded, since download managers and browsers don't show any progress. How can I fix this? Do I have to add something to my nginx configuration or do I have to pass along some other option to the send_file method in my rails controller? I have been searching online for quite some time but have been unsuccessful. Please Help! Thanks!
Here's my nginx.conf:
upstream unicorn {
server unix:/tmp/unicorn.awesomeapp.sock fail_timeout=0;
}
server {
listen 80 default_server deferred;
# server_name example.com;
root /home/deploy/apps/awesomeapp/current/public;
location ~ /downloads/(.*) {
internal;
alias /home/deploy/uploads/$1;
}
location ^~ /assets/ {
gzip_static on;
expires max;
add_header Cache-Control public;
}
try_files $uri/index.html $uri #unicorn;
location #unicorn {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_set_header X-Sendfile-Type X-Accel-Redirect;
proxy_set_header X-Accel-Mapping /downloads/=/home/deploy/uploads/;
proxy_pass http://unicorn;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 20M;
keepalive_timeout 10;
}
Okay, here's something. I don't know if it's the right way or not but I was able to fix the issue by manually sending the Content-Length Header from my Rails Controller. Here's what I'm doing:
def download
#file = Attachment.find(params[:id])
response.headers['Content-Length'] = #file.size.to_s
send_file(#file.path, x_sendfile: true)
end
nginx should be automatically able to set the header. There must be something that I'm missing; but until I find a 'proper' solution, I guess this will have to do.
P.S: The Header needs to be a string to work properly with some webservers, hence the .to_s

Rails, favicon.ico not found

This is soo odd, I've been receiving:
ActionController::RoutingError (No route matches "/favicon.ico")
but I have the favicon.ico in my public directory... any ideas how to solve this? Nginx doesn't throw an error at all.
Run
rake assets:precompile
then set
config.serve_static_assets = true
in config\environments\production.rb file.
Then restart your server.
But I think rake assets:precompile is not required.
It seems that nginx doesn't handle your static assets (since this request for static file goes to the ActionController). Check public root in nginx config file nginx.conf.
Here is an example with Capistrano deployments:
server {
listen 80;
root /var/www/my_project/current/public;
}
And do you use a favicon_link_tag helper in your head :) ?
If you want to keep config.serve_static_assets = false, which is recommended if you have nginx or apache, you can tell nginx to statically serve the files directly. This is especially important for performance reasons, as you don't want rails serving these assets.
Below is a sample which also correctly has nginx statically serve the assets directory:
server {
listen 80;
root /var/www/my_project/current/public;
location / {
proxy_pass http://mysite;
proxy_redirect off;
proxy_set_header X_FORWARDED_PROTO https;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
client_max_body_size 10m;
client_body_buffer_size 128k;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_buffer_size 4k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;
}
# static resource routing - both assets folder and favicon.ico
location ~* ^/assets/|favicon.ico {
# Per RFC2616 - 1 year maximum expiry
# http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
expires 1y;
add_header Cache-Control public;
# Some browsers still send conditional-GET requests if there's a
# Last-Modified header or an ETag header even if they haven't
# reached the expiry date sent in the Expires header.
add_header Last-Modified "";
add_header ETag "";
break;
}
}
Make sure that the favicon.ico file isn't empty (byte size > 0). For some reason I had an empty favicon.ico file which was triggering the same error, even though the file did exist.
delete slash sign before favicon.ico and try to use something like:
<link rel="shortcut icon" type="image/png" href="favicon.ico" />
I was facing the same problem when I first clone code from git repository and run with RAILS_ENV=production. Since there was no assets directory in my git repository, I needed to run rake assets:precompile.
Also I was run with rails s, so config.serve_static_assets = true worked. Thanks #Jiemurat

Resources