Cannot upload a larger file nginx - docker

I have configured Nginx with client_max_body_size 100M;
Although still, I am facing an error while uploading a file of 25 MB
The nginx.conf is as follows:
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
include /etc/nginx/conf.d/*.conf;
}
The ngnix.conf has an included conf file which is as follows:
client_max_body_size 100M;
upstream third-party{
server cognitiveview-thirdparty;
}
server {
listen 443 ssl;
ssl_certificate <key1>;
ssl_certificate_key <key2>;
server_name <>;
server_tokens off;
location / {
proxy_pass http://;
proxy_set_header Host "host.com";
proxy_connect_timeout 1500s;
proxy_send_timeout 1500s;
proxy_read_timeout 1500s;
send_timeout 1500s;
proxy_ssl_server_name on;
client_max_body_size 100M;
}
}
server {
listen 80;
server_name <>;
return 301 https://$host$request_uri;
client_max_body_size 100M;
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
client_max_body_size 100M;
}
location /uploads {
client_max_body_size 100M;
}
}
I tried multiple which suggests setting the client_max_body_size in server, location, and HTTP. As per my understanding, I have updated the values, although still I am not successful in uploading the file for a larger size.
The file is being uploaded through a react application which is going to nginx to a backemd

Related

Nginx 404 for /rails/active_storage/blobs/*filename(extension)

Below is my AWS NGINX config file. I am using rich text with active storage in my new rails app after upload when i am trying to open a file it gives me 404 error in production where in development it works really fine.
files:
/etc/nginx/conf.d/proxy.conf:
mode: "000755"
owner: root
group: root
content: |
upstream backend {
server unix:///var/run/puma/my_app.sock;
}
log_format logd '$msec"$uri"'
'$status"$request_time"$upstream_response_time"'
'$http_x_forwarded_for';
server {
listen 80;
server_name _ localhost; # need to listen to localhost for worker tier
return 301 https://$host$request_uri;
}
server {
listen 443;
charset UTF-8;
server_name _ localhost; # need to listen to localhost for worker tier
root /var/app/current/public;
# try_files $uri/index.html $uri /deploy/$uri /deploy/$uri.html /deploy/$uri.js #puma;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
if ($time_iso8601 ~ "^(\d{4})-(\d{2})-(\d{2})T(\d{2})") {
set $year $1;
set $month $2;
set $day $3;
set $hour $4;
}
access_log /var/log/nginx/access.log main;
access_log /var/log/nginx/healthd/application.log.$year-$month-$day-$hour logd;
# 413 Request Entity Too Large
client_max_body_size 50M;
large_client_header_buffers 8 32k;
location / {
try_files $uri /deploy/$uri /deploy/$uri.html /deploy/$uri.js #puma;
}
location #puma{
proxy_pass http://backend;
# proxy_pass http://backend; # match the name of upstream directive which is defined above
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
# prevents 502 bad gateway error
proxy_buffers 8 32k;
proxy_buffer_size 64k;
proxy_redirect off;
#break;
}
location /assets {
alias /var/app/current/public/assets;
allow all;
}
location ~ \.(png|jpg|jpeg|gif|ico|html|woff|woff2|ttf|svg|eot|otf|pdf)$ {
expires max;
access_log off;
add_header Cache-Control public;
add_header Access-Control-Allow-Origin *;
}
location ~* \.(?:jpg|jpeg|gif|png|ico|cur|gz|svg|mp4|ogg|ogv|webm|htc|pdf)$ {
access_log off;
add_header Cache-Control "max-age=2592000";
}
if ($request_method !~ ^(GET|HEAD|PUT|PATCH|POST|DELETE|OPTIONS)$ ){
return 405;
}
if (-f $document_root/system/maintenance.html) {
return 503;
}
}
Whenever i open Link with file extension it shows me 404 error but if i open the same link without file extension it works Link without file extension. Not sure what i am doing wrong please help
Try disabling the below blocks in your Nginx, this might fix it
location ~ \.(png|jpg|jpeg|gif|ico|html|woff|woff2|ttf|svg|eot|otf|pdf)$ {
gzip_static on;
gzip on;
gzip_comp_level 6;
gzip_buffers 16 8k;
gzip_vary on;
gzip_proxied any;
expires max;
access_log off;
add_header Cache-Control public;
add_header Access-Control-Allow-Origin *;
}
location ~* \.(?:jpg|jpeg|gif|png|ico|cur|gz|svg|mp4|ogg|ogv|webm|htc|pdf)$ {
access_log off;
add_header Cache-Control "max-age=2592000";
}

504 on long upload with Docker Nginx

I spent so much time trying to sort this out, it's ridiculous...
I need to upload files up to 15Gb.
nginx.conf:
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
sendfile on;
keepalive_timeout 800s;
include /etc/nginx/conf.d/*.conf;
}
Custom nginx conf
server {
listen 80;
listen [::]:80;
server_name example.com;
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
if ($http_x_forwarded_proto != "https") {
rewrite ^(.*)$ https://$server_name$REQUEST_URI permanent;
}
root /var/www/webroot;
index index.php;
client_body_timeout 800s;
client_header_timeout 800s;
client_max_body_size 15000m;
client_body_temp_path /store/nginx-tmp 1 2;
#fastcgi_buffers 8 1600k;
#fastcgi_buffer_size 3200k;
add_header X-Frame-Options sameorigin;
location / {
try_files $uri $uri/ /index.php?$args;
}
location ~ \.php$ {
try_files $uri =404;
include fastcgi_params;
fastcgi_pass web:9000;
fastcgi_index index.php;
fastcgi_intercept_errors on;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_connect_timeout 800s;
fastcgi_read_timeout 800s;
fastcgi_send_timeout 800s;
#proxy_connect_timeout 800s;
#proxy_send_timeout 800s;
#proxy_read_timeout 800s;
#send_timeout 800s;
}
}
The commented lines are additional configs I tried.
I added client_body_temp_path /store/nginx-tmp 1 2; for Nginx to use a mounted volume to store temp files if needed because the EC2 instance only has a 8Gb disk.
Relevant PHP config:
upload_max_filesize = 15000M
post_max_size = 15000M
max_execution_time = 0
request_terminate_timeout = 800s
I am running a CakePHP application, in case that helps cracking this case.
When I upload a large file, I get a 504 after 2 to 2.5 minutes. The upload in incomplete ($_FILES['file']['error'] = 3, UPLOAD_ERR_PARTIAL)
There nothing in the PHP logs.
Nginx logs only has this:
2020/04/17 06:38:04 [warn] 479#479: *943 a client request body is buffered to a temporary file /var/cache/nginx/client_temp/0000000007, client: 172.31.20.155, server: example.com, request: "POST /materials/add/38999 HTTP/1.1", host: "example.com", referrer: "https://example.com/jobs/view/38999"
Can anyone save my sanity here?
Thanks,

Welcome to Nginx page displays instead of actual webpage

I am working on a Ruby on Rails project on an Ubuntu server. Whenever I try and access the app, I am always greeted with:
Welcome to nginx!
If you see this page, the nginx web server is successfully installed and working. Further configuration is required.
Here is my code from the "sites_enabled" directory:
pstream unicorn_site {
server unix:/tmp/unicorn.site.sock fail_timeout=0;
}
server {
listen 80;
client_max_body_size 4G;
keepalive_timeout 10;
error_page 500 502 504 /500.html;
error_page 503 #503;
server_name http://[ip_address];
root /data/site/current/public;
try_files $uri/index.html $uri #unicorn_site;
location #unicorn_site {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://unicorn_site;
# limit_req zone=one;
access_log /var/log/nginx/site.access.log;
error_log /var/log/nginx/site.error.log;
}
location ^~ /assets/ {
gzip_static on;
expires max;
add_header Cache-Control public;
}
location = /50x.html {
root html;
}
location = /404.html {
root html;
}
location #503 {
error_page 405 = /system/maintenance.html;
if (-f $document_root/system/maintenance.html) {
rewrite ^(.*)$ /system/maintenance.html break;
}
rewrite ^(.*)$ /503.html break;
}
if ($request_method !~ ^(GET|HEAD|PUT|PATCH|POST|DELETE|OPTIONS)){
return 405;
}
if (-f $document_root/system/maintenance.html) {
return 503;
}
}
I am not sure what the problem is as the app seems to have everything it need to be working correctly. I can provide any part of the code anyone needs. Help is much appreciated.
EDIT: I am also always getting this error when I attempt to load the page:
2018/06/15 15:39:10 [warn] 15280#0: server name "http://[ip_address]" has suspicious symbols in /etc/nginx/sites-enabled/site:14
EDIT 2: Here is nginx.conf
user www-data;
worker_processes 4;
pid /run/nginx.pid;
events {
worker_connections 768;
}
http {
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# Logging Settings
##
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
##
# Gzip Settings
##
gzip on;
gzip_disable "msie6";
##
# Virtual Host Configs
##
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
Move 'try_files $uri/index.html $uri #unicorn_site;' under location /
location / {
try_files $uri/index.html $uri #unicorn_site;
}
The server_name http://[ip_address]; looks wrong. try server_name example.com
Have you configured ruby_passenger line ?
Follow this carefully and your deployment should be successful
Choose your ubuntu version there and proceed
https://gorails.com/deploy/ubuntu/16.04

Configure nginx for different ports

I have set up my nginx configuration on which I am running two websites on same port .My problem is I don't want to specify port number whenever I opening my website .I am running my websites on port 81 .My problem is if I have these two website abc.com:81 and xyz.com:81 .I need to include the port number while opening them which I don't want so can anyone tell me the possible solution ..
Here is my nginx configuration
user abc;
worker_processes 2;
error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
passenger_root /usr/local/rvm/gems/ruby-2.2.3/gems/passenger-5.0.30;
passenger_ruby /usr/local/rvm/gems/ruby-2.2.3/wrappers/ruby;
passenger_max_pool_size 50;
passenger_min_instances 5;
passenger_max_instances_per_app 0;
passenger_pool_idle_time 0;
passenger_max_preloader_idle_time 0;
passenger_app_env staging;
passenger_friendly_error_pages on;
server {
listen 81 default_server;
passenger_enabled on;
server_name abc.com;
root /var/www/project/current/public;
location / {
}
error_page 404 /404.html;
location = /40x.html {
root /var/www/project/current/public;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /var/www/project/current/public;
}
}
server {
listen 81 ;
passenger_enabled on;
root /var/www/new_project/current/public;
server_name xyz.com;
if (-f $document_root/system/maintenance.html)
{
rewrite ^(.*)$ /system/maintenance.html last;
break;
}
error_page 500 502 503 504 /500.html;
location = /500.html
{
root /var/www/new_project/current/public;
}
error_page 404 /404.html;
location = /404.html
{
root /var/www/new_project/current/public;
}
}
}
The default port for HTTP is port 80. Thus, if you listen on port 80, you don't need to specify the port number in the URL since clients will default to port 80 if no other port is specified in the URL.
For all other ports, it is necessary to add the port in the URL though since the client has no other way to figure out the port to connect to in any other way.

Missing Content-Length header when using Nginx + Gzip + Unicorn

I don't know why http response is missed "Content-Length header" when I use gzip in nginx, i'm really getting stuck,please somebody help me, thank you so much!
this is my config file,
nginx.conf
user nobody nobody ;
worker_processes 8;
events {
worker_connections 1024;
accept_mutex on; # "on" if nginx worker_processes > 1
use epoll; # enable for Linux 2.6+
}
http {
include mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
upstream backend-unicorn {
server unix:/tmp/unicorn_app.sock fail_timeout=0;
#server localhost:5000;
}
#access_log logs/access.log main;
sendfile on;
keepalive_timeout 100;
gzip on;
gzip_static on;
gzip_proxied any;
gzip_http_version 1.1;
gzip_comp_level 6;
gzip_min_length 1100;
gzip_buffers 16 8k;
gzip_types text/plain application/zip text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript image$
server {
listen 8080;
server_name misago;
access_log /var/log/nginx/unicorn.access.log main;
client_max_body_size 64M;
location /uploads/ {
root /usr/local/rails_apps/me_management_tool/current/public/uploads;
break;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
I don't know why http response is missed "Content-Length header" when I use gzip in nginx
Because in this case the length of content is unknown at the moment of sending headers. Nginx cannot know how well the content will be compressed.
Below is a performant Nginx solution to add x-file-size header to Gzip responses. Full discussion here: https://github.com/AnthumChris/fetch-progress-indicators/issues/13
location / {
## Nginx Lua module must be installed https://docs.nginx.com/nginx/admin-guide/dynamic-modules/lua/
## https://github.com/openresty/lua-nginx-module#header_filter_by_lua
header_filter_by_lua_block {
function file_len(file_name)
local file = io.open(file_name, "r")
if (file == nil) then return -1 end
local size = file:seek("end")
file:close()
return size
end
ngx.header["X-File-Size"] = file_len(ngx.var.request_filename);
}
}

Resources