I need send POST request to my nginx frontend server which should redirect it to upstream servers.
In details:
send request to http://192.168.0.10/foo/bar/blah and URL in this request should be changed to http://192.168.0.21[22,23]:8080/foo/blah
upstream myapp {
server 192.168.0.21:8080;
server 192.168.0.22:8080;
server 192.168.0.23:8080;
}
server {
listen 80;
server_name localhost;
location /foo/bar/blah/ {
rewrite ^/foo/blah^/ /$1 break;
proxy_pass http://myapp;
}
but in nginx error log I see that my request changed from POST to GET and also seems didn't change URL:
"POST /foo/bar/blah HTTP/1.1" 301 185 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64)
"GET /foo/bar/blah/ HTTP/1.1" 404 117 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64)
How can I keep my POST request and change the URL?
Also about my rewrite rule
rewrite ^/foo/blah^/ /$1 break;
I found a lot of examples for changing URL and all of them looks the same.
And it is really strange for me, how this rewrite rule can change URL from /foo/bar/blah/ to /foo/blah/:
in documentaion says:
http://nginx.org/en/docs/http/ngx_http_rewrite_module.html#rewrite
If the specified regular expression matches a request URI, URI is
changed as specified in the replacement string.
but in my case request URI is /foo/bar/blah/ so regular expression /foo/blah^/ doesn't matches URI, so this rule shouldn't work.
Am I right?
Would be helpful any advises.
UPD:
fixed:
location = /foo/bar/blah {
proxy_pass http://myapp/foo/blah;
}
My guess, you don't need rewrite.
location /foo/bar/ {
proxy_pass http://myapp/;
}
This should remove /foo/bar part from proxied URL.
Related
I am sending logs to elasticsearch via rsyslog. For the parsing of those logs, I am using liblognorm rule.
I want to create fields of nginx logs,
here is a log entry,
127.0.0.1 - kibanaadmin [13/Jun/2017:14:18:17 +0530] "GET /ui/favicons/favicon-32x32.png HTTP/1.1" 304 0 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:53.0) Gecko/20100101 Firefox/53.0"
Here is the pattern file,
version=2
rule=:%clientip:ipv4% - %user:word% [%timestamp:char-to:]%] %auth:word% "%verb:alpha% %request:word%" %response:number% %bytes:number% "%referrer:word"%" "%agent:char-to:{"extradata":"("}"
The reason for parsefailure is I believe due to the date-time format.
Can somebody help in creating a rule for parsing nginx logs ?
For the aforementioned Nginx access log line you need a rule like this:
version=2
rule=:%clientip:ipv4%%-:whitespace%-%-:whitespace%%user:word%%-:whitespace%[%timestamp:char-to{"extradata":"]"}%]%-:whitespace%"%verb:word%%-:whitespace%%request:word%%-:whitespace%%protocol:char-to{"extradata":"\""}%"%-:whitespace%%response_code:number{"format":"number"}%%-:whitespace%%bytes:number{"format":"number"}%%-:whitespace%"%referrer:char-to{"extradata":"\""}%"%-:whitespace%"%agent:char-to{"extradata":"\""}%"
which produces the following output (pretty printed):
{
"agent": "Mozilla\/5.0 (X11; Ubuntu; Linux x86_64; rv:53.0) Gecko\/20100101 Firefox\/53.0",
"referrer": "-",
"bytes": 0,
"response_code": 304,
"protocol": "HTTP\/1.1",
"request": "\/ui\/favicons\/favicon-32x32.png",
"verb": "GET",
"timestamp": "13\/Jun\/2017:14:18:17 +0530",
"user": "kibanaadmin",
"clientip": "127.0.0.1"
}
To test your log file along with a Liblognorm rule file more easily you can create a nginx.rule file for your rules and run the command:
lognormalizer -H -p -r nginx.rule < nginx.log
You can find more information about lognormalizer command here.
I have an app that lives in a sub-URI, http://myhost/app2. I'm having a hard time figuring out how I can configure the proper locale routing for this.
I've put this in routes.rb I've tried with both:
Rails.application.routes.draw do
get '' => redirect("/#{I18n.default_locale}")
scope "/:locale", locale: /#{I18n.available_locales.join("|")}/ do
root 'index#home'
end
end
as well as:
Rails.application.routes.draw do
prefix = Rails.application.config.relative_url_root
get "#{prefix}" => redirect("#{prefix}/#{I18n.default_locale}")
scope "#{prefix}/:locale", locale: /#{I18n.available_locales.join("|")}/ do
root 'index#home'
end
end
In my production.rb file, I have: config.relative_url_root = "/app2"
My nginx.conf file (taken from the Phusion Passenger docs):
http {
...
server {
...
location ~ ^/app2(/.*|$) {
alias /home/dani/app2/public$1;
passenger_base_uri /app2;
passenger_app_root /home/dani/app2/;
passenger_document_root /home/dani/app2/public/;
passenger_enabled on;
}
}
}
When I try accessing http://myhost/app2 in production, I get a Rails error page that says "The page you were looking for doesn't exist."
My production.log file looks like this with every request I make:
I, [2014-09-01T12:22:56.050499 #15230] INFO -- : Started GET "/app2" for myipaddress at 2014-09-01 12:22:56 -0400
I looked at the nginx access.log file and I see the following:
[01/Sep/2014:12:26:04 -0400] "GET /app2 HTTP/1.1" 301 91 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:31.0) Gecko/20100101 Firefox/31.0"
[01/Sep/2014:12:26:04 -0400] "GET /en HTTP/1.1" 404 1351 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:31.0) Gecko/20100101 Firefox/31.0"
So redirection from /app2 to /en is occurring but myhost/en doesn't resolve to anything on this setup, it should be myhost/app2/en.
Any advice would be helpful.
Note: Everything works fine in development.
I've deployed my RoR 4 app using Capistrano 2, Unicorn, Nginx.
The problem is that I get 404 on assets(stylesheets, javascripts).
Here's Nginx access log:
89.0.40.233 - - [16/Mar/2014:08:24:26 +0000] "GET /stylesheets/application.css HTTP/1.1" 404 650 "http://host.cloudapp.net/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.149 Safari/537.36"
89.0.40.233 - - [16/Mar/2014:08:24:26 +0000] "GET /javascripts/application.js HTTP/1.1" 404 650 "http://host.cloudapp.net/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.149 Safari/537.36"
My assets are in the app folder:
azureuser#host:~/apps/testify/current/public$ ls -a assets
. application-d65a0eaefe6ca2eef9400045f94ab52b.js
.. application-d65a0eaefe6ca2eef9400045f94ab52b.js.gz
application-71e2591e9586afebf3fb4ff70aaae199.css manifest-a348973e84698f7d898e8021bd6e5388.json
application-71e2591e9586afebf3fb4ff70aaae199.css.gz
My Nginx config:
upstream unicorn {
server unix:/tmp/unicorn.testify.sock fail_timeout=0;
}
server {
listen 80 default deferred;
root /home/azureuser/apps/testify/current/public;
location ^~ /assets/ {
expires max;
add_header Cache-Control public;
}
try_files $uri/index.html $uri #unicorn;
location #unicorn {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://unicorn;
}
error_page 500 502 503 504 /500.html;
client_max_body_size 4G;
keepalive_timeout 10;
}
Where do I start looking?
As it seems from access.log you just hardcoded application.css/.js in your layout. There is no such files in production public folder because of fingerprint name that asset pipeline gives them (look at your example ls output).
You may read about this here.
Fixing your problem really simple. Replace hardcoded links for application.css/.js with this code:
<%= javascript_include_tag "application" %>
<%= stylesheet_link_tag "application" %>
Adding to the answer of Sergey Moiseev
Try fetching the assets directly with the url
/assets/javascripts/application-d65a0eaefe6ca2eef9400045f94ab52b.js.If it doesnt work the your issue is in nginx rather than rails.
Also please check whether you are fetching the file with right fingerprint. In this case check whether its application-d65a0eaefe6ca2eef9400045f94ab52b.js or someother application.js with a different fingerprint. I got a similar issue with multiple servers.
For me, the root path was wrong in my nginx config, and I was experiencing this exact issue. Reference: https://stackoverflow.com/a/25341195/2544629
I would guess Capistrano is not using "production" as the environment name for Unicorn (-E option). "ps aux|grep unicorn" would probably tell you which environment it's using.
I'm running an app which accepts data from GPS loggers, being this data sent via SMS, HTTP or TCP. I was using the HTTP connection to transmit the data, which was straightforward to receive through a GET request on my Rails app.
For battery saving purposes, we changed the connection to pure TCP and nginx is not accepting these requests for now:
From nginx's access.log:
HTTP:
xx.xx.xx.xxx - - [03/Mar/2013:20:17:45 -0500] "GET /?imei=123456789012345&rmc=$GPRMC,035106.000,A,4145.1451,N,08506.8784,W,1.56,176.49,010213,,*13,3656mV,AUTO HTTP/1.1" 302 247 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.152 Safari/537.22"
TCP:
xx.xx.xx.xxx - - [03/Mar/2013:18:31:22 -0500] "imei=123456789012345&rmc=$GPRMC,233012.000,A,4221.6614,N,07106.1014,W,17.25,218.94,030313,,*21,4102mV,AUTO,310,260,ADB7,13EF,057,310,260,0000,0000,044,310,260,0000,0000,055,310,260,0000,0000,05A,310,260,0000,0000,059,310,260,0000,0000,05E,0,0,0000,0000,000,2" 400 172 "-" "-"
Does anyone know a way to filter these TCP requests on Nginx and process them as HTTP GET requests to be available through a Rails app?
Thanks in advance
Answer:
I was able to achieve this by creating a custom proxy, using em-proxy and faraday:
require 'em-proxy'
require 'faraday'
Proxy.start(:host => "0.0.0.0", :port => 8080, :debug => false) do |conn|
conn.on_data do |data|
http = Faraday.new(:url => 'http://example.com') do |faraday|
faraday.request :url_encoded # form-encode POST params
faraday.response :logger # log requests to STDOUT
faraday.adapter Faraday.default_adapter # make requests with Net::HTTP
end
http.get "/upload?#{data}"
end
end
You should look at websockets. Nginx now has native support for them, but there is an older module for Nginx that does the work as well.
Here's an older question about this with both solutions provided.
Nginx is a HTTP server so it expects clients to speak HTTP. It is not possible to have a client speaking a non-HTTP protocol to nginx in the manner you're describing (ok, maybe you could write a custom module to do it, but that seems silly).
I think what you need to do is write your own proxy server which will convert the raw TCP protocol into HTTP requests. Your GPS loggers would talk to the proxy which would then talk to nginx. This ought to be fairly simple since the TCP payload appears to just be a urlencoded string, so you could just use those directly.
I keep getting the two following errors from my server, I assumed they were just bots looking for potential targets, but does anyone know specifically why I'm getting these? I'm using the SslRequirement plugin to make sure all hits to the login/signup page are redirected to SSL, so all of these weird https requests to root should just be redirected to regular http.
A ActionController::UnknownHttpMethod occurred in application#index:
quit, accepted HTTP methods are get, head, put, post, delete, and options
/usr/local/lib/ruby/gems/1.9.1/gems/actionpack-2.3.4/lib/action_controller/request.rb:35:in `request_method'
PATH_INFO : /
REMOTE_ADDR : 99.19.208.249
REMOTE_PORT : 6376
REQUEST_METHOD : CONNECT
REQUEST_URI : /
SERVER_PORT : 443
SERVER_PROTOCOL : HTTP/1.0
SERVER_SOFTWARE : Apache
A ActionController::UnknownHttpMethod occurred in application#index:
CONNECT, accepted HTTP methods are get, head, put, post, delete, and options
/usr/local/lib/ruby/gems/1.9.1/gems/actionpack-2.3.4/lib/action_controller/request.rb:35:in `request_method'
HTTPS : on
HTTP_X_FORWARDED_PROTO : https
PATH_INFO : /
REMOTE_ADDR : 91.209.196.76
REMOTE_PORT : 50751
REQUEST_METHOD : quit
REQUEST_URI : /
SERVER_PORT : 443
SERVER_PROTOCOL : HTTP/0.9
The CONNECT command is used by HTTP proxy servers to indicate that the client wants to just connect a socket directly to another server; this is usually used for tunneling TLS over an HTTP proxy, but could be used for tunneling almost any protocol.
QUIT is not an HTTP command, but it is an SMTP command. It is possible that you are getting these commands from a bot that is trying to find open relays for sending spam; it's trying to figure out if you have an open SMTP relay, or an open HTTP proxy that allows the CONNECT command which could also be used to tunnel SMTP traffic.
So, likely you're just being hit by a spam botnet trying to find open relays. My advice would be to drop such requests as early as possible, and not worry about them.
CONNECT: http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.9
QUIT usually means 'close the connection'.
# Avoid annoying ActionController::UnknownHttpMethod exceptions like:
#
# ActionController::UnknownHttpMethod) "CONNECT, accepted HTTP methods are get, head, put, post, delete, and options"
#
# Install this file in app/metal and these requests will receive a 405
# "Method Not Allowed" status and will be logged under `info'.
class IgnoreUnknownHttpMethod
def self.call(env)
[
if ActionController::Request::HTTP_METHODS.include?(env["REQUEST_METHOD"].downcase)
404 # Not Found
else
Rails.logger.info("Ignoring unknown HTTP method; #{env.inspect}")
405 # Method Not Allowed
end, {"Content-Type" => "text/plain"}, []]
end
end
Credit https://gist.github.com/remvee/600569
I just noticed a few attempt to connect to my servers as follow, we see that it ends with a QUIT...
198.20.87.98 - - [22/Dec/2015:21:43:42 -0800] "GET / HTTP/1.1" 444 5666 "-" "-"
198.20.87.98 - - [22/Dec/2015:21:43:42 -0800] "GET /robots.txt HTTP/1.1" 444 5666 "-" "-"
198.20.87.98 - - [22/Dec/2015:21:43:42 -0800] "GET /sitemap.xml HTTP/1.1" 444 5666 "-" "-"
198.20.87.98 - - [22/Dec/2015:21:43:58 -0800] "quit" 405 5461 "-" "-"
As a side note, my server returns 444 which is not a legal HTTP code. It means NO RESPONSE and I do that because their "Agent String" is empty.
Looking at the IP address, I found a search engine that searches for things on the internet. Not a hacker trying to break in per se. (There intend is not evil, it seems.) I suppose that some of the applications they are checking for must understand the QUIT. So as a robot, it makes sense that you would want to try what would otherwise looks like weird commands such as CONNNECT and QUIT. There is no real limit to the commands an application can support, even if those commands are not official HTTP commands.
If your application does not understand these methods, then nothing will happen so you should not bother too much about these.
If your application receives those methods, then you may want to write a little something at the start to get the method and compare to GET and POST (and whatever else you support, like DELETE and PUT) and if it matches none of these, then reply with a 405 error code: "Method Not Allowed".
https://www.rfc-editor.org/rfc/rfc7231#section-6.5.5
If you cannot change your application and you are not sure whether it could react to a CONNECT / QUIT hit, then you could look into using a setup such as mod_security for Apache2.
as per jturkel at https://gist.github.com/remvee/600569.
In Rails 3.2 for non http actions I added the following at the end of config/application.rb and solved the problem for quit.
# silence ActionController::UnknownHttpMethod exceptions
ActionDispatch::ExceptionWrapper.rescue_responses.merge!( 'ActionController::UnknownHttpMethod' => :method_not_allowed )