Route uwsgi logs by HTTP method (HEAD/GET/etc) - uwsgi

I'm using uwsgi behind nginx, monitored by monit.
Monit checks the status of the site by (by default) sending a HEAD / HTTP/1.1 request. This works fine, but I end up with an awful lot of these lines in the logfile:
[pid: 28317|app: 0|req: 1/1] xxx.ip.addr.xxx () {34 vars in 366 bytes} [Wed Aug 2 16:17:58 2017] HEAD / => generated 0 bytes in 13 msecs (HTTP/1.1 200) 2 headers in 81 bytes (0 switches on core 0)
It looks like log-route and log-req-route are designed for this, see the docs and other questions on this site, but I can't seem to route HEAD requests to their own file.
My uwsgi.ini:
# Log Monit pings separately
req-logger = monitrequests file:uwsgi-monitrequests.log
# All other requests
req-logger = file:uwsgi-requests.log
# Everything else
logger = file:uwsgi.log
log-req-route = monitrequests .*HEAD.*
All the logged requests just end up in uwsgi-requests.log.
One work-around could be to have Monit probe a health-check url (e.g. my.site/heartbeat) and have uwsgi route based on that, are there any better options?

Related

How can I use Redis in my Ruby on Rails application with Windows OS?

First of all I am sorry for my newbie question, but I am kinda stuck here.
Could anyone please tell me step by step how to use Redis in my Rails 6 application, if I have Windows OS?
I have installed Redis, currently it is in my C:\Program Files,
with these files inside it
I started the redis-server.exe, when it starts running it says:
[8020] 20 Nov 17:26:06 # Warning: no config file specified, using the default config.
In order to specify a config file use 'redis-server /path/to/redis.conf'
[8020] 20 Nov 17:26:06 * Server started, Redis version 2.4.6
[8020] 20 Nov 17:26:06 # Open data file dump.rdb: No such file or directory
[8020] 20 Nov 17:26:06 * The server is now ready to accept connections on port 6
379
[8020] 20 Nov 17:26:07 - 0 clients connected (0 slaves), 672768 bytes in use
[8020] 20 Nov 17:26:12 - 0 clients connected (0 slaves), 672768 bytes in use
[8020] 20 Nov 17:26:17 - 0 clients connected (0 slaves), 672768 bytes in use
.......... (it just keep outputs this same text every 5 seconds)
Also in my Rails application, I configured some things. I changed config/cable.yml file to:
development:
adapter: redis
url: redis://localhost:6379/1
test:
adapter: test
production:
adapter: redis
url: <%= ENV.fetch("REDIS_URL") { "redis://localhost:6379/1" } %>
channel_prefix: actioncable_test_production
My concept was to create a channel, named 'room', and output a message in the console, to see its successfully connected. But my problem is that it doesn't output anything.
I set app/javascript/channels/room_channel.js to:
import consumer from "./consumer"
consumer.subscriptions.create("RoomChannel", {
connected() {
// Called when the subscription is ready for use on the server
console.log("Connected succesfully!")
},
disconnected() {
// Called when the subscription has been terminated by the server
},
received(data) {
// Called when there's incoming data on the websocket for this channel
}
});
and my app/channels/room_channel.rb file to:
class RoomChannel < ApplicationCable::Channel
def subscribed
stream_from "room_channel"
end
def unsubscribed
# Any cleanup needed when channel is unsubscribed
end
end
Also, on my command prompt shows this: (after the usual rendering messages and things like that):
Started GET "/cable" for 127.0.0.1 at 2021-11-20 18:24:05 +0100
Started GET "/cable/" [WebSocket] for 127.0.0.1 at 2021-11-20 18:24:06 +0100
Successfully upgraded to WebSocket (REQUEST_METHOD: GET, HTTP_CONNECTION: keep-a
live, Upgrade, HTTP_UPGRADE: websocket)
When I go to the browser console, it says Firefox cannot create connection with ws://localhost:3000/cable
Any idea what's the problem here?
Thanks for the answers and best regards, Rererbit!
You are using a very old and deprecated version of Redis, an abandoned project by Microsoft called OpenTech Redis.
I suggest you checkout Memurai.
Memurai is an up-to-date native Windows port of Redis that derives from that project (see this commit).
Memurai is also available with a free Developer Edition.
Disclaimer: I work in Memurai.

Configuring uWSGI to not interpret PATH_INFO

How can I configure uwsgi to pass in the request path unmodified as PATH_INFO? I.e. if there is a request https://example.com/foo%5F/../bar?x=y, I want PATH_INFO to be literally /foo/../%5Fbar, and not /_bar.
The uWSGI documentation says uWSGI is able to rewrite request variables in lot of advanced ways, but I am unable to find any way to set individual request variables, at least not without modifying the source code of uwsgi.
The reason I want to do is that I have a frontend application which takes user input and then sends a request to http://backend.app/get/USER_INPUT. Trouble is, there is an uwsgi in between, and when the user input is ../admin/delete-everything, the request goes to http://backend.app/admin/delete-everything!
(This uwsgi change I desire will not be the only fix; the frontend app should certainly validate user input, and the backend app should not offer /admin to the frontend app in the first place. But as a measure of defense-in-depth, I'd like my requests to pass uwsgi unmodified.)
I am running bare uWSGI without nginx, i.e. uwsgi --http 0.0.0.0:8000 --wsgi-file myapp/wsgi.py --master --processes 8 --threads 2.
For what it's worth, the backend app that looks into PATH_INFO is Django.
My previous answer holds true for the clients which do url parsing at the source. This answer is applicable, when you can actually get the correct request.
The wsgi.py is run by uwsgi and the application object is called as callable. This in case of Django is WSGIHanlder, which has below code
class WSGIHandler(base.BaseHandler):
request_class = WSGIRequest
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.load_middleware()
def __call__(self, environ, start_response):
set_script_prefix(get_script_name(environ))
signals.request_started.send(sender=self.__class__, environ=environ)
print(environ)
request = self.request_class(environ)
response = self.get_response(request)
response._handler_class = self.__class__
status = '%d %s' % (response.status_code, response.reason_phrase)
response_headers = [
*response.items(),
*(('Set-Cookie', c.output(header='')) for c in response.cookies.values()),
]
start_response(status, response_headers)
if getattr(response, 'file_to_stream', None) is not None and environ.get('wsgi.file_wrapper'):
response = environ['wsgi.file_wrapper'](response.file_to_stream)
return response
I created a sample view to test the same
from django.http import HttpResponse
def index(request, **kwargs):
return HttpResponse("Hello, world. You're at the polls index. " + request.environ['PATH_INFO'])
def index2(request, **kwargs):
return HttpResponse("Hello, world. You're at the polls index2. " + request.environ['PATH_INFO'])
and registered them using below code
from django.urls import include, path
from polls.views import index2, index
urlpatterns = [
path('polls2/', index2, name='index2'),
path('polls2/<path:resource>', index2, name='index2'),
path('polls/', index, name='index'),
path('polls/<path:resource>', index, name='index'),
]
So what you need is overriding this class. Below is an example
import django
from django.core.handlers.wsgi import WSGIHandler
class MyWSGIHandler(WSGIHandler):
def get_response(self, request):
request.environ['ORIGINAL_PATH_INFO'] = request.environ['PATH_INFO']
request.environ['PATH_INFO'] = request.environ['REQUEST_URI']
return super(MyWSGIHandler, self).get_response(request)
def get_wsgi_application():
"""
The public interface to Django's WSGI support. Should return a WSGI
callable.
Allows us to avoid making django.core.handlers.WSGIHandler public API, in
case the internal WSGI implementation changes or moves in the future.
"""
django.setup()
return MyWSGIHandler()
application = get_wsgi_application()
After this can you can see the below results
$ curl --path-as-is "http://127.0.0.1:8000/polls/"
Hello, world. You're at the polls index. /polls/
$ curl --path-as-is "http://127.0.0.1:8000/polls2/"
Hello, world. You're at the polls index2. /polls2/
$ curl "http://127.0.0.1:8000/polls2/../polls/"
Hello, world. You're at the polls index. /polls/
$ curl --path-as-is "http://127.0.0.1:8000/polls2/../polls/"
Hello, world. You're at the polls index. /polls2/../polls/%
As you can see the change to PATH_INFO doesn't change which view is picked. As polls2 still picks index function
After digging a bit more, I realised there is another path and path_info variable. The class for the same is picked using path_info
So we update our function like below
class MyWSGIHandler(WSGIHandler):
def get_response(self, request):
request.environ['ORIGINAL_PATH_INFO'] = request.environ['PATH_INFO']
request.environ['PATH_INFO'] = request.environ.get('REQUEST_URI', request.environ['ORIGINAL_PATH_INFO'])
request.path = request.environ['PATH_INFO']
request.path_info = request.environ.get('REQUEST_URI', request.environ['PATH_INFO'])
return super(MyWSGIHandler, self).get_response(request)
After this change, we get the desired results
$ curl --path-as-is "http://127.0.0.1:8000/polls2/../polls/"
Hello, world. You're at the polls index2. /polls2/../polls/
So your problem has mostly nothing to do with uwsgi or Django as such. To demonstrated the issue, I created a simple flask app with a catch all handler
from flask import Flask
app = Flask(__name__)
#app.route('/', defaults={'path': ''})
#app.route('/<path:path>')
def catch_all(path):
return 'You want path: %s' % path
if __name__ == '__main__':
app.run()
Now when you run this and make a curl request
$ curl -v http://127.0.0.1:5000/tarun/../lalwani
* Rebuilt URL to: http://127.0.0.1:5000/lalwani
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 5000 (#0)
> GET /lalwani HTTP/1.1
> Host: 127.0.0.1:5000
> User-Agent: curl/7.54.0
> Accept: */*
>
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Content-Type: text/html; charset=utf-8
< Content-Length: 22
< Server: Werkzeug/0.15.2 Python/3.7.3
< Date: Fri, 26 Jul 2019 07:45:16 GMT
<
* Closing connection 0
You want path: lalwani%
As you can see that the server never had a chance to even know we requested this. Now lets do it again and ask curl not to tamper the url
$ curl -v --path-as-is http://127.0.0.1:5000/tarun/../lalwani
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 5000 (#0)
> GET /tarun/../lalwani HTTP/1.1
> Host: 127.0.0.1:5000
> User-Agent: curl/7.54.0
> Accept: */*
>
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Content-Type: text/html; charset=utf-8
< Content-Length: 31
< Server: Werkzeug/0.15.2 Python/3.7.3
< Date: Fri, 26 Jul 2019 07:48:17 GMT
<
* Closing connection 0
You want path: tarun/../lalwani%
Now you can see that my app did receive the actual path. Now let's see the same case in a browser, with app not even running
As you can even though my service is not even running but the browser itself refactored the call to /lalwani instead of /tarun/../lalwani. So there is nothing that could have been done at your end to even correct the issue, until unless you are using a client which supports disabling the url parsing at source

Why does FreeRadius fail to process the accounting response from Fortigate?

I have configured a freeradius proxy (3.0.16) on Ubuntu (4.15.0-47-generic). It receives the radius accounting packets from another radius server running on Ubuntu and writes those to another radius server on running on Fortigate.
Radius Server ---> Proxy Radius Server ---> Fortigate Radius Server
I have configured copy-acct-to-home-server to include the Realm in proxy.conf
proxy.conf ( Realm definition )
home_server myFortigate {
type = acct
ipaddr = <IP address of Fortigate Interface Running Radius>
port = 1813
secret = superSecret
}
home_server_pool myFortigatePool {
type = fail-over
home_server = myFortigate
}
realm myFortigateRealm {
acct_pool = myFortigatePool
nostrip
}
copy-acct-to-home-server entry
preacct {
preprocess
update control {
Proxy-To-Realm := myFortigateRealm
}
suffix
}
After I run the freeradius -X, I also run tcpdump from a new session
tcpdump -ni eth01 port 1812 or port 1813
and get the following log
15:03:40.225570 IP RADIUS_PROXY_IP.56813 > FORTIGATE_INTERFACE_IP.1813: RADIUS, Accounting-Request (4), id: 0x31 length: 371
15:03:40.236155 IP FORTIGATE_INTERFACE_IP.1813 > RADIUS_PROXY_IP.56813: RADIUS, Accounting-Response (5), id: 0x31 length: 27
Which basically shows it is sending the account request to fortigate radius server and receiving the accounting response.
But strangely freeradius -X debug output shows a request time out for the same radius server on Fortigate and it ultimately tags the server as zombie
Starting proxy to home server FORTIGATE_INTERFACE_IP port 1813
(14) Proxying request to home server FORTIGATE_INTERFACE_IP port 1813 timeout 30.000000
Waking up in 0.3 seconds.
(14) Expecting proxy response no later than 29.667200 seconds from now
Waking up in 3.5 seconds.
and Finally it gives up
25) accounting {
(25) [ok] = ok
(25) } # accounting = ok
(25) ERROR: Failed to find live home server: Cancelling proxy
(25) WARNING: No home server selected
(25) Clearing existing &reply: attributes
(25) Found Post-Proxy-Type Fail-Accounting
(25) Post-Proxy-Type sub-section not found. Ignoring.
So the situation is the Radius proxy is sending accounting packets to Fortigate Radius server (could be seen in both freeradius and fortigate logs)
tcpdump shows that Radius proxy is receiving accounting response from the fortigate, but for some reason freeradius process doesn't recognize (or can not read) accounting response. It may be some interoperability issue or I have missed to set some flag. Requesting help from the experts to isolate and rectify the issue.

mod_fcgid: read data timeout in 45 seconds and Premature end of script headers: index.php

One of my website clients had issues while placing orders.
When I checked my error log I could see this :
[warn] mod_fcgid: read data timeout in 45 seconds, referer: https://myDomain/cart
[error] Premature end of script headers: index.php, referer: https://myDomain/cart
What does this error mean? What should I do to eliminate this error? Are there any settings to be changed in Plesk Control panel? will it be solved if I change 'max_execution_time' in 'Php settings' to 3600?
I am using Plesk 12.0.18, CentOS 5.11
The error means that website code in index.php file fails to be executed in the time limit, which set for Apache FastCGI module and/or PHP.
Most likely, there is an error in the index.php, which makes it inoperable at all. In this case, you should increase the PHP error reporting level in Plesk > Domains > example.com > PHP Settings and review the script itself.
Less likely that script is meant to take a long time to execute. In this case, you may simply increase timeout via Plesk. To set 120 seconds instead of default 45, do the following:
1. Set max_execution_time to 120 in Plesk > Domains > example.com > PHP settings.
2. Increase FastCGI timeout by adding the following Apache dirctives in Plesk > Domains > example.com > Apache & Nginx settings > Additional Apache directives:
<IfModule mod_fcgid.c>
FcgidIOTimeout 120
</IfModule>

uWSGI as a standalone http server with lua

I'm trying to set up uWSGI to run as a standalone server running a simple LUA script(right now, as a POC, using the hello world from http://uwsgi-docs.readthedocs.org/en/latest/Lua.html).
Here's my uwsgi.ini file:
[uwsgi]
master = true
workers = 1
threads = 8
listen = 4096
max-request = 512
pidfile = /uwsgi/logs/uwsgi.pid
procname-master = uWSGI master
auto-procname = true
lua = /uwsgi/hello.lua
socket-timeout = 30
socket = /uwsgi/uwsgi_1.sock
http = 127.0.0.1:80
http-to = /uwsgi/uwsgi_1.sock
When sending a web request, an empty response is received, and uWSGI process outputs:
-- unavailable modifier requested: 0 --
I've read this usually means a plugin is missing, however, LUA plugin is installed, and when doing the same but through NGINX everything works fine, which means there's no problem loading LUA.
Any help please?
Thanks.
Somebody told me I had to add http-modifier1 = 6 and now it works.
Still don't understand what does '6' mean, but whatever.

Resources