Followed guide ( https://michalklempa.com/2019/04/nifi-registry-nginx-proxy-tls-basic-auth/ ) to set up nginx basic auth, however instead of proxy for nifi-registry I set it up for nifi. Auth is working and page is accessible but somehow processor configure window not opening. The issue is due to nginx since direct access to nifi through HTTP exposed ports works ,just not behind nginx proxy.
below is the config I am using:
server {
listen 9988 ssl;
root /usr/share/nginx/html;
index index.html;
server_name _;
ssl_certificate /etc/nginx/server_cert.pem;
ssl_certificate_key /etc/nginx/server_key.pem;
ssl_client_certificate /etc/nginx/client_cert.pem;
ssl_verify_client optional;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
# enables server-side protection from BEAST attacks
ssl_prefer_server_ciphers on;
# Disabled insecure ciphers suite. For example, MD5, DES, RC4, PSK
ssl_ciphers "ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:HIGH:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4:#STRENGTH";
# -!MEDIUM:exclude encryption cipher suites using 128 bit encryption.
# -!LOW: exclude encryption cipher suites using 64 or 56 bit encryption algorithms
# -!EXPORT: exclude export encryption algorithms including 40 and 56 bits algorithms.
# -!aNULL: exclude the cipher suites offering no authentication. This is currently the anonymous DH algorithms and anonymous ECDH algorithms.
# These cipher suites are vulnerable to a "man in the middle" attack and so their use is normally discouraged.
# -!eNULL:exclude the "NULL" ciphers that is those offering no encryption.
# Because these offer no encryption at all and are a security risk they are disabled unless explicitly included.
# #STRENGTH:sort the current cipher list in order of encryption algorithm key length.
location / {
if ($ssl_client_verify = SUCCESS) {
set $auth_basic off;
}
if ($ssl_client_verify != SUCCESS) {
set $auth_basic "Restricted Content. Please provide Nifi Authentication:";
}
auth_basic $auth_basic;
auth_basic_user_file /etc/nginx/nginx.htpasswd;
proxy_pass http://172.18.0.77:8181/; # actual container ip/port of nifi
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-User $remote_user;
proxy_set_header Authorization "";
proxy_set_header X-ProxyScheme $scheme;
proxy_set_header X-ProxyHost $hostname;
proxy_set_header X-ProxyPort $server_port;
proxy_set_header X-ProxyContextPath "/";
}
}
I tried passing container ip of nifi/host/nginx for X-ProxyHost but instead of giving "Unable to communicate to nifi" immediately it spins for a while and eventually gives the same error. What needs to be modified here? Any help would be appreciated.
nginx noob here!
After much fiddling with multiple ip/hostname combinations I was able to fix it with below config changes.
Had to add nifi env properties to the docker-compose:
environment:
- NIFI_REMOTE_INPUT_HOST=<private ip of nifi container e.g. 172.18.0.77>
- NIFI_WEB_PROXY_CONTEXT_PATH=/
- NIFI_WEB_HTTP_HOST=<private ip of nifi container>
- NIFI_WEB_HTTP_PORT=8181
And for nginx config: modified proxy_set_header to "localhost" (since nginx server needed proxyHost defined as loopback server):
proxy_set_header X-ProxyHost localhost;
Hope this helps someone scratching their head who are in the same boat :)
Related
How to call GRPC Server which is located in docker container on Swarm cluster from NGINX reverse proxy?
GRPC Server in container/service called webui with kestrel development certificate installed
NGINX Proxy which is located outside the stack and routes access to Swarm stacks
GRPC Client is located on a separate virtual machine on another network, the browser page at https://demo.myorg.com is available
part nginx.conf
server {
listen 443 ssl;
server_name demo.myorg.com;
...
location / {
proxy_pass https://namestack_webui;
}
GRPC Client appsetting.json
{
"ConnectionStrings": {
"Database": "Data Source=Server_name;Initial Catalog=DB;User Id=user;Password=pass;MultipleActiveResultSets=True;"
}
...
"GRPCServerUri": "https://demo.myorg.com/",
...
}
}
Problem when connecting GRPC Client to Server, i get error
END] GetOpcDaServerSettingsQuery. Time spent: 7,7166ms
fail: Grpc.Net.Client.Internal.GrpcCall[6]
Error starting gRPC call.
System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
---> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception.
---> System.ComponentModel.Win32Exception (0x80090367): No common application protocol exists between the client and the server. Application protocol negotiation failed..
--- End of inner exception stack trace ---
Tried to write and specify a kestrel development certificate (for GRPC Client) that is loaded into the Swarm stack (namestack) through which the other containers in the stack are authenticated, the error is the same.
I understand that it is necessary to specify in appsetting.json the GRPC Server container address (https://namestack_webui), but it is behind NGINX, and I can only specify the GRPC host address (https://demo.myorg.com), tell me what is wrong?
The perfect solution for such a case was not found online.
I finally figured out and found a solution to my question, and I publish it for discussion.
If there are no comments against, then mark it as correct, at least it works for me and will work for YOU.
to proxy grpc connections through NGINX in the configuration, the location section must specify something similar to the url /PackageName.ServiceName/MethodName (This is indicated here by https://learn.microsoft.com/en-aspnetus/aspnet/core/grpc/troubleshoot?view=aspnetcor7.0#unable-to-start-aspnet-core-grpc-app-on-macos )
This URL can be checked with the developer or in the logs when grpc client connects
Should be used to proxy directive grpc_pass grpcs://namecontainer;
Should use http2 protocol.
So the correct configuration file for nginx in my case should look like this
server {
listen 443 ssl **http2**;
server_name demo.myorg.com;
ssl_certificate ...;
ssl_certificate_key ...;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers RC4:HIGH:!aNULL:!MD5:!kEDH;
add_header Strict-Transport-Security 'max-age=604800';
underscores_in_headers on;
large_client_header_buffers 4 16k;
location / {
proxy_pass https://name_container;
# Configuration for WebSockets
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_cache off;
# WebSockets were implemented after http/1.0
proxy_http_version 1.1;
# Configuration for ServerSentEvents
proxy_buffering off;
# Configuration for LongPolling or if your KeepAliveInterval is longer than 60 seconds
proxy_read_timeout 100s;
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-URL-SCHEME https;
}
location /App.Name.Api.Contract.ApiService/UpdateOpcDaTags {
grpc_pass grpcs://name_container;
}
}
Rails application(4.2) is hosted on nginx and serves at localhost:5478. The ip_hash in the code snippet below maintains the server request response consistency and works as expected.
upstream rails {
ip_hash;
To share the load, ip_hash was commented. Now the login for the user starts failing since passing of session cookie is required while works in similar way for Rails3. This is related to something around sticky session but unable to trace the exact way of handling it.
nginx.conf
upstream mongrel {
server 127.0.0.1:5469;
}
upstream rails {
#ip_hash;
server 127.0.0.1:5479;
server 127.0.0.1:5480;
server 127.0.0.1:5481;
server 127.0.0.1:5482;
}
location / {
# Setup redirection headers
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
# Pass the request thru
proxy_pass http://mongrel;
}
}
server {
listen 5478 default;
server_name _;
root "../games/public";
location ~ ^/assets/ {
root "../d2/public";
expires 1y;
add_header Cache-Control public;
add_header ETag "";
break;
}
I tried using consistent_hash $scheme $request_uri;
as suggested but consistent_hash as a directive is not recognized and fails. Let me know if any config change is required for nginx. I also found the same nginx config with ip_hash commented works for Rails3 application, not sure if this is related
There are two ways to do this, either:
you share your sessions between your backends, or
you pass a cookie to allow nginx to stick a client to an upstream server.
Let me know if any config change is required for nginx.
if you cannot modify the application, e.g. letting multiple application instances using a common storage, you can try to use sticky directive of nginx (>=1.5.7).
Using your example, it should be something like
http {
...
upstream rails {
server 127.0.0.1:5479;
server 127.0.0.1:5480;
server 127.0.0.1:5481;
server 127.0.0.1:5482;
sticky rails_sticky expires=1d domain=.rails.local path=/ httponly secure;
}
...
server {
listen 5478;
server_name rails.local;
root "../games/public";
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_pass http://rails;
}
...
}
...
}
You may need to adjust your configuration according to your environment.
Using sticky, nginx should check and bound a client using a sticky cookie called rails_sticky, if it's not yet bound. Bounding a client still checks whatever balancing method you set in upstream directive, weighted round-robin by default.
If client has been bound to a server, any subsequent requests will be forwarded to designated upstream server. In an event that designated upstream server cannot be used, nginx will re-bound the client to another server.
I'm receiving the error Authentication required after I login in the Wildfly 13 Management Console.
If I type a user or password wrong, it asks again, but if I type correctly it shows the page with the error message (so I assume the user and password are correct, but something else after that gives the error).
I'm using docker to run a nginx container and a wildfly container.
The nginx listens externally on port 9991 and proxy pass the request to the wildfly container, but it shows the error described before.
It just happens with the Wildfly Console, every other request proxied, even request proxied to a websocket or to Wildfly on port 8080, are done successfully.
The Wildfly container listens externally on port 9990 and I can access the console successfully in this port. If on docker I map the port "9992:9990" I still can access the console successfully through port 9992.
So, it seems that this is not related to docker, but to the Wildfly Console itself. Probably some kind of authentication that is not happening successfully when using a reverse proxy in the middle.
I have a demo docker project on https://github.com/lucasbasquerotto/pod/tree/0.0.6, and you can download the tag 0.0.6 that has everything setup to work with Wildfly 13 and nginx, and to simulate this error.
git clone -b 0.0.6 --single-branch --depth 1 https://github.com/lucasbasquerotto/pod.git
cd pod
docker-compose up -d
Then, if you access the container directly in http://localhost:9990 with user monitor and password Monitor#70365 everything works.
But if you access http://localhost:9991 with the same credentials, through the nginx reverse proxy, you receive the error.
My nginx.conf file:
upstream docker-wildfly {
server wildfly:9990;
}
location / {
proxy_pass http://docker-wildfly;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
I've also tried with:
proxy_set_header X-Forwarded-Proto $scheme;
And also with the Authorization header (just the 2nd line and also with both):
proxy_set_header Authorization $http_authorization;
proxy_pass_header Authorization;
And also defining the host header with the port (instead of just $host):
proxy_set_header Host $server_addr:$server_port;
I've tried the above configurations isolated and combined together. All to no avail.
Any sugestions?
Has anyone successfully accessed the Wildfly Console through a reverse proxy?
Update (2018-09-22)
It seems Wildfly uses a digest authentication (instead of basic).
I see the header in the console like the following:
Authorization: Digest username="monitor", realm="ManagementRealm", nonce="AAAAAQAAAStPzpEGR3LxjJcd+HqIX2eJ+W8JuzRHejXPcGH++43AGWSVYTA=", uri="/console/index.html", algorithm=MD5, response="8d5b2b26adce452555d13598e77c0f63", opaque="00000000000000000000000000000000", qop=auth, nc=00000005, cnonce="fe0e31dd57f83948"
I don't see much documentation about using nginx to proxy pass requests with digest headers (but I think it should be transparent).
One question I saw here in SO is https://serverfault.com/questions/750213/http-digest-authentication-on-proxied-server, but there is no answer so far.
I saw that there is the nginx non-official module https://www.nginx.com/resources/wiki/modules/auth_digest/, but in the github repository (https://github.com/atomx/nginx-http-auth-digest) it says:
The ngx_http_auth_digest module supplements Nginx's built-in Basic
Authentication module by providing support for RFC 2617 Digest
Authentication. The module is currently functional but has only been
tested and reviewed by its author. And given that this is security
code, one set of eyes is almost certainly insufficient to guarantee
that it's 100% correct. Until a few bug reports come in and some of
the ‘unknown unknowns’ in the code are flushed out, consider this
module an ‘alpha’ and treat it with the appropriate amount of
skepticism.
Also it doesn't seem to me allright to hardcode the user and pass in a file to be used by nginx (the authentication should be transparent to the reverse proxy in this case).
In any case, I tried it and it correctly asks me to authenticate, even if the final destination does not have a digest authentication, like when trying to connect to the wildfly site (not console), it asks when trying to connect to nginx (before proxying the request), then it forwards successfully to the destination, except in the case of wildfly console, it keeps asking me to authenticate forever.
So I think this is not the solution. The problem seems to be in what the nginx is passing to the Wildfly Console.
I had the same problem with the HAL management console v3.3 and 3.2
I could not get ngnix HTTPS working due to authentication errors, even though the page prompted http basic auth user and pass
This was tested in standalone mode on the same server
My setup was :
outside (https) -> nginx -> http://halServer:9990/
This resulted in working https but with HAL authentication errors (seen in the browsers console) the webpage was blank.
At first access the webpage would ask http basic auth credentials normally, but then almost all https requests would return an authentication error
I managed to make it work correctly by first enabling the HAL console https with a self signed certificate and then configuring nginx to proxy pass to the HAL HTTPS listener
Working setup is :
outside (https) -> nginx (https) -> https://halServer:9993/
Here is the ngnix configuration
server {
listen 80;
listen [::]:80;
listen 443 ssl;
listen [::]:443 ssl;
server_name halconsole.mywebsite.com;
# SSL
ssl_certificate /keys/hal_fullchain.pem;
ssl_certificate_key /keys/hal_privkey.pem;
ssl_trusted_certificate /keys/hal_chain.pem;
# security
include nginxconfig.io/security.conf;
# logging
access_log /var/log/nginx/halconsole.mywebsite.com.access.log;
error_log /var/log/nginx/halconsole.mywebsite.com.error.log warn;
# reverse proxy
location / {
# or use static ip, or nginx upstream
proxy_pass https://halServer:9993;
include nginxconfig.io/proxy.conf;
}
# additional config
include nginxconfig.io/general.conf;
include nginxconfig.io/letsencrypt.conf;
}
# subdomains redirect
server {
listen 443 ssl;
listen [::]:443 ssl;
server_name *.halconsole.mywebsite.com;
# SSL
ssl_certificate /keys/hal_fullchain.pem;
ssl_certificate_key /keys/hal_privkey.pem;
ssl_trusted_certificate /keys/hal_chain.pem;
return 301 https://halconsole.mywebsite.com$request_uri;
}
proxy.conf
proxy_http_version 1.1;
proxy_cache_bypass $http_upgrade;
# Proxy headers
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Forwarded $proxy_add_forwarded;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Forwarded-By $server_addr;
# Proxy timeouts
proxy_connect_timeout 60s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
The easiest way to enable https console is by using the console itself
generate a java JKS keystore using either the command line keytool or a GUI program
I like GUIs, so I used Key Store Explorer https://github.com/kaikramer/keystore-explorer
copy keystore file on the halServer server where it has read access (no need to keep it secret) i placed mine inside wildfly data dir in a "keystore" directory.
# your file paths might differ, don't copy paste
cp /home/someUser/sftp_uploads/managementKS /opt/wildfly/standalone/data/keystore/managementKS
set permissions
# your file paths might differ, don't copy paste
chown --recursive -H wildfly:wildfly /opt/wildfly/standalone/data/keystore
(use vpn) login to cleartext console http://halServer:9990/
add keystore : navigate :
configuration -> subsystems -> security (elytron) -> other settings (click view button)
stores -> keystore -> add
...
Name = managementKS
Type = JKS
Path = keystore/managementKS
Relative to = jboss.server.data.dir
Credential Reference Clear Text = keystore-password click Add
result in standalone.xml
<key-store name="managementKS">
<credential-reference clear-text="keystore-password"/>
<implementation type="JKS"/>
<file path="keystore/managementKS" relative-to="jboss.server.data.dir"/>
</key-store>
add key manager : navigate :
ssl -> key manager -> add
...
Name = managementKM
Credential Reference Clear Text = keystore-password
Key Store = managementKS
result in standalone.xml
<key-manager name="managementKM" key-store="managementKS">
<credential-reference clear-text="keystore-password"/>
</key-manager>
add ssl context : navigate :
ssl -> server ssl context -> add
...
Name = managementSSC
Key Manager = managementKM
...
Edit added : Protocols = TLSv1.2
save
result in standalone.xml
<server-ssl-contexts>
<server-ssl-context name="managementSSC" protocols="TLSv1.2" key-manager="managementKM"/>
</server-ssl-contexts>
go back
runtime -> server (click view button)
http management interface (edit)
set secure socket binding = management-https
set ssl context = managementSSC
save
restart wildfly
systemctl restart wildfly
I have an official Nginx image in which I added certificates and the .conf file so it listens over port 443 and 80. I just added on Nginx official dockerfile:
ADD gp-search2.conf /etc/nginx/conf.d/
ADD STAR.GREY.COM.crt /etc/ssl/certs/
ADD wildcard_grey.com.key /etc/ssl/certs/
I´m using Azure container instaces and I´m creating the container with this command successfully
az container create --resource-group RG --name nginx --image xxxxx.azurecr.io/api-s:nginx4 --cpu 1 --memory 1.5 --registry-username xxxxxx --registry-password xxxxxxxxxx --ip-address public --ports 443 --dns-name-label prod3
After this I get a container created on azure successfully with a public IP and with the FQDN that I provided prod3eastus.azurecontainer.io. Also I create in Dynect.net a new node for the domain we have: newcontainer.example.com and added there the public IP of the new container so that the valid certificates I have on container are ok with than domain.
If I access the container with the public FQDN or the IP that Azure provides I can access ok, but if I try to access with HTTPS I get:
This page isn’t working If the problem continues, contact the site owner. HTTP ERROR 400
and
*WARNING: cannot verify newcontainer.example.com's certificate, issued by ‘CN=Network Solutions OV Server CA 2,O=Network Solutions L.L.C.,L=Herndon,ST=VA,C=US’: Unable to locally verify the issuer's authority.*
Even though that:
the certificates are valid
the certificates are on the same path that I indicate on file.conf in Nginx container (if not container won´t be up).
This is the .conf file I have:
upstream searchapl {
server 40.x.x.x:8080 fail_timeout=0;
}
server {
listen 80;
server_name _;
return 301 https://$host$request_uri;
}
server {
listen 443 ssl;
server_name _;
#status_zone go-backend-servers;
ssl_certificate /etc/ssl/certs/STAR.client.COM.pem;
ssl_certificate_key /etc/ssl/certs/STAR.client.COM.key;
ssl_protocols TLSv1.2 TLSv1.1 TLSv1;
ssl_ciphers HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
proxy_pass http://searchapl;
proxy_redirect http:// https://;
# Socket.IO Support
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
# time out settings
proxy_connect_timeout 159s;
proxy_send_timeout 600;
proxy_read_timeout 600;
proxy_buffer_size 64k;
proxy_buffers 16 32k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;
proxy_pass_header Set-Cookie;
}
}
This Nginx is used as a reverse proxy that redirects to a Tomcat container that is working ok. Redirection works successfully if I enter container IP. It takes me to Tomcat. But over 443 I get certificates issue. What else can I check? Certificate and key have Read Access.
OK Now I move forward a little step but still getting error. I created the .pem file instead of .crt whit all the chain of primary certificates and intermediate + root.
Don´t get anymore error of unable to check certificate authority but now I´m getting:
***--2018-08-24 21:05:05-- https://xxxxxxxxxxxx.com/
Resolving xxxxxxxxxxxx.com (gp_searchv2.grey.com)... 23.x.x.x
Connecting to xxxxxxxxx.com (xxxxxxxxxx.com)|23.x.x.x|:443... connected.
HTTP request sent, awaiting response... 400
2018-08-24 21:05:05 ERROR 400: (no description).***
Is this somehow possible? Is it possible to do something like this in Ruby on top of Rack? I've seen there's websockets-rack but as far as I understand, that is only a rack module to serve ONLY websocket traffic not http also.
So basically, as the question states, is it possible to serve both protocols with just one server on the same port, instead of firing of something like Faye, websockets-rack or em-websockets?
Websockets are just an in-protocol upgrade of HTTP(s), so they are not normal TCP sockets but reuse the existing HTTP(S) connection (and thus use the same port). So, in theory it should work and from what I know it works with the Perl Mojolicious framework. But I don't know if it works work ruby/rack.
The short answer is - (AFAIK) no.
Currently, a ruby HTTP server (like rails or sinatra) and a websocket server are mutually exclusive.
After saying that, you could use a third party to emulate that. Specifically Ngnix. With Nginx you can listen to a single port, but, accroding to a path, decide whether you want to dispatch the request to the HTTP server or the Websocket server.
For example, you can run the HTTP server on port 3000, and the Websocket server on port 3020, and then configure the nginx.conf like this:
upstream http_app {
server 127.0.0.1:3000;
}
upstream websocket_app {
server 127.0.0.1:3020;
}
server {
listen 80;
server_name .example.com;
access_log /var/www/myapp.example.com/log/access.log;
error_log /var/www/myapp.example.com/log/error.log;
root /var/www/myapp.example.com;
index index.html;
location /web {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://http_app;
}
location /socket {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://websocket_app;
}
}
Now any request to http://www.example.com/web/... will reach the HTTP server, and any request to http://www.example.com/socket will reach the Websocket server.