Docker can not up Nginx and Nginx htpasswd never detecting - docker

Hi I am trying to establish ELK stack + Nginx(for load balancing and basic Auth) by using Docker. My question is : I couldn't use Nginx as loadbalancer and when I try to visit kibana web ui, Never asking to me any password. How can I rearrange my docker compose for security and load balancing? Iam using windows 10.
My file structure is below:
My docker-compose.yml:
version: '2'
services:
elasticsearch:
container_name: esc
image: esi:1.0.0
build: ./es
volumes:
- ./data/es:/usr/share/elasticsearch/data
ports:
- 9200:9200
expose:
- 9300
kibana:
container_name: kibanac
image: kibanai:1.0.0
build: ./kibana
links:
- elasticsearch
ports:
- 5601:5601
nginx:
image: nginx:latest
restart: unless-stopped
volumes:
- ./nginx/config:/etc/nginx/conf.d:ro,Z
- ./nginx/htpasswd.users:/etc/nginx/htpasswd.users:ro,Z
ports:
- "8890:8890"
depends_on:
- elasticsearch
- kibana
Docker deamons:
Nginx.conf:
upstream elasticsearch {
server localhost:9200;
keepalive 15;
}
upstream kibana {
server localhost:5601;
keepalive 15;
}
server {
listen 8888;
location / {
auth_basic "Protected Elasticsearch";
auth_basic_user_file /etc/nginx/htpasswd.users;
proxy_pass http://localhost:9200;
proxy_redirect off;
proxy_buffering off;
proxy_http_version 1.1;
proxy_set_header Connection "Keep-Alive";
proxy_set_header Proxy-Connection "Keep-Alive";
}
}
server {
listen 8889;
location / {
auth_basic "Protected Kibana";
auth_basic_user_file /etc/nginx/htpasswd.users;
proxy_pass http://localhost:5601;
proxy_redirect off;
proxy_buffering off;
proxy_http_version 1.1;
proxy_set_header Connection "Keep-Alive";
proxy_set_header Proxy-Connection "Keep-Alive";
}
}
Kibana.yml: ( calling for kibana ui-> localhost:5601)
server.name: kibana
server.host: "0"
elasticsearch.url: http://elasticsearch:9200
elasticsearch.username: elastic
elasticsearch.password: changeme
xpack.monitoring.ui.container.elasticsearch.enabled: true
Elasticsearch.yml: ( calling for elasticsearch-> localhost:9200)
http.host: 0.0.0.0
### x-pack functions
xpack.security.enabled: false
xpack.monitoring.enabled: true
xpack.graph.enabled: false
xpack.watcher.enabled: false
Dockerfile for Kibana:
FROM docker.elastic.co/kibana/kibana:6.6.2
COPY ./config/kibana.yml /opt/kibana/config/kibana.yml
Dockerfile for Elasticsearch:
FROM docker.elastic.co/elasticsearch/elasticsearch:6.6.2
COPY ./config/elasticsearch.yml
/usr/share/elasticsearch/config/elasticsearch.yml

please add ip address or domain name in nginx configuration which you are accessing Kibana web UI.
listen ip_address_or_domain_name:8889;
and:
proxy_pass http://kibana;
In docker-compose.yml file:
ports:
- "8890:8889"
Please note, its port 8889.
Access : http://ip_address_or_domain_name:8890

Related

how to setup pgadmin with docker compose and nginx?

I am trying to set up pgadmin with docker compose and nginx but there is something weird happened.
every time I enter the site, pgadmin will redirect to /browser and also replaces host to container name, which make me browsing https://pgadmin_container/browser,but sometimes I directly go to https://my_url.com/browser it works, is it bug or I am missing something?
here is the nginx config:
server {
listen 80;
server_name some_name;
limit_conn conn_limit_per_ip 10;
limit_req zone=req_limit_per_ip burst=10 nodelay;
location / {
resolver 127.0.0.11 valid=30s;
set $upstream_pgadmin pgadmin_container;
proxy_pass http://$upstream_pgadmin:80;
proxy_redirect off;
proxy_buffering off;
}
and here is the docker-compose contents:
pgadmin:
container_name: pgadmin_container
image: dpage/pgadmin4
environment:
PGADMIN_DEFAULT_EMAIL: someEmail
PGADMIN_DEFAULT_PASSWORD: somePassword
PGADMIN_CONFIG_SERVER_MODE: 'False'
volumes:
- ./pgadmin:/root/.pgadmin2
ports:
- "5050:80"
networks:
- shared
restart: unless-stopped
sorry for my bad English

Change nginx server name in Docker

I have a project running on docker. I use Nginx reverse proxy to run my app.
All works fine but trying to personalize the server_name on nginx but couldn't figure out how.
Docker yml file
I've added server name to /etc/hosts by docker
version: "3"
services:
nginx:
container_name: nginx
volumes:
- ./nginx/logs/nginx:/var/log/nginx
build:
context: ./nginx
dockerfile: ./Dockerfile
depends_on:
- menu-app
ports:
- "80:80"
- "433:433"
extra_hosts:
- "www.qr-menu.loc:172.18.0.100"
- "www.qr-menu.loc:127.0.0.1"
networks:
default:
ipv4_address: 172.18.0.100
menu-app:
image: menu-app
container_name: menu-app
volumes:
- './menu-app/config:/var/www/config'
- './menu-app/core:/var/www/core'
- './menu-app/ecosystem.json:/var/www/ecosystem.json'
- './menu-app/tsconfig.json:/var/www/tsconfig.json'
- './menu-app/tsconfig-build.json:/var/www/tsconfig-build.json'
- "./menu-app/src:/var/www/src"
- "./menu-app/package.json:/var/www/package.json"
build:
context: .
dockerfile: menu-app/.docker/Dockerfile
tmpfs:
- /var/www/dist
ports:
- "3000:3000"
extra_hosts:
- "www.qr-menu.loc:127.0.0.1"
- "www.qr-menu.loc:172.18.0.100"
networks:
default:
ipam:
driver: default
config:
- subnet: 172.18.0.0/24
And I have Nginx conf
server_names_hash_bucket_size 1024;
upstream local_pwa {
server menu-app:3000;
keepalive 8;
}
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name www.qr-menu.loc 172.18.0.100;
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
location / {
proxy_set_header X-Forwarded-For $remote_addr;
proxy_pass http://local_pwa/;
}
}
but unfortunately, app runs on localhost instead of www.qr-menu.loc
I couldn't figure out how to change server_name on Nginx.
This is a really, really late answer. The server_name directive tells nginx which configuration block to use on receipt of a request. Also see: http://nginx.org/en/docs/http/server_names.html
I think the docker-compose extra_hosts directive might only work for domain-name resolution within the docker network. In other words, on your computer that's running docker the name "www.qr-menu.loc" is not available, but in a running docker container that name should be available.

Migrating Nginx reverse proxy to traefik => Services do not get served correctly

I want to switch from Nginx as Reverse Proxy to traefik, since traefik offers sticky sessions, which I need in a Docker Swarm environment. This is part my Nginx Setup which worked fine:
location / {
proxy_pass http://127.0.0.1:5000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 600s;
proxy_redirect off;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
location /auth/ {
proxy_pass https://127.0.0.1:8443;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 600s;
proxy_redirect off;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
This is my traefik.toml:
debug = false
logLevel = "ERROR"
defaultEntryPoints = ["https","http"]
[entryPoints]
[entryPoints.http]
address = ":80"
[entryPoints.http.redirect]
entryPoint = "https"
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
cipherSuites = [
"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
"TLS_RSA_WITH_AES_256_GCM_SHA384"
]
[entryPoints.keycloak]
address = ":8443"
[entryPoints.shinyproxy]
address = ":5000"
[retry]
[docker]
exposedByDefault = false
[acme]
email = "langmarkus#hotmail.com"
storage = "acme/certs.json"
entryPoint = "https"
onHostRule = true
[acme.httpChallenge]
entryPoint = "http"
And this is my compose file:
version: "3.7"
services:
shinyproxy:
build: /home/shinyproxy
deploy:
#replicas: 3
user: root:root
hostname: shinyproxy
image: shinyproxy-example
labels:
- "traefik.enable=true" # Enable reverse-proxy for this service
- "traefik.frontend.rule=Host:analytics.data-mastery.com" # Domain name for the app
- "traefik.port=443"
ports:
- 5000:5000
keycloak:
image: jboss/keycloak
labels:
- "traefik.enable=true" # Enable reverse-proxy for this service
- "traefik.frontend.rule=Host:analytics.data-mastery.com" # Domain name for the app
- "traefik.port=443"
networks:
- sp-example-net
volumes:
- type: bind
source: /home/certs/fullchain.pem
target: /etc/x509/https/tls.crt
- type: bind
source: /home/certs/privkey.pem
target: /etc/x509/https/tls.key
- /home/theme/:/opt/jboss/keycloak/themes/custom/
environment:
- PROXY_ADDRESS_FORWARDING=true
- KEYCLOAK_USER=myadmin
- KEYCLOAK_PASSWORD=mypassword
ports:
- 8443:8443
reverseproxy:
image: traefik:v1.7.16
volumes:
- /var/run/docker.sock:/var/run/docker.sock # So that Traefik can listen to the Docker events
- ./traefik/traefik.toml:/traefik.toml # Traefik configuration file
- ./volumes/traefik-acme:/acme # Tell Traefik to save SSL certs here
command: --api # Enables the web UI
ports:
- "80:80" # The HTTP port
- "443:443" # The HTTPS port
- "8080:8080" # The web UI
networks:
sp-example-net:
driver: overlay
attachable: true
SSL is working, my keycloak service is running here: https://analytics.data-mastery.com:8443/auth/ . However, I want to archieve the same behaviour like with proxy_pass where I will not have to use ports in the URL. What do I have to change?
in case you want to keep using the old traefik version, you can use the below stack files (you can also get rid of the traefik.toml and use only CLI commands)
With the below stack file, you will be able to access shinyproxy on analytics.data-mastery.com and keycloak on analytics.data-mastery.com/auth The import thing here is the defined rule https://docs.traefik.io/routing/routers/
you also don't need to expose the ports for this service, traefik will use the internal ones
version: "3.7"
services:
shinyproxy:
build: /home/shinyproxy
deploy:
replicas: 3
user: root:root
hostname: shinyproxy
image: shinyproxy-example
labels:
- traefik.enable=true
- traefik.backend.loadbalancer.swarm=true
- traefik.backend=shinyproxy
- traefik.frontend.rule=Host:analytics.data-mastery.com;
- traefik.port=5000
- traefik.docker.network=sp-example-net
keycloak:
image: jboss/keycloak
labels:
- traefik.enable=true
- traefik.backend.loadbalancer.swarm=true
- traefik.backend=keycloak
- traefik.frontend.rule=Host:analytics.data-mastery.com;Path:/auth
- traefik.port=8443
- traefik.docker.network=sp-example-net
networks:
- sp-example-net
volumes:
- type: bind
source: /home/certs/fullchain.pem
target: /etc/x509/https/tls.crt
- type: bind
source: /home/certs/privkey.pem
target: /etc/x509/https/tls.key
- /home/theme/:/opt/jboss/keycloak/themes/custom/
environment:
- PROXY_ADDRESS_FORWARDING=true
- KEYCLOAK_USER=myadmin
- KEYCLOAK_PASSWORD=mypassword
reverseproxy:
image: traefik:v1.7.16
networks:
- sp-example-net
volumes:
- /var/run/docker.sock:/var/run/docker.sock # So that Traefik can listen to the Docker events
- ./traefik/traefik.toml:/traefik.toml # Traefik configuration file
- ./volumes/traefik-acme:/acme # Tell Traefik to save SSL certs here
command:
- '--docker'
- '--docker.swarmmode'
- '--docker.domain=analytics.data-mastery.com'
- '--docker.watch'
- '--accessLog'
- '--checkNewVersion=false'
- '--api'
- '--ping.entryPoint=http'
# if you want to get reid of the toml file at all
# - '--entrypoints=Name:http Address::80 Redirect.EntryPoint:https'
# - '--entrypoints=Name:https Address::443 TLS'
# - '--defaultentrypoints=http,https'
# - '--acme.entrypoint=https'
# - '--acme.email=langmarkus#hotmail.com'
# - '--acme.storage=/var/lib/traefik/acme.json'
# - '--acme.acmelogging=true'
# - '--acme.httpChallenge.entryPoint=http'
# - '--acme.domains=*.analytics.data-mastery.com,analytics.data-mastery.com'
ports:
- "80:80"
- "443:443"
- "8080:8080"
networks:
sp-example-net:
driver: overlay
attachable: true
if you want to jump directly to traefik2.1, here is a link that includes good examples for using it

Docker container's ports clashing with nginx that is not inside container

I have a docker-compose.yml as follows setup at my root. For context, I have a Ghost CMS blog hosted on a Digital Ocean droplet. I want to install Commento using Docker (an open source commenting solution), but as I'm routing my traffic through Cloudflare DNS, I require SSL on both the server side and the frontend side.
However, I installed Ghost through Digital Ocean's one click Ghost setup, which configured nginx to be the reverse proxy for my site. Nginx is NOT in the container (installed on server). Nginx listens on port 80 and 443. When I try docker-compose up, it says the following error:
Error starting userland proxy: listen tcp 0.0.0.0:443: bind: address already in use
Traefik cannot listen on the same ports at nginx (which is not within the container, but installed on the server itself). How can I fix this problem, and have my commento server reverse proxied through SSL as well? My docker-compose is as below:
version: '3.7'
services:
proxy:
restart: always
image: traefik
command:
- "--api"
- "--entrypoints=Name:http Address::80 Redirect.EntryPoint:https"
- "--entrypoints=Name:https Address::443 TLS"
- "--defaultentrypoints=http,https"
- "--acme"
- "--acme.storage=/etc/traefik/acme/acme.json"
- "--acme.entryPoint=https"
- "--acme.httpChallenge.entryPoint=http"
- "--acme.onHostRule=true"
- "--acme.onDemand=false"
- "--acme.email=changeme#example.com" # TODO: Replace with your email address
- "--docker"
- "--docker.watch"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./traefik/acme:/etc/traefik/acme
networks:
- web
ports:
- "80:80"
- "443:443"
labels:
- "traefik.enable=false"
server:
image: registry.gitlab.com/commento/commento:latest
ports:
- 8080:8080
environment:
COMMENTO_ORIGIN: https://commento.example.com # TODO: Replace commento.example.com with your domami$ COMMENTO_PORT: 8080
COMMENTO_POSTGRES: postgres://postgres:passwordexample#db:5432/commento?s$
depends_on:
- db
networks:
- db_network
- web
db:
image: postgres
environment:
POSTGRES_DB: commento
POSTGRES_USER: postgres
POSTGRES_PASSWORD: examplepassword #TODO: Replace STRONG_PASSWORD with th$ networks:
- db_network
volumes:
- postgres_data_volume:/var/lib/postgresql/data
volumes:
postgres_data_volume:
networks:
web:
external
db_network:
Here is my nginx server config under available sites:
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name example.com;
root /var/www/ghost/system/nginx-root; # Used for acme.sh SSL verification (https://acme.sh)
ssl_certificate /etc/letsencrypt/example.com/fullchain.cer;
ssl_certificate_key /etc/letsencrypt/example.com/example.com.key;
include /etc/nginx/snippets/ssl-params.conf;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $http_host;
proxy_pass http://127.0.0.1:2368;
}
location ~ /.well-known {
allow all;
}
client_max_body_size 50m;
}
Sorry, kind of new to this. Thank you!
docker-compose.yml
...
ports:
- "80:80"
- "443:443"
...
nginx/conf
...
listen 443 ssl http2;
listen [::]:443 ssl http2;
...
Nginx used HOST port 443, so you cannot reuse it on your docker-compose, you must another one that is free.

Docker service redirection based on url path

I am using docker swarm and deploying 3 tomcat services each of the running on 8443 within the container and on 8444,8445,8446 on host containers.
I am looking to use a proxy server running on 8443 which will redirect the incoming request to the corresponding service based on the url path
https://hostname:8443/a – > https://hostname:8444/a
https://hostname:8443/b – > https://hostname:8445/b
https://hostname:8443/c – > https://hostname:8446/c
My sample Docker-compose file
version: "3"
services:
tomcat1 :
image: tomcat:1
ports:
- "8446:8443"
tomcat2 :
image: tomcat:1
ports:
- "8444:8443"
tomcat3 :
image: tomcat:1
ports:
- "8445:8443"
I have explored traeffik and nginx but was not able to find to re route based on url. Any suggestions.
You could use traefik based in rule with labels Host and Path
http://docs.traefik.io/basics/#frontends
Something like
version: "3"
services:
traefik:
image: traefik
command: --web --docker --docker.swarmmode --docker.watch --docker.domain=hostname
ports:
- "80:80"
- "8080:8080"
- "443:443"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
deploy:
placement:
constraints: [node.role == manager]
restart_policy:
condition: on-failure
tomcat1:
image: tomcat:1
labels:
- traefik.backend=tomcat1
- traefik.frontend.rule=Host:hostname;PathPrefixStrip:/a
- traefik.port=8443
You can try the way I did it using nginx.
ON UBUNTU
Inside the /etc/nginx/sites-available you will find the default file.
Inside the server block add a new location block.
server {
listen 8443;
#this is a comment
location /a {
proxy_pass http://[::]:8444/.;
#i have commented these out because i don't know if you need them
#proxy_http_version 1.1;
#proxy_set_header Upgrade $http_upgrade;
#proxy_set_header Connection keep-alive;
#proxy_set_header Host $host;
#proxy_cache_bypass $http_upgrade;
#proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
#proxy_set_header X-Forwarded-Proto $scheme;
}
location /b {
proxy_pass http://[::]:8445/.;
}
location /c {
proxy_pass http://[::]:8446/.;
}
}

Resources