nginx unable to forward request to a service running in a different container - docker

I want to use nginx reverse proxy as an APIGateway in my microservice architecture
Problem: Nginx is unable to proxy_pass to my payment_service running in a different container. However, when I try to curl payment_service:3000 from inside nginx container, it works. So the network is ok.
docker-compose.yml
version: '3'
services:
payment_service:
container_name: payment_service
build: ./payment
ports:
- "3000:3000"
volumes:
- ./payment:/usr/app
networks:
- microservice-network
api_gateway:
image: nginx:latest
container_name: api_gateway
restart: always
volumes:
- ./default.conf:/etc/nginx/conf.d/default.conf
ports:
- 8080:8080
- 443:443
depends_on:
- payment_service
networks:
- microservice-network
networks:
microservice-network:
driver: bridge
default.conf
upstream payment_server {
server payment_service:3000 max_fails=10;
}
server {
listen 8080;
location /api/v1/payment {
proxy_pass http://payment_server;
}
}
Payment service is working fine when I directly access it using http://localhost:3000
But dont work with http://localhost:8080/api/v1/payment

According to the docs
If proxy_pass is specified without a URI, the request URI is passed to the server in the same form as sent by a client when the original request is processed, or the full normalized request URI is passed when processing the changed URI...
I don't know what your payment service is expecting, but I'm assuming you're trying to hit the root path. You need to add a trailing slash to the proxy_pass:
location /api/v1/payment {
proxy_pass http://payment_server/;
}
otherwise the request will be made as
http://payment_service:3000/api/v1/payment

Related

how can I reverse proxy request to server using caddy

I am implementing reverse proxy using caddy in docker container. i have docker-compose file in which i specifed three service caddy, nginx, httpd. i also configured caddyfile in which i redirected my request to servers. when i run '$docker-compose up' all the containers working perfectlly fine. but when i go to
http://localhost/ - show blank white page
http://localhost:8080/ - this site cant be reach
http://localhost:8081/ - this site cant be reach
nginx_container_IP - show nginx page
httpd_container_IP - show httpd page
docker-compose.yaml
version: '3'
services:
proxy:
image: caddy:2.3.0-alpine
ports:
- "80:80"
volumes:
- ./Caddyfile:/etc/caddy/Caddyfile
service1:
image: nginx:latest
expose:
- "8080"
service2:
image: httpd:latest
expose:
- "8081"
Caddyfile
hostname:80 {
reverse_proxy service1:8080
reverse_proxy service2:8081
}
Caddy container logs
{"level":"info","ts":1675247772.029432,"msg":"using provided configuration","config_file":"/etc/caddy/Caddyfile","config_adapter":"caddyfile"}
{"level":"info","ts":1675247772.033348,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["localhost:2019","[::1]:2019","127.0.0.1:2019"]}
{"level":"info","ts":1675247772.033649,"logger":"http","msg":"server is listening only on the HTTP port, so no automatic HTTPS will be applied to this server","server_name":"srv0","http_port":80}
{"level":"info","ts":1675247772.0337453,"logger":"tls.cache.maintenance","msg":"started background certificate maintenance","cache":"0xc0002f52d0"}
{"level":"info","ts":1675247772.0342908,"logger":"tls","msg":"cleaned up storage units"}
{"level":"info","ts":1675247772.034485,"msg":"autosaved config","file":"/config/caddy/autosave.json"}
{"level":"info","ts":1675247772.0345104,"msg":"serving initial configuration"}
I want that when i request to localhost my request should be forwarded to nginx and httpd backend servers.

simple nginx reverse proxy not working in docker compose

fhsmgr proxy works at / but not any other location with 404
fhsdir proxy gives 404 at /dir, though when i browse directly to it on localhost:5000 i get the expected output, so the host is up and running. also, nginx does not complain about invalid host and exit like i've seen it do before.
i have tried trailing '/' so '/dir/' to no avail.
i have tried putting fhsmgr at '/mgr' and i get the expected 404 at index '/' but then 404 again at '/mgr'.
i have tried without proxy_redirect off; as well.
i have removed the upstream statements and just directly put in container names
seemingly the only thing it'll let me proxy is at '/', though i know i've proxied to other servers at different location paths in setups like this before.
-- docker compose
version: "3.7"
services:
fhsmgr:
build: fhsmgr
restart: always
fhsdir:
build: fhsdir
restart: always
ports:
- 5000:5000
nginx:
build: nginx
restart: always
ports:
- 80:80
environment:
- NGINX_ENVSUBST_OUTPUT_DIR=/etc/nginx
- FHSMGR_HOST=fhsmgr
- FHSMGR_PORT=5000
- FHSDIR_HOST=fhsdir
- FHSDIR_PORT=5000
-- nginx conf
events {}
http {
# upstream fhsmgr {
# server ${FHSMGR_HOST}:${FHSMGR_PORT};
# }
# upstream fhsdir {
# server ${FHSDIR_HOST}:${FHSDIR_PORT};
# }
# a simple reverse-proxy
server {
listen 80 default_server;
location / {
proxy_pass http://fhsmgr:5000;
proxy_redirect off;
}
location /dir {
proxy_pass http://fhsdir:5000;
proxy_redirect off;
}
}
}
i am modifying this project https://github.com/AwsGeek/lightsail-containers-nginx to get it to work for my use case. Haven't gotten to lightsail, just using docker compose locally
You miss the container_name in your docker-compose,try this
version: "3.7"
services:
fhsmgr:
build: fhsmgr
restart: always
container_name: fhsmgr # allow other containers to access by container_name
fhsdir:
build: fhsdir
restart: always
container_name: fhsdir
ports:
- 5000:5000
nginx:
build: nginx
restart: always
ports:
- 80:80
environment:
- NGINX_ENVSUBST_OUTPUT_DIR=/etc/nginx
- FHSMGR_HOST=fhsmgr
- FHSMGR_PORT=5000
- FHSDIR_HOST=fhsdir
- FHSDIR_PORT=5000

setting nginx container as a proxy for another app

I am trying to setup an nginx container that will show at the "http://server_ip/" path the nginx html page and on the "/app" path the tutum/hello-world container. as a follow up, want to be able to get to the "hello-world" container only from the "http://server_ip/app" path and not via http://server_ip:1500.
I created the following docker-compose:
version: '3'
services:
proxy:
container_name: proxy
image: nginx
ports:
- "80:80"
volumes:
- $PWD/html:/usr/share/nginx/html
- $PWD/config/nginx.conf:/etc/nginx/conf.d/nginx.conf
networks:
- backend
webapp:
container_name: webapp
image: tutum/hello-world
ports:
- "1500:80"
networks:
- backend
networks:
backend:
then I have the following nginx.conf file:
server {
listen 80; # not really needed, but more informative
location = / {
root /usr/share/nginx/html;
}
location = /app/ {
proxy_pass http://localhost:1500/;
}
}
If I try to get to each of the containers by their http://server_ip:PORT, I get there. if I try http://server_ip/app I get "404 not found". what am I missing? did I put the conf file in the wrong folder? how do I limit the availability of the "hello-world" only to the "http://server_ip/app" path and not via "http://server_ip:1500".
Your containers using the "backend" docker network, as you stated in the compose file.
Inside that they reach each other with the service names, so from the "proxy" service you can reach "webapp" service on http://webapp (or http://webapp:80) and from the "webapp" service you can reach "proxy" on http://proxy or (http://proxy:80).
On your computer if you type http://localhost:1500/ you will reach the webapp service and if you type http://localhost:80/ you will reach proxy service.
The port mapping 1500:80 means that your computer 1500 port is mapped to the webapp container 80 port.
So in nginx.conf do this:
proxy_pass http://webapp:80/;
Also if you want to make your webapp not accessible from your host on localhost:1500 remove the ports part in the webapp service spec:
version: '3'
services:
proxy:
container_name: proxy
image: nginx:1.11
ports:
- "80:80"
volumes:
- $PWD/html:/usr/share/nginx/html
- $PWD/nginx.conf:/etc/nginx/conf.d/default.conf
networks:
- backend
webapp:
container_name: webapp
image: tutum/hello-world
networks:
- backend
networks:
backend:

Vue container unable to access URL from app but accessible from container's bash

I have 3 different containers which are as below
1. UI Container having Vue SPA
2. Backend Container having Spring Boot application
3. MYSql DB container
below is my docker-compose file
version: '3'
services:
mysqldb:
image: mysql:latest
container_name: mysqldb
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: simple-bug
ports:
- "3306:3306"
simple-bug-back:
build: ./..
container_name: simple-bug-back
ports:
- "8082:8082"
restart: on-failure
depends_on:
- mysqldb
environment:
SPRING_PROFILES_ACTIVE: dev
SPRING_DATASOURCE_URL: jdbc:mysql://mysqldb/simple-bug
simple-bug-ui:
build: ./../ui
ports:
- "8080:8080"
depends_on:
- simple-bug-back
Now I am trying to access
http://simple-bug-back:8082
in vue application but it is giving net::ERR_NAME_NOT_RESOLVED but if i try the same inside container i.e.
docker exec -it docker_simple-bug-ui_1 sh
and execute curl command inside that for above URL I am getting the response
Can anyone help me what am I missing
The reason is because your host doesn't know the container by this domain, this domain is valid only into network cretated for to containers by docker-compose, if you want use this domain you should add it into your /etc/hosts file like this (because you have exposed the port 8082):
127.0.0.1 simple-bug-back
To resolve the above issue, I used Nginx as my server (https://v2.vuejs.org/v2/cookbook/dockerize-vuejs-app.html) and used it's reverse proxy. I levarage the .env file to route all my request to particular uri for all my backend call (in my case /back/) and below is my nginx.conf file
upstream spring_back_8082 {
server simple-bug-back:8082;
}
server {
..
..
location / {
try_files $uri $uri/ =404;
}
location /back/ {
proxy_pass http://spring_back_8082/;
}
}
Now to call backend my url is /back/...

GET http://api:1337/games net::ERR_NAME_NOT_RESOLVED for nuxt.js pages using asyncData

I have somewhat complicated setup with docker. Everything's working as expected except I have this weird problem.
Visiting index page or /pages/_id pages I have no errors. But when I try to open /other-page it crashes. All are using the same API url.
Error found in the console when opening /other-page:
GET http://api:1337/games net::ERR_NAME_NOT_RESOLVED
Not sure what to do, any suggestions?
nuxt.config.js
axios: {
baseURL: 'http://api:1337'
},
docker-compose.yml
version: '3'
services:
api:
build: .
image: strapi/strapi
environment:
- APP_NAME=strapi-app
- DATABASE_CLIENT=mongo
- DATABASE_HOST=db
- DATABASE_PORT=27017
- DATABASE_NAME=strapi
- DATABASE_USERNAME=
- DATABASE_PASSWORD=
- DATABASE_SSL=false
- DATABASE_AUTHENTICATION_DATABASE=strapi
- HOST=api
- NODE_ENV=production
ports:
- 1337:1337
volumes:
- ./strapi-app:/usr/src/api/strapi-app
#- /usr/src/api/strapi-app/node_modules
depends_on:
- db
restart: always
links:
- db
nuxt:
# build: ./app/
image: "registry.gitlab.com/username/package:latest"
container_name: nuxt
restart: always
ports:
- "3000:3000"
links:
- api:api
command:
"npm run start"
nginx:
image: nginx:1.14.2
expose:
- 80
container_name: nginx
restart: always
ports:
- "80:80"
volumes:
- ./nginx:/etc/nginx/conf.d
depends_on:
- nuxt
links:
- nuxt
index.vue
...
async asyncData({ store, $axios }) {
const games = await $axios.$get('/games')
store.commit('games/emptyList')
games.forEach(game => {
store.commit('games/add', {
id: game.id || game._id,
...game
})
})
return { games }
},
...
page.vue
...
async asyncData({ store, $axios }) {
const games = await $axios.$get('/games')
store.commit('games/emptyList')
games.forEach(game => {
store.commit('games/add', {
id: game.id || game._id,
...game
})
})
return { games }
},
...
Nginx conf
upstream webserver {
ip_hash;
server nuxt:3000;
}
server {
listen 80;
access_log off;
connection_pool_size 512k;
large_client_header_buffers 4 512k;
location / {
proxy_pass http://webserver;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_max_temp_file_size 0;
}
UPDATE:
Tried what Thomasleveil suggested. Now I'm receiving following error:
nuxt | [2:09:35 PM] Error: connect ECONNREFUSED 127.0.0.1:80
So, it seems like now /api is being forwarded to 127.0.0.1:80. Not sure why ^^
nuxt.config.js
axios: {
baseURL: '/api'
},
server: {
proxyTable: {
'/api': {
target: 'http://localhost:1337',
changeOrigin: true,
pathRewrite: {
"^/api": ""
}
}
}
}
docker-compose.yml
version: '3'
services:
reverse-proxy:
image: traefik # The official Traefik docker image
command: --api --docker # Enables the web UI and tells Traefik to listen to docker
ports:
- "80:80" # The HTTP port
- "8080:8080" # The Web UI (enabled by --api)
volumes:
- /var/run/docker.sock:/var/run/docker.sock # listen to the Docker events
networks:
- mynet
api:
build: .
image: strapi/strapi
container_name: api
environment:
- APP_NAME=strapi-app
- DATABASE_CLIENT=mongo
- DATABASE_HOST=db
- DATABASE_PORT=27017
- DATABASE_NAME=strapi
- DATABASE_USERNAME=
- DATABASE_PASSWORD=
- DATABASE_SSL=false
- DATABASE_AUTHENTICATION_DATABASE=strapi
- HOST=api
- NODE_ENV=development
ports:
- 1337:1337
volumes:
- ./strapi-app:/usr/src/api/strapi-app
#- /usr/src/api/strapi-app/node_modules
depends_on:
- db
restart: always
networks:
- mynet
labels:
- "traefik.backend=api"
- "traefik.docker.network=mynet"
- "traefik.frontend.rule=Host:example.com;PathPrefixStrip:/api"
- "traefik.port=1337"
db:
image: mongo
environment:
- MONGO_INITDB_DATABASE=strapi
ports:
- 27017:27017
volumes:
- ./db:/data/db
restart: always
networks:
- mynet
nuxt:
# build: ./app/
image: "registry.gitlab.com/username/package:latest"
container_name: nuxt
restart: always
ports:
- "3000:3000"
command:
"npm run start"
networks:
- mynet
labels:
- "traefik.backend=nuxt"
- "traefik.frontend.rule=Host:example.com;PathPrefixStrip:/"
- "traefik.docker.network=web"
- "traefik.port=3000"
networks:
mynet:
external: true
Visiting index page or /pages/_id pages I have no errors. But when I try to open /other-page it crashes.
To reformulate:
I have a main page at / that shows some links targeting pages at /pages/_id (where _id is a valid game id)
When I open / or /pages/_id, the content shows up
But if I click a link from page / targeting /pages/xxx (where xxx is a valid id), I got an error
Furthermore if I refresh the page, I then see the content and not the error
content for those pages comes from an api server. Pages are supposed to fetch the content by calling the api server and then render the page contents with the response.
What's happening here?
AsyncData
The way asyncData works in nuxt.js is the following:
on first page load
the user enters the url http://yourserver/pages/123 in its browser
the nuxt web server handles the request, resolve the route and mount the vue component for that page
the asyncData method from the vue component is called from the nuxt.js server side
the nuxt.js server (not the user browser) then fetch the content by making different call to http://api:1337/games/123, receive the response and the content renders.
when the user clicks a link for another page
Something a bit different happens now.
The user is still on the page http://api:1337/games/123 which has a link to the main page listing all the games (http://yourserver/) and click it.
the browser does not load any new page. Instead, the user browser makes an ajax call to http://api:1337/games to try to fetch the new content. And fails due to a name resolution error
Why?
This is a feature brought to you by nuxt.js to speed up page content loading time. from the documentation, the important bit of information is:
asyncData is called every time before loading the page component. It will be called server-side once (on the first request to the Nuxt app) and client-side when navigating to further routes.
server-side means the call is made from the nuxt server to the api server
client-side means the call is made from the user browser to the api server
Now the fun part:
the nuxt server is running in a first container
the api server is running in a second container and is listening on port 1337
from the nuxt container, the url for calling the api server is http://api:1337/, and this works fine
from the user browser, calling http://api:1337 fails (net::ERR_NAME_NOT_RESOLVED) because the user computer does not know how to translate the domain name api to an IP address. And even if it could, this IP Address would be unreachable anyway.
What can be done?
You need to set up a reverse proxy that will forward requests made by the user browsers to url starting with http://yourserver/api/ to the api container on port 1337.
And you need to configure nuxt.js so that links to the api made client-side (from the user browser) use the url http://yourserver/api instead of http://api:1337/
And you need to configure nuxt.js so that it keeps calling http://api:1337 for calls made server-side.
adding a reverse proxy for calls made from nuxt (server-side)
Since you are using the nuxt.js Axios module to make calls to the api container, you are half way there.
The Axios module has a proxy option that can be set to true in nuxtjs.config.js
Bellow is an example of setting up a reverse proxy for your project using Traefik, but the documentation state that the proxy is incompatible with the usage of the baseURL option. The prefix option must be used instead.
Your nuxt.config.js should then look like this:
axios: {
prefix: '/api',
proxy: true
},
proxy: {
'/api/': {
target: 'http://localhost:1337',
pathRewrite: {
'^/api/': ''
}
}
},
This works fine from your development computer, where if strapi is running and responding at http://localhost:1337. But this won't work in a container because we there need to replace http://localhost:1337 with http://api:1337.
To do so, we can introduce an environment variable (STRAPI_URL):
axios: {
prefix: '/api',
proxy: true
},
proxy: {
'/api/': {
target: process.env.STRAPI_URL || 'http://localhost:1337',
pathRewrite: {
'^/api/': ''
}
}
},
We will later set the STRAPI_URL in the docker-compose.yml file.
adding a reverse proxy for calls made from the user browser (client-side)
Since I gave up on implementing reverse proxies with nginx when using docker, here's an example with Traefik:
docker-compose.yml:
version: '3'
services:
reverseproxy: # see https://docs.traefik.io/#the-traefik-quickstart-using-docker
image: traefik:1.7
command: --docker
ports:
- "80:80"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
api:
image: strapi/strapi
environment:
- ...
expose:
- 1337
labels:
traefik.frontend.rule: PathPrefixStrip:/api
traefik.port: 1337
nuxt:
image: ...
expose:
- 3000
command:
"npm run start"
labels:
traefik.frontend.rule: PathPrefixStrip:/
traefik.port: 3000
Now all HTTP requests made by the user browser to http://yourserver will be handled by the Traefik reverse proxy.
Traefik will configure forwarding rules by looking at labels starting with traefik. on the nuxt and api containers.
What changed?
You now have 2 reverse proxies:
one for server-side requests (the nuxt.js Proxy module)
one for client-side requests (Traefik)
It's not done yet
We now need to instruct the nuxt.js Proxy module that it must forward requests to http://api:1337/. We are going to use the STRAPI_URL environment variable for that.
And we need to instruct nuxt Axios module that the user browser must call the api on http://yourserver/api. This is done with the API_URL_BROWSER environment variable.
All together
nuxt.config.js
axios: {
prefix: '/api',
proxy: true
},
proxy: {
'/api/': {
target: process.env.STRAPI_URL || 'http://localhost:1337',
pathRewrite: {
'^/api/': ''
}
}
},
docker-compose.yml
version: '3'
services:
reverseproxy: # see https://docs.traefik.io/#the-traefik-quickstart-using-docker
image: traefik:1.7
command: --docker
ports:
- "80:80"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
api:
image: strapi/strapi
environment:
- ...
expose:
- 1337
labels:
traefik.frontend.rule: PathPrefixStrip:/api
traefik.port: 1337
nuxt:
image: ...
expose:
- 3000
command:
"npm run start"
environment:
NUXT_HOST: 0.0.0.0
STRAPI_URL: http://api:1337/
API_URL_BROWSER: /api
labels:
traefik.frontend.rule: PathPrefixStrip:/
traefik.port: 3000

Resources