limit_req in NGINX in Docker using webdevops/php-nginx - docker

I don't suppose anyone would be prepared to outline for me how I might go about setting values for limit_req in NGINX in a Docker container using webdevops/php-nginx. I'm getting very occasional rate limit issues with a React/Laravel application and would like to try adjusting some rate limit settings.
The customization section of the webdevops/php-nginx docs suggests that to set "global configuration options the directory /opt/docker/etc/nginx/conf.d can be used. For vhost configuration options the directory /opt/docker/etc/nginx/vhost.common.conf can be used". However, I'm not sure how I bring that into my Docker process:
docker build --file Dockerfile.prod -t ghcr.io/theotherdy/laravel-xmap-php-nginx:latest .
using Dockerfile.prod:
FROM webdevops/php-nginx:8.1-alpine
# These ENV variables refer to options in webdevops/php-nginx - https://dockerfile.readthedocs.io/en/latest/content/DockerImages/dockerfiles/php-nginx.html
ENV WEB_DOCUMENT_ROOT=/app/public
ENV PHP_DISMOD=bz2,calendar,exiif,ffi,intl,gettext,ldap,imap,pdo_pgsql,pgsql,soap,sockets,sysvmsg,sysvsm,sysvshm,shmop,xsl,zip,gd,apcu,vips,yaml,imagick,mongodb,amqp
# sets working directory for any future actions
WORKDIR /app
# ie copying from wherever docker is run to WORKDIR
COPY . .
COPY composer.lock composer.lock
COPY .env.prod .env
# recommended optimization from: https://laravel.com/docs/9.x/deployment
RUN composer install --no-interaction --optimize-autoloader --no-dev
RUN php artisan key:generate
RUN php artisan config:cache
RUN php artisan route:cache
RUN php artisan view:cache
# Ensure all of our files are owned by the same user and group.
RUN chown -R application:application .
and docker-compose.yml:
version: "3" services:
app:
image: ghcr.io/theotherdy/laravel-xmap-php-nginx:latest
ports:
- '9000:80'
volumes:
- ./storage:/app/storage
#env_file: '.env'
depends_on:
- db
restart: always
db:
image: 'mysql/mysql-server:8.0'
container_name: xmap-db
environment:
MYSQL_ROOT_PASSWORD: 'xx'
MYSQL_ROOT_HOST: "%"
MYSQL_DATABASE: 'xx'
MYSQL_USER: 'xx'
MYSQL_PASSWORD: 'xx'
MYSQL_ALLOW_EMPTY_PASSWORD: 0
volumes:
- db-data:/var/lib/mysql
restart: always
volumes:
db-data:
Any advice/pointers very gratefully received!

Related

Flask app running in docker container cannot be accessed: ERR_CONNECTION_RESET

I've found a lot of questions on this topic. Always the answer was to use the '0.0.0.0' IP-adress. But I'm already doing this and still I get the error.
So I'm running a docker compose file that runs a database and a flask front end. The dockerfile runs fine on my own computer but on the server in the cloud I am getting this error.
This code launches the application:
app.run(host="0.0.0.0", port=5000, ssl_context="adhoc", debug=cfg.app_debug_mode)
This is my docker compose file:
version: "3.8"
services:
app:
image: app_image #the beginning is the unique uri of my amazon qccount. Then follows the repository name (ratio) and then the tag of the image (app) https://docs.aws.amazon.com/AmazonECR/latest/userguide/docker-push-ecr-image.html
build: ./app
links:
- database
ports:
- "5000:5000"
environment:
AM_I_IN_A_DOCKER_CONTAINER: 'Yes'
CONFIG_NAME: 'config' #Name of the config file to use
database:
image: database_image
container_name: database
build: ./sql
restart: always
ports:
- "32000:32000"
environment:
MYSQL_ROOT_HOST: '%' #This allows the root user to access the database from any ip. For some reason amazon requires it.
MYSQL_ROOT_PASSWORD: 'zv2yRCt79AsGvz'
MYSQL_DATABASE: 'education'
volumes:
- ratio_volume:/var/lib/mysql #use a named volume for the database.
networks:
default:
name: my-network
volumes:
ratio_volume:
This is the docker file:
FROM python:3
RUN pip3 install --upgrade pip
WORKDIR /app
COPY . /app
RUN pip3 --no-cache-dir install -r requirements.txt
EXPOSE 5000
ENTRYPOINT ["python3"]
CMD ["/app/run.py"]
What am I doing wrong? The error is also not really helpful.. I am not getting any errors in the console, just the browser is complaining.

docker-compose: how to automatically propagate changes (both frontend and backend)?

In Docker Compose, we have two services (a backend in Flask and a frontend in React) running at the same time in different directories. What are best practices for automatically updating the frontend service or backend service when ha change to the respective code is made?
In our case, we have:
frontend/
index.html
docker-compose.yml
Dockerfile
src
App.js
index.js
..
And our backend is:
backend/
app.py
Dockerfile
docker-compose.yml
This is our docker-compose.yml file:
version: '3.8'
services:
frontend:
image: node:alpine
build:
context: ../frontend
dockerfile: ../frontend/Dockerfile
command: npm start
depends_on:
- database # dont start until the database is up
- app
ports:
- 3000:3000
volumes:
- .:/frontend
app:
image: python:3.9
build:
context: .
dockerfile: ./Dockerfile
command: app.py
depends_on:
- database # dont start until the database is up
ports:
- 8080:8080
environment:
- PGPASSWORD=magical_password
- POSTGRESQL_PASSWORD=magical_password
- POSTGRESQL_HOST=backend-database-1
- POSTGRESQL_USER_NAME=unicorn_user
- LOCAL_ENVIRONMENT=True
- FLASK_ENV=development
- REPLICATE_API_TOKEN
volumes:
- .:/app
database:
image: "postgres" # use latest official postgres version
env_file:
- database.env # configure postgres
volumes:
- database-data:/var/lib/postgresql/data/ # persist data even if container shuts down
- ./schema.sql:/docker-entrypoint-initdb.d/schema.sql
ports:
- "5432:5432"
volumes:
database-data: # named volumes can be managed easier using docker-compose
Typically, we reload the app (on change) almost instantly from a volume in the volume's section. This approach correctly changes the backend service when the backend code is changed, but not the frontend service. Also, we have 2 docker-compose files, one in frontend, one in backend, which we hope to somehow learn how to consolidate.
Edit: These are the logs that work for the backend (app_1 is the backend) but do not work for the frontend:
app_1 | * Detected change in '/app/app.py', reloading
app_1 | environ({'PATH': '/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'HOSTNAME': '***', 'PGPASSWORD': '***', 'POSTGRESQL_PASSWORD': 'magical_password', 'POSTGRESQL_HOST': 'backend-database-1', 'POSTGRESQL_USER_NAME': '***', 'LOCAL_ENVIRONMENT': 'True', 'FLASK_ENV': 'development', 'LANG': 'C.UTF-8', 'GPG_KEY': '***', 'PYTHON_VERSION': '3.9.13', 'PYTHON_PIP_VERSION': '22.0.4', 'PYTHON_SETUPTOOLS_VERSION': '58.1.0', 'PYTHON_GET_PIP_URL': 'https://github.com/pypa/get-pip/raw/6ce3639da143c5d79b44f94b04080abf2531fd6e/public/get-pip.py', 'PYTHON_GET_PIP_SHA256': '***', 'HOST': '0.0.0.0', 'PORT': '8080', 'HOME': '/root', 'KMP_INIT_AT_FORK': 'FALSE', 'KMP_DUPLICATE_LIB_OK': 'True', 'WERKZEUG_SERVER_FD': '3', 'WERKZEUG_RUN_MAIN': 'true'})
app_1 | * Restarting with stat
app_1 | * Tip: There are .env or .flaskenv files present. Do "pip install python-dotenv" to use them.
app_1 | * Debugger is active!
app_1 | * Debugger PIN: 203-417-897
Edit 2: We followed the link suggested in the comments. We attempted setting both WATCHPACK_POLLING and CHOKIDAR_USEPOLLING to “true” but no luck. And we refactored our docker-compose file to be outside the directories like so:
docker-compose.yml
frontend/
index.html
Dockerfile
src
App.js
index.js
..
backend/
app.py
Dockerfile
Here is the new docker-compose
version: '3.8'
services:
frontend:
image: node:alpine
build:
context: ./frontend
cache_from:
- node:alpine
dockerfile: ./Dockerfile
command: npm start
depends_on:
- database # dont start until the database is up
- app
ports:
- 3000:3000
environment:
- CHOKIDAR_USEPOLLING="true"
volumes:
- /app/node_modules
- ./frontend:/app
app:
image: python:3.9
build:
context: ./backend
cache_from:
- python:3.9
dockerfile: ./Dockerfile
command: backend/app.py
depends_on:
- database # dont start until the database is up
ports:
- 8080:8080
environment:
- PGPASSWORD=magical_password
- POSTGRESQL_PASSWORD=magical_password
- POSTGRESQL_HOST=backend-database-1
- POSTGRESQL_USER_NAME=unicorn_user
- LOCAL_ENVIRONMENT=True
- FLASK_ENV=development
- REPLICATE_API_TOKEN
volumes:
- .:/app
database:
image: "postgres" # use latest official postgres version
env_file:
- backend/database.env # configure postgres
volumes:
- database-data:/var/lib/postgresql/data/ # persist data even if container shuts down
- ./schema.sql:/backend/docker-entrypoint-initdb.d/schema.sql
ports:
- "5432:5432"
volumes:
database-data: # named volumes can be managed easier using docker-compose
app:
And here are our Dockerfile for frontend
FROM node:alpine
RUN mkdir -p /frontend
WORKDIR /frontend
# We copy just the package.json first to leverage Docker cache
COPY package.json /frontend
RUN npm install --legacy-peer-deps
COPY . /frontend
# Bind to all network interfaces so that it can be mapped to the host OS
ENV HOST=0.0.0.0 PORT=3000
EXPOSE ${PORT}
CMD ["npm", "start"]
and backend
FROM python:3.9
# We copy just the requirements.txt first to leverage Docker cache
COPY ./requirements.txt /app/requirements.txt
WORKDIR /app
RUN pip3 install -r requirements.txt
COPY . /app
ENTRYPOINT [ "python" ]
# Bind to all network interfaces so that it can be mapped to the host OS
ENV HOST=0.0.0.0 PORT=8080
EXPOSE ${PORT}
# This runs the app in the container
CMD [ "app.py" ]
Still backend hot reloads and every time we make a change the change is detected and picked up and reflected in docker-compose immediately. But frontend requires a restart with this command docker-compose down --volumes && docker-compose build --no-cache && docker-compose up the output we get from docker-compose is no logs. It’s like docker-compose can’t see the changes.
Edit 3: Any help would be much appreciated!

Trouble running startup scripts for containers

I am trying to start three docker containers using docker-compose; a postgresql database, a rest api and a frontend web app.
Previously I had the rest api and the database working perfectly (ran migrations and started the server) but when I wanted to send requests to it from my web app I had trouble connecting to the docker network. All the discussion on the internet was centered around connecting a web app in a container to an api in another container and I didn't find any promising method to connect to it (besides enabling port forwarding in the kernel and exposing myself to the network?) so I just decided to package the web app in a container as well.
My directory structure:
ProjectName
|-> projectapi
|-> |-> api.docker
|-> |-> api_start.sh
|-> projectapp
|-> |-> front.docker
|-> |-> front_start.sh
|-> docker-compose.yml
The problem is running these startup scripts in the right context (tbh I only need to run a npm start and a python manage.py runserver).
I can't think of anything I haven't tried, but most of my efforts have been centered around mucking around with paths because for the longest time the issue was not finding the file which I think I have fixed.
This is the docker-compose file, I have tried having all sorts of command entries in here to run the desired startup script. I have also used entrypoint.
version: '3.7'
services:
db:
container_name: projectdb
image: postgres:9.6-alpine
restart: always
volumes:
- projectdb:/var/lib/postgresql/data/
environment:
POSTGRES_DB: projectdb
POSTGRES_PASSWORD: root
ports:
- "8001:5432"
api:
container_name: projectapi
build:
context: projectapi/
dockerfile: api.docker
ports:
- "8000:8000"
expose:
- "8000"
depends_on:
- db
restart: always
environment:
POSTGRES_DB: 'projectdb'
POSTGRES_USER: 'postgres'
POSTGRES_PASSWORD: 'root'
POSTGRES_HOST: 'db'
front:
container_name: projectapp
build:
context: projectapp/
dockerfile: front.docker
ports:
- "3000:3000"
restart:
always
volumes:
projectdb:
Then there are the Dockerfiles:
front
FROM node:8
RUN mkdir /projectapp
COPY $HOSTDIR/package*.json /projectapp/
RUN npm install /projectapp
COPY $HOSTDIR/* /projectapp/
ENTRYPOINT ["npm", "start", "/projectapp"]
back
FROM python:3.6-slim
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONBUFFERED 1
RUN mkdir /projectapi
COPY $HOSTDIR/requirements.txt /projectapi/
RUN pip install -r /projectapi/requirements.txt
COPY $HOSTDIR/* /projectapi/
CMD ["python", "manage.py", "migrate"]
That last line could be runserver as well. These are just some examples of the permutations I've gone through but at this point I feel the problem is some conceptual misunderstanding, I've read the docs.
The error messages are all different permutations of not finding the startup script. I think there was a point where I managed to run a startup script and the error became it couldn't find manage.py, at which point I started to look into how to write the script better than just: python manage.py runserver but didn't get very far.
Try using a WORKDIR. Your current run path doesn't have manage.py -
I tried changing it momentarily -
WORKDIR /projectapi
COPY requirements.txt ./
RUN pip install -r requirements.txt
COPY . .
CMD ["python", "manage.py", "runserver"]
I have solved this issue but still haven't gotten the network to work, however that's another question. While I solved the problem with the startup scripts it turned out not to be needed, here's the current state of things:
Directory structure is the same as before.
docker-compose.yml
version: '3.7'
services:
db:
container_name: compdb
image: postgres:9.6-alpine
restart: always
volumes:
- compdb:/var/lib/postgresql/data/
environment:
POSTGRES_DB: compdb
POSTGRES_PASSWORD: root
networks:
- internal
ports:
- "8001:5432"
api:
container_name: back
build:
context: back/
dockerfile: api.docker
entrypoint: ["python", "/back/manage.py", "runserver", "0.0.0.0:8000"]
networks:
- internal
ports:
- "8000:8000"
expose:
- "8000"
depends_on:
- db
restart: always
environment:
POSTGRES_DB: 'compdb'
POSTGRES_USER: 'postgres'
POSTGRES_PASSWORD: 'root'
POSTGRES_HOST: 'db'
front:
container_name: front
build:
context: front/
dockerfile: front.docker
entrypoint: ["npm","start", "--prefix", "/front/"]
networks:
- internal
ports:
- "3000:3000"
expose:
- "3000"
depends_on:
- api
restart:
always
staff:
container_name: staff
build:
context: staff/
dockerfile: staff.docker
entrypoint: ["npm","start","--prefix","/staff/"]
networks:
- internal
ports:
- "3006:3006"
expose:
- "3006"
depends_on:
- api
restart:
always
volumes:
compdb:
networks:
internal:
front
FROM node:8
RUN mkdir /front
COPY package*.json /front/
RUN npm install /front
COPY . /front/
back
FROM python:3.6-slim
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONBUFFERED 1
RUN mkdir /back
COPY requirements.txt /back/
RUN pip install -r /back/requirements.txt
COPY . /back/
staff is similar to front.
The problem was solved by moving the build context into each directory with docker compose. Running startup scripts can be done by changing the entrypoint, however for local development attaching to the container to run migrations or similar is more convenient.

docker-compose running 2 services with dockerfiles, The task "phx.server" could not be found, main.go: no such file or directory

I have an issue running my docker-compose.yml file with 4 services. They are my go microservice, phoenix web server, mongodb and redis images.
I specified in both my phoenix and golang dockerfiles to change working directory before running both services. I currently get the following errors when I do docker-compose up.
The task "phx.server" could not be found
main.go: no such file or directory
Here is my Dockerfile.go.development:
# base image elixer to start with
FROM golang:latest
# create app folder
RUN mkdir /goApp
COPY ./genesys-api /goApp
WORKDIR /goApp/cmd/genesys-server
# install dependencies
RUN go get gopkg.in/redis.v2
RUN go get github.com/gorilla/handlers
RUN go get github.com/dgrijalva/jwt-go
RUN go get github.com/gorilla/context
RUN go get github.com/gorilla/mux
RUN go get gopkg.in/mgo.v2/bson
RUN go get github.com/graphql-go/graphql
# run phoenix in *dev* mode on port 8080
CMD go run main.go
Here is my Dockerfile.phoenix.development:
# base image elixer to start with
FROM elixir:1.6
# install hex package manager
RUN mix local.hex --force
RUN mix local.rebar --force
# install the latest phoenix
RUN mix archive.install https://github.com/phoenixframework/archives/raw/master/phx_new.ez --force
# create app folder
RUN mkdir /app
COPY ./my_app /app
WORKDIR /app
# install dependencies
RUN mix deps.get
# run phoenix in *dev* mode on port 4000
CMD mix phx.server
Here is my docker-compose.yml file:
version: '3.6'
services:
go:
build:
context: .
dockerfile: Dockerfile.go.development
ports:
- 8080:8080
volumes:
- .:/goApp
depends_on:
- db
- redis
phoenix:
# tell docker-compose which Dockerfile it needs to build
build:
context: .
dockerfile: Dockerfile.phoenix.development
# map the port of phoenix to the local dev port
ports:
- 4000:4000
# mount the code folder inside the running container for easy development
volumes:
- .:/app
# make sure we start mongodb when we start this service
# links:
# - db
depends_on:
- db
- redis
environment:
GOOGLE_CLIENT_ID: ${GOOGLE_CLIENT_ID}
GOOGLE_CLIENT_SECRET: ${GOOGLE_CLIENT_SECRET}
FACEBOOK_CLIENT_ID: ${FACEBOOK_CLIENT_ID}
FACEBOOK_CLIENT_SECRET: ${FACEBOOK_CLIENT_SECRET}
db:
container_name: db
image: mongo:latest
volumes:
- ./data/db:/data/db
ports:
- 27017:27017
redis:
container_name: redis
image: redis:latest
ports:
- "6379:6379"
volumes:
- ./data/redis:/data/redis
entrypoint: redis-server
restart: always
For the error related to go microservice, Since the go binary is not found in PATH, you may need to set the GOPATH env variable via your docker file for go:
export GOPATH=

Override Dockerfile ARG from docker-compose.yml

I can't override the Dockerfile arg from docker-compose.yml when I set a default value at Dockerfile arg.
If the user just run de Dockerfile whiteout any parameter I want that docker don't break and at docker-compose.yml I want to set a better architecture.
This is my Dockerfile:
FROM python:3.6 as flask_api
LABEL maintainer 'https://about.me/leandro.garcias'
ARG DEBUG=False
# BD Config
ARG DATABASE_URL='sqlite:///data/app.db'
# Max register per page, when you try to get all
ARG MAX_PER_PAGE=25
# Collect log errors. https://sentry.io
ARG COLLECT_LOG_ERRORS=False
ARG SENTRY_DSN=''
RUN adduser api
USER api
WORKDIR /home/api
COPY requirements.txt manage.py contrib/boot.sh ./
COPY tests tests
COPY app app
RUN mkdir data
ENV PYTHONUNBUFFERED 1
ENV DEBUG $DEBUG
ENV DATABASE_URL $DATABASE_URL
ENV MAX_PER_PAGE $MAX_PER_PAGE
ENV COLLECT_LOG_ERRORS $COLLECT_LOG_ERRORS
ENV SENTRY_DSN $SENTRY_DSN
RUN python -m venv venv
RUN venv/bin/pip install -r requirements.txt
CMD bash boot.sh
EXPOSE 5000:5000
This is my docker-compose.yml:
version: '3'
volumes:
local_data:
data:
networks:
web:
app:
db:
services:
frontend:
image: nginx:1.13
volumes:
- ./contrib/nginx.conf:/etc/nginx/conf.d/default.conf
ports:
- 80:80
networks:
- web
- app
depends_on:
- app
app:
image: flask_api
restart: always
volumes:
- local_data:/home/api/data
networks:
- app
- db
depends_on:
- db
build:
context: .
args:
- DEBUG = False
# BD Config
- DATABASE_URL = postgres://postgres:#db:5432/people
# Max register per page, when you try to get all
- MAX_PER_PAGE = 25
# Collect log errors. https://sentry.io
- COLLECT_LOG_ERRORS = False
- SENTRY_DSN = ''
db:
image: postgres:9.6
volumes:
- data:/var/lib/postgresql/data
networks:
- db
environment:
- POSTGRES_DB=people
I need help ...
I found a solution! Just need to set environment values. :-)
app:
image: flask_api
environment:
- DEBUG=False
# BD Config
- DATABASE_URL=postgres://postgres:#db:5432/people
# Max register per page, when you try to get all
- MAX_PER_PAGE=25
# Collect log errors. https://sentry.io
- COLLECT_LOG_ERRORS=False
- SENTRY_DSN=''
Update 2022
ERROR: environment variable name 'DEBUG ' may not contain whitespace.
This might be the error you're getting. There shouldn't be spaces around the env. variable.
If we are passing args to the build from docker-compose, it would override the default arg value specified in Dockerfile.

Resources