Using docker-compose to build against an existing dockerfile with some slight changes fails with:
Step 1/10 : FROM alpine:latest
---> 055936d39205
Step 2/10 : LABEL MAINTAINER="Peter Winter <peter#pwntr.com>" Description="Simple and lightweight Samba docker container, based on Alpine Linux." Version="1.0.2"
---> Using cache
---> e99eafd27cc6
Step 3/10 : RUN apk --no-cache upgrade && apk --no-cache add samba samba-common-tools supervisor
---> Running in 99c71d23252f
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
WARNING: Ignoring http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz: could not connect to server (check repositories file)
WARNING: Ignoring http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz: could not connect to server (check repositories file)
OK: 6 MiB in 14 packages
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
WARNING: Ignoring http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz: could not connect to server (check repositories file)
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
WARNING: Ignoring http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz: could not connect to server (check repositories file)
ERROR: unsatisfiable constraints:
samba (missing):
required by: world[samba]
samba-common-tools (missing):
required by: world[samba-common-tools]
supervisor (missing):
required by: world[supervisor]
ERROR: compose.cli.main.main: Service 'samba' failed to build: The command '/bin/sh -c apk --no-cache upgrade && apk --no-cache add samba samba-common-tools supervisor' returned a non-zero code: 3
Appears to be not using the proxy?
docker-compose.yml
version: "3.7"
services:
samba:
#image: pwntr/samba-alpine
build:
context: .
dockerfile: Dockerfile
container_name: samba
volumes:
- /some/path:/shared:ro
restart: unless-stopped
I have tried setting the args and environment sections with the proxy details, however makes no difference.
version: "3.7"
services:
samba:
#image: pwntr/samba-alpine
build:
context: .
dockerfile: Dockerfile
# args: # Environment variables available at build-time
# - http_proxy=http://127.0.0.1:3128
# - https_proxy=http://127.0.0.1:3128
# - HTTP_PROXY=http://127.0.0.1:3128
# - HTTPS_PROXY=http://127.0.0.1:3128
# environment: # Environment variables available at container run-time
# - http_proxy=http://127.0.0.1:3128
# - https_proxy=http://127.0.0.1:3128
# - HTTP_PROXY=http://127.0.0.1:3128
# - HTTPS_PROXY=http://127.0.0.1:3128
container_name: samba
volumes:
- /some/path:/shared:ro
restart: unless-stopped
If I use docker run or via image everything works fine.
version: "3.7"
services:
samba:
image: pwntr/samba-alpine
container_name: samba
volumes:
- /some/path:/shared:ro
restart: unless-stopped
Server/Host is running Alpine Linux with connections working fine.
docker info | grep -i proxy
HTTP Proxy: http://127.0.0.1:3128/
HTTPS Proxy: http://127.0.0.1:3128/
Try this
docker build -t localhost/smb:v1 .
You better change RUN apk --no-cache upgrade && apk --no-cache add samba samba-common-tools supervisor to RUN apk --no-cache --upgrade add samba samba-common-tools supervisor to avoid multiple network requests.
In case someone else has the issue.
Run ifconfig and find the IP address of docker0
Edit cntlm.conf and add the following
Listen 172.17.0.1:3128 # IP address of docker0
Restart cntlm
Create config file at /root/.docker/config.json and add the following
{
"proxies":
{
"default":
{
"httpProxy": "http://172.17.0.1:3128",
"httpsProxy": "http://172.17.0.1:3128",
"ftpProxy": "http://172.17.0.1:3128"
}
}
}
Related
I have two apps in go language. user_management app, which I run (docker-compose up --build) first, then I run(docker-compose up --build) sport_app. sport_app is dependent from user_management app.
sport_app Dockerfile file as below.
FROM golang:alpine
RUN apk update && apk upgrade && apk add --no-cache bash git openssh curl
WORKDIR /go-sports-entities-hierarchy
COPY . /go-sports-entities-hierarchy/
RUN rm -rf /go-sports-entities-hierarchy/.env
RUN go mod download
RUN chmod +x /go-sports-entities-hierarchy/scripts/*
RUN ./scripts/build.sh
ADD https://github.com/ufoscout/docker-compose-wait/releases/download/2.2.1/wait /wait
RUN chmod +x /wait
ENV GIN_MODE="debug" \
GQL_SERVER_HOST="localhost" \
GQL_SERVER_PORT=7777 \
ALLOWED_ORIGINS=* \
USER_MANAGEMENT_SERVER_URL="http://localhost:8080/user/me" \
# GQLGen config
GQL_SERVER_GRAPHQL_PATH="graphql" \
GQL_SERVER_GRAPHQL_PLAYGROUND_ENABLED=true \
GQL_SERVER_GRAPHQL_PLAYGROUND_PATH="playground" \
# Export necessary port
EXPOSE 7777
CMD /wait && ./scripts/run.sh
sport_app docker-compose.yml file as below.
version: '3'
volumes:
postgres_data:
driver: local
services:
go-sports-entities-hierarchy:
restart: always
build:
dockerfile: Dockerfile
context: .
environment:
WAIT_HOSTS: postgres:5432
# Web framework config
GIN_MODE: debug
GQL_SERVER_HOST: go-sports-entities-hierarchy
GQL_SERVER_PORT: 7777
ALLOWED_ORIGINS: "*"
USER_MANAGEMENT_SERVER_URL: http://localhost:8080/user/me
# GQLGen config
GQL_SERVER_GRAPHQL_PATH: graphql
GQL_SERVER_GRAPHQL_PLAYGROUND_ENABLED: "true"
GQL_SERVER_GRAPHQL_PLAYGROUND_PATH: playground
ports:
- 7777:7777
depends_on:
- postgres
- redisearch
go-sports-events-workflow:
restart: always
build:
dockerfile: Dockerfile
context: .
environment:
WAIT_HOSTS: postgres:5432
# Web framework config
GIN_MODE: debug
GQL_SERVER_HOST: go-sports-events-workflow
GQL_SERVER_PORT: 7778
ALLOWED_ORIGINS: "*"
# GQLGen config
GQL_SERVER_GRAPHQL_PATH: graphql
GQL_SERVER_GRAPHQL_PLAYGROUND_ENABLED: "true"
GQL_SERVER_GRAPHQL_PLAYGROUND_PATH: playground
depends_on:
- postgres
- redisearch
- go-sports-entities-hierarchy
user_management app Dockerfile as below:
FROM golang:alpine
RUN apk update && apk add --no-cache git ca-certificates && update-ca-certificates
# Set necessary environmet variables needed for our image
ENV GO111MODULE=on \
CGO_ENABLED=0 \
GOOS=linux \
GOARCH=amd64
# Move to working directory /build
WORKDIR /build
# Copy and download dependency using go mod
COPY go.mod .
COPY go.sum .
RUN go mod download
# Copy the code into the container
COPY . .
# Build the application
RUN go build -o main .
# Move to /dist directory as the place for resulting binary folder
WORKDIR /dist
# Copy binary from build to main folder
RUN cp -r /build/html .
RUN cp /build/main .
# Environment Variables
ENV DB_HOST="127.0.0.1" \
APP_PROTOCOL="http" \
APP_HOST="localhost" \
APP_PORT=8080 \
ALLOWED_ORIGINS="*"
# Export necessary port
EXPOSE 8080
ADD https://github.com/ufoscout/docker-compose-wait/releases/download/2.2.1/wait /wait
RUN chmod +x /wait
# Command to run when starting the container
CMD /wait && /dist/main
user_management app docker-compose.yml file as below:
version: '3'
volumes:
postgres_data:
driver: local
services:
postgres:
image: postgres
volumes:
- postgres_data:/var/lib/postgresql/data
ports:
- 5432:5432
go-user-management:
restart: always
build:
dockerfile: Dockerfile
context: .
environment:
# Postgres Details
DB_PORT: 5432
# APP details
APP_PROTOCOL: http
APP_HOST: localhost
APP_PORT: 8080
# System Configuration Details
ALLOWED_ORIGINS: "*"
ports:
- 8080:8080
depends_on:
- postgres
In sport_app I write below code and get error:
client := resty.New()
resp, err := client.R().SetHeader("Content-Type", "application/json").SetHeader("Authorization", "Bearer "+token).Get("http://localhost:8080/user/me")
Error is: Get "http://localhost:8080/user/me": dial tcp 127.0.0.1:8080: connect: connection refused:"
This API(http://localhost:8080/user/me) is written in the user_management app and this is working, I check with the postman.
I already read this question answers, but can not solve my problem.
I am new to docker, please help.
For communicating between multiple docker-compose clients, you need to make sure that the containers you want to talk to each other are on the same network.
For example, (edited for brevity) here you have one of the docker-compose.yml
# sport_app docker-compose.yml
version: '3'
services:
go-sports-entities-hierarchy:
...
networks:
- some-net
go-sports-events-workflow
...
networks:
- some-net
networks:
some-net:
driver: bridge
And the other docker-compose.yml
# user_management app docker-compose.yml
version: '3'
services:
postgres:
...
networks:
- some-net
go-user-management
...
networks:
- some-net
networks:
some-net:
external: true
Note: Your app’s network is given a name based on the project name, which is based on the name of the directory it lives in, in this case a prefix user_ was added.
They can then talk to each other using the service name, i.e. go-user-management, etc.
You can, after running the docker-compose up --build commands, run the docker network ls command to see it, then docker network inspect bridge, etc.
Are there any ways to share data between containers. There is following docker-compose file
version: '3'
services:
app_build_prod:
container_name: 'app'
build:
context: ../
dockerfile: docker/Dockerfile
args:
command: build:prod
nginx:
container_name: 'nginx'
image: nginx:alpine
ports:
- "80:80"
depends_on:
- app_build_prod
Dockerfile content is:
FROM node:10-alpine as builder
## Installing missing packages, fixing git self signed certificate issue
RUN apk update && apk upgrade && \
apk add --no-cache bash git openssh && \
rm -rf /var/cache/apk/* && \
git config --global http.sslVerify false
## Defigning app directory
WORKDIR /usr/app
## Copying files. Files listed in .dockerignore are omitted
COPY . .
## node_modules are on a separate intermediate image will prevent unnecessary npm installs at each build
RUN npm ci
## Declaring arguments and environment variables. Important to declara env var to consume them on run stage
ARG command=build:prod
ENV command=$command
ENTRYPOINT npm run ${command}
Tried with #Robert's solution, but couldn't make it work - app container crashes because of:
EBUSY: resource busy or locked, rmdir '/usr/app/dist
Error: EBUSY: resource busy or locked, rmdir '/usr/app/dist'
My assumption is that /usr/app/dist directory is mounted with read-only access, therefore when Angular attempt to remove it prior the build, it throws an error.
Need to send data following direction
app_build_prod:/usr/app/dist => nginx:/usr/share/nginx/html
I have the same problem and change the sharing to use multi-stage build :
FROM alpine:latest AS builder
...build app_build_prod
FROM nginx:alpine
COPY --from=builder /usr/app/dist /usr/share/nginx/html
and change docker-compose to:
version: '3'
services:
nginx:
container_name: 'nginx'
build:
...
ports:
- "80:80"
My application is dockerized. Its python/django application. We are using a local sms sending api that is restricted on IP based. So I have given them my EC2 ip address. And I am running my docker container in this EC2 machine. But my python app is not able to send requests to that machine. Because this docker container has different IP.
How do I solve this problem ?
Dockerfile
# ToDo use alpine image
FROM python:3.6
# Build Arguments with defaults
ARG envior
ARG build_date
ARG build_version
ARG maintainer_name='Name'
ARG maintainaer_email='email#email.com'
# Adding Labels
LABEL com.example.service="Service Name" \
com.example.maintainer.name="$maintainer_name" \
com.example.maintainer.email="$maintainaer_email" \
com.example.build.enviornment="$envior" \
com.example.build.version="$build_version" \
com.example.build.release-date="$build_date"
# Create app directory
RUN mkdir -p /home/example/app
# Install Libre Office for pdf conversion
RUN apt-get update -qq \
&& apt-get install -y -q libreoffice \
&& apt-get remove -q -y libreoffice-gnome
# Cleanup after apt-get commands
RUN apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* \
/var/cache/apt/archives/*.deb /var/cache/apt/*cache.bin
# Activate WORKING DIR
WORKDIR /home/example/app
# Copying requirements
COPY requirements/${envior}.txt /tmp/requirements.txt
# Install the app dependencies
# ToDo Refactor requirements
RUN pip install -r /tmp/requirements.txt
# Envs
ENV DJANGO_SETTINGS_MODULE app.settings.${envior}
ENV ENVIORNMENT ${envior}
# ADD the source code and entry point into the container
ADD . /home/example/app
ADD entrypoint.sh /home/example/app/entrypoint.sh
# Making entry point executable
RUN chmod +x entrypoint.sh
# Exposing port
EXPOSE 8000
# Entry point and CMD
ENTRYPOINT ["/home/example/app/entrypoint.sh"]
docker-compose.yml
version: '3'
services:
postgres:
image: onjin/alpine-postgres:9.5
restart: unless-stopped
ports:
- "5432:5432"
environment:
LC_ALL: C.UTF-8
POSTGRES_USER: django
POSTGRES_PASSWORD: django
POSTGRES_DB: web
volumes:
- postgres_data:/var/lib/postgresql/data/
web:
build:
context: .
args:
environ: local
command: gunicorn app.wsgi:application -b 0.0.0.0:8000
ports:
- "8000:8000"
depends_on:
- postgres
environment:
DATABASE_URL: 'postgres://django:django#postgres/web'
DJANGO_MANAGEPY_MIGRATE: 'on'
DJANGO_MANAGEPY_COLLECTSTATIC: 'on'
DJANGO_LOADDATA: 'off'
DOMAIN: '0.0.0.0'
volumes:
postgres_data:
You should try putting the container in the same network as your EC2 instance. It means using networks with host driver.
suggested docker-compose file
version: '3'
services:
postgres:
[...]
networks:
- host
volumes:
- postgres_data:/var/lib/postgresql/data/
web:
[...]
networks:
- host
volumes:
postgres_data:
networks:
host:
In case it wouldn't work, you might define your own network by:
networks:
appnet:
driver: host
and connect to that network form services:
postgres:
[..]
networks:
- appnet
Further reading about networks official ref.
An interesting read too from official networking tutorial.
Publish port from docker container to base machine, then configure ec2IP:port in sms application.
Problem
Substitution doesn't work for the build phase
Files
docker-compose.yml (only kibana part):
kibana:
build:
context: services/kibana
args:
KIBANA_VERSION: "${KIBANA_VERSION}"
entrypoint: >
/scripts/wait-for-it.sh elasticsearch:9200
-s --timeout=${ELASTICSEARCH_INIT_TIMEOUT}
-- /usr/local/bin/kibana-docker
environment:
ELASTICSEARCH_URL: http://elasticsearch:9200
volumes:
- ./scripts/wait-for-it.sh:/scripts/wait-for-it.sh
ports:
- "${KIBANA_HTTP_PORT}:5601"
links:
- elasticsearch
depends_on:
- elasticsearch
networks:
- frontend
- backend
restart: always
Dockerfile for the services/kibana:
ARG KIBANA_VERSION=6.2.3
FROM docker.elastic.co/kibana/kibana:${KIBANA_VERSION}
USER root
RUN yum install -y which && yum clean all
USER kibana
COPY kibana.yml /usr/share/kibana/config/kibana.yml
RUN ./bin/kibana-plugin install https://github.com/sivasamyk/logtrail/releases/download/v0.1.27/logtrail-${KIBANA_VERSION}-0.1.27.zip
COPY logtrail.json /usr/share/kibana/plugins/logtrail/logtrail.json
EXPOSE 5601
Env file (only kibana part):
KIBANA_VERSION=6.2.3
KIBANA_HTTP_PORT=5601
KIBANA_ELASTICSEARCH_HOST=elasticsearch
KIBANA_ELASTICSEARCH_PORT=9200
Actual output (Problem is here: substitution doesn't work)
#docker-compose up --force-recreate --build kibana
.........
Step 8/10 : RUN ./bin/kibana-plugin install https://github.com/sivasamyk/logtrail/releases/download/v0.1.27/logtrail-${KIBANA_VERSION}-0.1.27.zip
---> Running in d28b1dcb6348
Attempting to transfer from https://github.com/sivasamyk/logtrail/releases/download/v0.1.27/logtrail--0.1.27.zip
Attempting to transfer from https://artifacts.elastic.co/downloads/kibana-plugins/https://github.com/sivasamyk/logtrail/releases/download/v0.1.27/logtrail--0.1.27.zip/https://github.com/sivasamyk/logtrail/releases/download/v0.1.27/logtrail--0.1.27.zip-6.2.3.zip
Plugin installation was unsuccessful due to error "No valid url specified."
ERROR: Service 'kibana' failed to build: The command '/bin/sh -c ./bin/kibana-plugin install https://github.com/sivasamyk/logtrail/releases/download/v0.1.27/logtrail-${KIBANA_VERSION}-0.1.27.zip' returned a non-zero code: 70
Expected output (something similar):
Step 8/10 : RUN ./bin/kibana-plugin install https://github.com/sivasamyk/logtrail/releases/download/v0.1.27/logtrail-${KIBANA_VERSION}-0.1.27.zip
---> Running in d28b1dcb6348
Attempting to transfer from https://github.com/sivasamyk/logtrail/releases/download/v0.1.27/logtrail-6.2.3-0.1.27.zip
I've found answer after 5 min when I posted this question ... damn
The solution is stupid, but works: I only need to redefine args for the new user. See:
ARG KIBANA_VERSION=6.2.3
FROM docker.elastic.co/kibana/kibana:${KIBANA_VERSION}
USER root
RUN yum install -y which && yum clean all
USER kibana
ARG KIBANA_VERSION=${KIBANA_VERSION}
COPY kibana.yml /usr/share/kibana/config/kibana.yml
RUN ./bin/kibana-plugin install https://github.com/sivasamyk/logtrail/releases/download/v0.1.27/logtrail-${KIBANA_VERSION}-0.1.27.zip
COPY logtrail.json /usr/share/kibana/plugins/logtrail/logtrail.json
EXPOSE 5601
The solution is the following lines:
USER kibana
ARG KIBANA_VERSION=${KIBANA_VERSION}
I've got the following dir tree:
project_root
|- build_script.sh
|- dir_1/Dockerfile and docker-compose.yml
|- tarball_dir/
I've got the build_script.sh which calls docker-compose like so:
docker-compose -f ./dir_1/docker-compose.yml build --build-arg BUILD_ID=$BUILD_ID dev
This is the Dockerfile:
FROM elixir:1.5.2
ARG BUILD_ID
ENV APP_HOME /app
RUN mkdir $APP_HOME
# Copy release tarball and unpack
~ ADD ../dir_1/${BUILD_ID}.tar.gz $APP_HOME/
CMD $APP_HOME/bin/my_app foreground
And this is the docker-compose-yml:
version: '2'
services:
common:
build:
context: .
dockerfile: ./Dockerfile
networks:
- default
dev:
extends:
service: common
env_file:
- ./dev.env
environment:
POSTGRES_HOST: "postgres.dev"
MIX_ENV: dev
ports:
- "4000:4000"
depends_on:
- postgres
I want to be able to ADD (so that it copies and unpacks the tarball) into the image. However, after trying endless combinations of Docker context directories and trying to include the tarball into the image, I always get one of two errors:
ERROR: Service 'dev' failed to build: ADD failed: Forbidden path outside the build context: ../tarball_dir/tarball.tar.gz ()
or
ERROR: Service 'dev' failed to build: ADD failed: stat /var/lib/docker/tmp/docker-builder887135091/tarball_dir/tarball.tar.gz: no such file or directory
I would like to keep this directory structure, if possible. I've managed to build and run the container with the following command:
docker build \
--build-arg BUILD_ID=$BUILD_ID \
-t my_app:$BUILD_ID \
-f dir_1/Dockerfile .
docker run \
--network my_app_network \
--env-file ./dir_1/$ENV.env \
my_app:$BUILD_ID
But I can't reproduce this same behaviour using docker-compose no matter what.
I'm on Mac OS Sierra and I'm using Docker Edge Version 17.10.0-ce-mac36 (19824) Channel: edge a7c7e01149