Problem
Substitution doesn't work for the build phase
Files
docker-compose.yml (only kibana part):
kibana:
build:
context: services/kibana
args:
KIBANA_VERSION: "${KIBANA_VERSION}"
entrypoint: >
/scripts/wait-for-it.sh elasticsearch:9200
-s --timeout=${ELASTICSEARCH_INIT_TIMEOUT}
-- /usr/local/bin/kibana-docker
environment:
ELASTICSEARCH_URL: http://elasticsearch:9200
volumes:
- ./scripts/wait-for-it.sh:/scripts/wait-for-it.sh
ports:
- "${KIBANA_HTTP_PORT}:5601"
links:
- elasticsearch
depends_on:
- elasticsearch
networks:
- frontend
- backend
restart: always
Dockerfile for the services/kibana:
ARG KIBANA_VERSION=6.2.3
FROM docker.elastic.co/kibana/kibana:${KIBANA_VERSION}
USER root
RUN yum install -y which && yum clean all
USER kibana
COPY kibana.yml /usr/share/kibana/config/kibana.yml
RUN ./bin/kibana-plugin install https://github.com/sivasamyk/logtrail/releases/download/v0.1.27/logtrail-${KIBANA_VERSION}-0.1.27.zip
COPY logtrail.json /usr/share/kibana/plugins/logtrail/logtrail.json
EXPOSE 5601
Env file (only kibana part):
KIBANA_VERSION=6.2.3
KIBANA_HTTP_PORT=5601
KIBANA_ELASTICSEARCH_HOST=elasticsearch
KIBANA_ELASTICSEARCH_PORT=9200
Actual output (Problem is here: substitution doesn't work)
#docker-compose up --force-recreate --build kibana
.........
Step 8/10 : RUN ./bin/kibana-plugin install https://github.com/sivasamyk/logtrail/releases/download/v0.1.27/logtrail-${KIBANA_VERSION}-0.1.27.zip
---> Running in d28b1dcb6348
Attempting to transfer from https://github.com/sivasamyk/logtrail/releases/download/v0.1.27/logtrail--0.1.27.zip
Attempting to transfer from https://artifacts.elastic.co/downloads/kibana-plugins/https://github.com/sivasamyk/logtrail/releases/download/v0.1.27/logtrail--0.1.27.zip/https://github.com/sivasamyk/logtrail/releases/download/v0.1.27/logtrail--0.1.27.zip-6.2.3.zip
Plugin installation was unsuccessful due to error "No valid url specified."
ERROR: Service 'kibana' failed to build: The command '/bin/sh -c ./bin/kibana-plugin install https://github.com/sivasamyk/logtrail/releases/download/v0.1.27/logtrail-${KIBANA_VERSION}-0.1.27.zip' returned a non-zero code: 70
Expected output (something similar):
Step 8/10 : RUN ./bin/kibana-plugin install https://github.com/sivasamyk/logtrail/releases/download/v0.1.27/logtrail-${KIBANA_VERSION}-0.1.27.zip
---> Running in d28b1dcb6348
Attempting to transfer from https://github.com/sivasamyk/logtrail/releases/download/v0.1.27/logtrail-6.2.3-0.1.27.zip
I've found answer after 5 min when I posted this question ... damn
The solution is stupid, but works: I only need to redefine args for the new user. See:
ARG KIBANA_VERSION=6.2.3
FROM docker.elastic.co/kibana/kibana:${KIBANA_VERSION}
USER root
RUN yum install -y which && yum clean all
USER kibana
ARG KIBANA_VERSION=${KIBANA_VERSION}
COPY kibana.yml /usr/share/kibana/config/kibana.yml
RUN ./bin/kibana-plugin install https://github.com/sivasamyk/logtrail/releases/download/v0.1.27/logtrail-${KIBANA_VERSION}-0.1.27.zip
COPY logtrail.json /usr/share/kibana/plugins/logtrail/logtrail.json
EXPOSE 5601
The solution is the following lines:
USER kibana
ARG KIBANA_VERSION=${KIBANA_VERSION}
Related
FROM node:latest
WORKDIR /frontend/
ENV PATH /frontend/node_modules/.bin:$PATH
COPY package.json /frontend/package.json
COPY . /frontend/
RUN npm install --silent
RUN npm install react-scripts#3.0.1 -g --silent
CMD ["npm", "run", "start"]
This is my Dockerfile for the frontend of my project.
I put this as one of the services in my docker-compose.yml file, and when I run docker-compose up -d --build, it gives me
Step 6/8 : RUN npm install --silent
---> Running in 09a4f59a96fa
ERROR: Service 'frontend' failed to build: The command '/bin/sh -c npm install --silent' returned a non-zero code: 1
My docker-compose file looks like below for your reference:
# Docker Compose
version: '3.7'
services:
frontend:
container_name: frontend
build:
context: frontend
dockerfile: Dockerfile
ports:
- "3000:3000"
volumes:
- '.:/frontend'
- '/frontend/node_modules'
backend:
build: ./backend
ports:
- "5000:5000"
volumes:
- .:/code
Thanks in advance
EDIT: Error in the frontend after build
For docker-compose, I think it should be like
- ./frontend/:/frontend'
as the build context is frontend.
Second, thing if you are using volume then why you are installing and copying code in Dockerfile? If you are using bind volume then remove these from your Dockerfile as these will be overridden from the host code.
COPY package.json /frontend/package.json
COPY . /frontend/
Using docker-compose to build against an existing dockerfile with some slight changes fails with:
Step 1/10 : FROM alpine:latest
---> 055936d39205
Step 2/10 : LABEL MAINTAINER="Peter Winter <peter#pwntr.com>" Description="Simple and lightweight Samba docker container, based on Alpine Linux." Version="1.0.2"
---> Using cache
---> e99eafd27cc6
Step 3/10 : RUN apk --no-cache upgrade && apk --no-cache add samba samba-common-tools supervisor
---> Running in 99c71d23252f
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
WARNING: Ignoring http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz: could not connect to server (check repositories file)
WARNING: Ignoring http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz: could not connect to server (check repositories file)
OK: 6 MiB in 14 packages
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
WARNING: Ignoring http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz: could not connect to server (check repositories file)
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
WARNING: Ignoring http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz: could not connect to server (check repositories file)
ERROR: unsatisfiable constraints:
samba (missing):
required by: world[samba]
samba-common-tools (missing):
required by: world[samba-common-tools]
supervisor (missing):
required by: world[supervisor]
ERROR: compose.cli.main.main: Service 'samba' failed to build: The command '/bin/sh -c apk --no-cache upgrade && apk --no-cache add samba samba-common-tools supervisor' returned a non-zero code: 3
Appears to be not using the proxy?
docker-compose.yml
version: "3.7"
services:
samba:
#image: pwntr/samba-alpine
build:
context: .
dockerfile: Dockerfile
container_name: samba
volumes:
- /some/path:/shared:ro
restart: unless-stopped
I have tried setting the args and environment sections with the proxy details, however makes no difference.
version: "3.7"
services:
samba:
#image: pwntr/samba-alpine
build:
context: .
dockerfile: Dockerfile
# args: # Environment variables available at build-time
# - http_proxy=http://127.0.0.1:3128
# - https_proxy=http://127.0.0.1:3128
# - HTTP_PROXY=http://127.0.0.1:3128
# - HTTPS_PROXY=http://127.0.0.1:3128
# environment: # Environment variables available at container run-time
# - http_proxy=http://127.0.0.1:3128
# - https_proxy=http://127.0.0.1:3128
# - HTTP_PROXY=http://127.0.0.1:3128
# - HTTPS_PROXY=http://127.0.0.1:3128
container_name: samba
volumes:
- /some/path:/shared:ro
restart: unless-stopped
If I use docker run or via image everything works fine.
version: "3.7"
services:
samba:
image: pwntr/samba-alpine
container_name: samba
volumes:
- /some/path:/shared:ro
restart: unless-stopped
Server/Host is running Alpine Linux with connections working fine.
docker info | grep -i proxy
HTTP Proxy: http://127.0.0.1:3128/
HTTPS Proxy: http://127.0.0.1:3128/
Try this
docker build -t localhost/smb:v1 .
You better change RUN apk --no-cache upgrade && apk --no-cache add samba samba-common-tools supervisor to RUN apk --no-cache --upgrade add samba samba-common-tools supervisor to avoid multiple network requests.
In case someone else has the issue.
Run ifconfig and find the IP address of docker0
Edit cntlm.conf and add the following
Listen 172.17.0.1:3128 # IP address of docker0
Restart cntlm
Create config file at /root/.docker/config.json and add the following
{
"proxies":
{
"default":
{
"httpProxy": "http://172.17.0.1:3128",
"httpsProxy": "http://172.17.0.1:3128",
"ftpProxy": "http://172.17.0.1:3128"
}
}
}
I need to install other node.js modules on the bitnami docker container.
I would like to install body-parser module to the container. I've started the container with sudo docker-compose up and it runs fine. i tried to modify the dockerfile and docker-compose.yml file to install the body-parser but i get EACCES permission denied, access '/app/node_modules' error. Can you help?
TIA,
Thomas
**** UPDATE 4/23/2019 ***
This is the docker file.
I added body-parser line.
## Dockerfile for building production image
FROM bitnami/express:4.16.4-debian-9-r166
LABEL maintainer "John Smith <john.smith#acme.com>"
ENV DISABLE_WELCOME_MESSAGE=1
ENV NODE_ENV=production \
PORT=3000
# Skip fetching dependencies and database migrations for production image
ENV SKIP_DB_WAIT=0 \
SKIP_DB_MIGRATION=1 \
SKIP_NPM_INSTALL=1 \
SKIP_BOWER_INSTALL=1
COPY . /app
RUN sudo chown -R bitnami: /app
RUN npm install
RUN npm install --save body-parser
EXPOSE 3000
CMD ["npm", "start"]
docker-compose.yml
version: '2'
services:
mongodb:
image: 'bitnami/mongodb:latest'
express:
tty: true # Enables debugging capabilities when attached to this container.
image: 'bitnami/express:4'
command: npm start
environment:
- PORT=3000
- NODE_ENV=development
- DATABASE_URL=mongodb://mongodb:27017/myapp
- SKIP_DB_WAIT=0
- SKIP_DB_MIGRATION=0
- SKIP_NPM_INSTALL=0
- SKIP_BOWER_INSTALL=0
depends_on:
- mongodb
ports:
- 3000:3000
volumes:
- .:/app
I have a PostgreSQL container and a Swift server container. I need to pass the DB IP to the Server to start it. So I created an alias for DB in my custom bridge network. Have a look at my docker-compose.yml
version: '3'
services:
db:
build: database
image: postgres
networks:
mybridgenet:
aliases:
- mydbalias
web:
image: mywebserver:latest
ports:
- "8000:8000"
depends_on:
- db
networks:
- mybridgenet
environment:
WAIT_HOSTS: db:5432
networks:
mybridgenet:
driver: bridge
Dockerfile to build webserver.
FROM swift:4.2.1
RUN apt-get update && apt-get install --no-install-recommends -y libpq-dev uuid-dev && rm -rf /var/lib/apt/lists/*
EXPOSE 8000
WORKDIR /app
COPY client ./client
COPY Package.swift ./
COPY Package.resolved ./
COPY Sources ./Sources
RUN swift build
COPY pkg-swift-deps.sh ./
RUN chmod +x ./pkg-swift-deps.sh
RUN ./pkg-swift-deps.sh ./.build/debug/bridgeOS
FROM busybox:glibc
COPY --from=0 /app/swift_libs.tar.gz /tmp/swift_libs.tar.gz
COPY --from=0 /app/.build/debug/bridgeOS /usr/bin/
RUN tar -xzvf /tmp/swift_libs.tar.gz && \
rm -rf /tmp/*
ADD https://github.com/ufoscout/docker-compose-wait/releases/download/2.2.1/wait /wait
RUN chmod +x /wait
CMD /wait && mywebserver db "10.0.2.2"
Database Dockerfile
FROM postgres
COPY init.sql /docker-entrypoint-initdb.d/
The server is started using mybinary mydbalias. Like I said earlier, I pass the alias to start the server. While doing this, I get the following error.
message: "could not translate host name \"mydbalias\" to address: Temporary failure in name resolution\n"
What could be the problem?
UPDATE
After 4 days of a grueling raid, I finally found the rat. He is busybox container. I changed it to ubuntu:16.04 and it's a breeze. Feeling so good about this whole conundrum. Thanks, everyone who helped.
Simplify. There is no need in your explicit network declaration (it is done automatically by docker-compose, nor aliases (services get their host names based on service names)
docker-compose.yml
version: '3'
services:
db:
build: database
image: postgres
web:
image: mywebserver:latest
ports:
- "8000:8000"
depends_on:
- db
environment:
WAIT_HOSTS: db:5432
Then just use db as a hostname to connect to database from web
I added RUN go get to install packages during "docker-compose". However, the following cannot find package error was occurred when I run go build. I found that the packages are saved in /go/pkg/linux_amd64/.
run docker-compose and go build
$ docker-compose up -d
$ docker exec -it explorer-cli /bin/bash
# pwd
/go
# ls
bin pkg src
# echo $GOPATH
/go
# ls /go/pkg/linux_amd64/github.com/
go-sql-driver
# go build -i -o /go/bin/explorer-cli src/main.go
src/main.go:6:2: cannot find package "github.com/go-sql-driver/mysql" in any of:
/usr/local/go/src/github.com/go-sql-driver/mysql (from $GOROOT)
/go/src/github.com/go-sql-driver/mysql (from $GOPATH)
(it worked if I run "go get" manually)
# go get github.com/go-sql-driver/mysql
# ls src/
github.com main.go
# go build -i -o /go/bin/explorer-cli src/main.go
docker-compose.yml
version: '3.4'
services:
mysql:
image: mysql:latest
container_name: database
volumes:
- ./docker/:/etc/mysql/conf.d
- ./docker/:/docker-entrypoint-initdb.d
environment:
- MYSQL_RANDOM_ROOT_PASSWORD=true
- MYSQL_DATABASE=explorer
- MYSQL_USER=admin
- MYSQL_PASSWORD=12dlql*41
app:
build: .
tty: true
image: explorer-cli:latest
container_name: explorer-cli
volumes:
- ./src:/go/src
external_links:
- database
Dockerfile
FROM golang:latest
RUN apt-get update
RUN apt-get upgrade -y
ENV GOBIN /go/bin
RUN go get github.com/go-sql-driver/mysql
main.go
package main
import (
"database/sql"
_ "github.com/go-sql-driver/mysql"
)
func main() {
db, err := sql.Open("mysql", "XUSER:XXXX#(database:3306)/explorer")
if err != nil {
panic(err.Error())
}
defer db.Close()
}
Update 1
I noticed big differences between the following directories.
# ls /go/pkg/linux_amd64/github.com/go-sql-driver/
mysql.a
# ls /go/src/github.com/go-sql-driver/mysql/
AUTHORS connection_go18_test.go packets.go
CHANGELOG.md connection_test.go packets_test.go
CONTRIBUTING.md const.go result.go
LICENSE driver.go rows.go
README.md driver_go18_test.go statement.go
appengine.go driver_test.go statement_test.go
benchmark_go18_test.go dsn.go transaction.go
benchmark_test.go dsn_test.go utils.go
buffer.go errors.go utils_go17.go
collations.go errors_test.go utils_go18.go
connection.go fields.go utils_go18_test.go
connection_go18.go infile.go utils_test.go
Update 2
As #aerokite said, the "volumes" was overwriting the downloaded packages. I changed like the followings and it worked.
Dockerfile
version: '3.4'
FROM golang:latest
RUN apt-get update
RUN apt-get upgrade -y
ENV GOBIN /go/bin
RUN go get github.com/go-sql-driver/mysql
RUN mkdir /go/src/explorer-cli
docker-compose
services:
mysql:
image: mysql:latest
container_name: database
volumes:
- ./docker/:/etc/mysql/conf.d
- ./docker/:/docker-entrypoint-initdb.d
environment:
- MYSQL_RANDOM_ROOT_PASSWORD=true
- MYSQL_DATABASE=explorer
- MYSQL_USER=XUSER
- MYSQL_PASSWORD=XXXX
app:
build: .
tty: true
image: explorer-cli:latest
container_name: explorer-cli
volumes:
- ./src/explorer-cli:/go/src/explorer-cli
external_links:
- database
go build
go build -i -o /go/bin/explorer-cli src/explorer-cli/main.go
I have tried to recreate your problem.
FROM golang:latest
RUN apt-get update
RUN apt-get upgrade -y
ENV GOBIN /go/bin
RUN go get github.com/go-sql-driver/mysql
You have provided this Dockerfile. I have build it
$ docker build -t test .
Now I exec into this image to run your go build command.
$ docker run -it test bash
Then I have created main.go, you provided, in /go/src directory.
And finally, I have built successfully without any error
$ go build -i -o /go/bin/explorer-cli src/main.go
And I think I have found your problem. I have never used docker-compose. But you will understand.
Problem is here:
app:
build: .
tty: true
image: explorer-cli:latest
container_name: explorer-cli
volumes:
- ./src:/go/src <-- problem is here
external_links:
- database
You are mounting ./src into /go/src directory in your docker. This process is overwriting directory /go/src in your docker with your local ./src. And this is removing data you got from go get github.com/go-sql-driver/mysql
Do you understand?
But when you are running go get github.com/go-sql-driver/mysql, its now getting data again.
Solution (01):
Mount your local volume into somewhere else.
volumes:
- ./src:/tmp/src
And modify your Dockerfile to move this main.go to /go/src
Solution (02):
Copy main.go into your docker. Add this line in Dockerfile
COPY ./src/main.go /go/src