I'm trying to build a Python docker image which pip installs from a private repository using ssh. The details of which are in a requirements.txt file.
I've spent a long time reading guides from StackOverflow as well as the official Docker documentation on the subject ...
https://docs.docker.com/develop/develop-images/build_enhancements/#using-ssh-to-access-private-data-in-builds
https://docs.docker.com/compose/compose-file/build/#ssh
... and have come up with a Dockerfile which builds and runs fine when using:
$ docker build --ssh default -t build_tester .
However, when I try to do the same in a docker-compose.yml file, I get the following error:
$ docker-compose up
services.build-tester.build Additional property ssh is not allowed
This is the same even when enabling buildkit:
$ COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose up
services.build-tester.build Additional property ssh is not allowed
Project structure
- docker-compose.yml
- build_files
- Dockerfile
- requirements.txt
- app
- app.py
Dockerfile
# syntax=docker/dockerfile:1.2
FROM python:bullseye as builder
RUN mkdir -p /build/
WORKDIR /build/
RUN apt-get update; \
apt-get install -y git; \
rm -rf /var/lib/apt/lists/*
RUN mkdir -p -m 0600 ~/.ssh; \
ssh-keyscan -H github.com >> ~/.ssh/known_hosts
RUN python3 -m venv env; \
env/bin/pip install --upgrade pip
COPY requirements.txt .
RUN --mount=type=ssh \
env/bin/pip install -r requirements.txt; \
rm requirements.txt
FROM python:slim as runner
RUN mkdir -p /app/
WORKDIR /app/
COPY --from=builder /build/ .
COPY app/ .
CMD ["env/bin/python", "app.py"]
docker-compose.yml
services:
build-tester:
container_name: build-tester
image: build-tester
build:
context: build_files
dockerfile: Dockerfile
ssh:
- default
If I remove ...
ssh:
- default
... the docker-compose up command builds the image OK but obviously doesn't run as app.py doesn't have the required packages installed from pip.
I'd really like to be able to get this working in this way if possible so any advice would be much appreciated.
OK - so ended up being a very simple fix... Just needed to ensure docker-compose was updated to version 2.6 on my Mac.
For some reason brew wasn't updating my docker cask properly so was still running a package from early Jan 2022. Seems --ssh compatibility was added sometime between then and now.
Related
I am trying to build a Docker image from a dockerfile command I received from the previous developer:
bash-5.1$ ls
data_collection demo.py examples requirements.txt start.py
demonstrateur.ipynb Dockerfile README.md serious_game test
bash-5.1$ docker build Dockerfile .
Usage: docker build [OPTIONS] PATH | URL | -
Build an image from a Dockerfile
I also tried with
bash-5.1$ docker build -t serious-game:0.0.1 -t serious-game:latest Dockerfile .
and already completely reinstalled docker by following this tutorial but it gave the same error.
Here is my Dockerfile content:
bash-5.1$ cat Dockerfile
FROM nvidia/cuda:10.2-base-ubuntu18.04
MAINTAINER me
EXPOSE 5555
EXPOSE 8886
ENV DEBIAN_FRONTEND noninteractive
ENV WD=/home/serious-game/
WORKDIR ${WD}
# Add git and ssh
RUN apt-get -y update && \
apt-get -y upgrade && \
apt-get -y install git ssh pkg-config python3-pip python3-opencv
# Dépendances python
COPY requirements.txt /requirements.txt
RUN cd / && \
python3 -m pip install --upgrade pip && \
pip3 install -r requirements.txt
CMD ["start.py"]
If you are trying to build an image from a local Dockerfile, given your current bash location is in the same folder where Dockerfile resides - all you have to do is
docker build .
As written in the docs, Docker uses the file named Dockerfile by default. If you want to specify the file you can use the option --file or -f of the docker build command.
In your case you can just use to solve your problem:
docker build -t serious-game:0.0.1 -t serious-game:latest .
But if you want to specify another file named TestDockerfile (example for testing):
docker build -t serious-game:0.0.1 -t serious-game:latest -f TestDockerfile .
I am new to docker and trying to learn it by it's documentation. AS i need to create a NextJS build using docker image for nginx server i have followed the below process.
Install the nginx
Seeding the port 80 to 3000 in the default config.
Symlink the out directory to base nginx directory
CMD to take care the production build and symlinking of the out directory.
FROM node:alpine AS deps
RUN apk add --no-cache libc6-compat git
RUN apt-get install nginx -y
WORKDIR /sample-app
COPY package.json yarn.lock ./
RUN yarn install --frozen-lockfile
FROM node:alpine AS builder
WORKDIR /sample-app
COPY . .
COPY --from=deps /sample-app/node_modules ./node_modules
RUN yarn build
FROM node:alpine AS runner
WORKDIR /sample-app
ENV NODE_ENV production
RUN ls -SF /sample-app/out /usr/share/nginx/html
RUN -p 3000:80 -v /sample-app/out:/usr/share/nginx/html:ro -d nginx
RUN addgroup -g 1001 -S nodejs
RUN adduser -S nextjs -u 1001
RUN chown -R nextjs:nodejs /sample-app/out
USER nextjs
CMD ["nginx -g daemon=off"]
While running the docker build shell script command as sudo docker build . -t sample-app it throws the error The command '/bin/sh -c apt-get install nginx -y' returned a non-zero code: 127
I do not have much experience with alpine images, but I think that you have to use apk (Alpine Package Keeper) for installing packages
try apk add nginx instead of apt-get install nginx -y
Why Docker image build is getting failed when build with - ?
Host Details
- docker desktop community 2.1.0.5 for Windows
- Windows 10
Dockerfile:
FROM ubuntu:latest
MAINTAINER "rizwan#gm.com"
RUN apt-get update \
&& apt-get install -y python3-pip python3-dev \
&& cd /usr/local/bin \
&& ln -s /usr/bin/python3 python \
&& pip3 install --upgrade pip
WORKDIR /app
COPY . /app
COPY requirements.txt /app/requirements.txt
RUN pip3 --no-cache-dir install -r requirements.txt
EXPOSE 5000
CMD ["python3", "my_service.py","--input-path= /input.csv", "--output-path=/output.csv"]
Folder Structure
-Root
-Application.py
-Dockerfile
-requirements.txt
COMMAND
- Failing : docker build - < Dockerfile
Message: ERROR: Could not open requirements file: [Errno 2] No such
file or directory: 'requirements.txt'
- Successful: docker build .
When you run
docker build - < Dockerfile
it sends only the Dockerfile to the Docker daemon, but no other files. When you tell Docker to COPY a file into the image, you haven't actually sent it the file. It's very similar to including everything in your source tree in the .dockerignore file.
Typically you'll send Docker the current directory as the context directory instead:
docker build . # knows to look for Dockerfile by default
I'm trying to run a pretty simple Flask API in OpenShift Origin 3.6. I can build this container just fine locally and remotely, but when I go to deploy on OpenShift, I get permissions errors on the RUN chmod -R 777 ... lines. Has anyone found a way around this? I wonder if my corporate environment doesn't allow this type of copying, but it's all within a container...
Edit: providing a completely minimal example
Directory structure:
project
├── Dockerfile
└── app
└── api.py
Dockerfile to build base image:
FROM docker.io/ubuntu:16.04
RUN apt-get update && apt-get install -y --no-install-recommends \
cmake curl git make gunicorn nginx python3 python3-pip python3-setuptools build-essential \
&& apt-get clean && rm -rf /var/lib/apt/lists/*
RUN pip3 install --upgrade pip
RUN pip3 install pandas numpy scikit-learn scipy xgboost flask-restful nltk gunicorn
RUN mkdir -p /home/app
WORKDIR /
RUN python3 -c 'import nltk; nltk.download("punkt")'
RUN mv /root/nltk_data /home/app
I then run docker build . -t project:latest --no-cache. Next, the Dockerfile that uses the base image from above to deploy the actual application (I basically just comment out the "base image" lines from above and uncomment these ones from below, using the same Dockerfile file):
FROM project:latest
COPY app /home/app
RUN chmod -R 777 /home/app
WORKDIR /home/app
EXPOSE 5000
CMD ["python3", "api.py"]
I build the container to be deployed using docker build . -t project:app --no-cache.
api.py:
import time
if __name__ == '__main__':
while True:
print('This app is running')
time.sleep(30)
I am a newbie to Docker. I'm using ubuntu 14.04 as my OS and I've installed Docker Community Edition by following instructions from https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/#set-up-the-repository
I have a created a docker file for my project and run it using docker-compose file.
My Dockerfile is as follows.
# ImageName
FROM node:8.8.1
# Create app required directories
ENV appDir /usr/src/app
RUN mkdir -p /usr/src/app /usr/src/app/datas /usr/log/supervisor
# Change working directory
WORKDIR ${appDir}
# Install dependencies
RUN apt-get update && \
apt-get -y install vim\
supervisor \
python3 \
python3-pip \
python3-setuptools \
groff \
less \
&& pip3 install --upgrade pip \
&& apt-get clean
RUN pip3 --no-cache-dir install --upgrade awscli
# Install app dependencies
COPY graphql/package.json /usr/src/app
RUN npm install
RUN npm install -g webpack
# Copy app source code
COPY graphql/ /usr/src/app
COPY datas/ /usr/src/app/datas
# Set Environment Variables
RUN echo export DATA_DIR=/usr/src/app/datas/ >> ~/.data_variables && \
echo "source ~/.data_variables" >> ~/.bash_login && \
echo "source ~/.data_variables" >> ~/.bashrc
COPY supervisord.conf /etc/supercvisor/conf.d/supervisord.conf
# Expose API port to the outside
EXPOSE 5000
# Launch application
CMD ["/usr/bin/supervisord", "-c", "/etc/supercvisor/conf.d/supervisord.conf"]
My docker-compose file
version: '3'
services:
web:
build: .
image: graphql_img
container_name: graphql_img_master
ports:
- "5000:5000"
My supervisord.conf file
[supervisord]
nodaemon=true
[program:babelWatch]
command=npm run babelWatch
[program:monitor]
command=npm run monitor
As you can see I've exposed the port 5000, but when I try to check the output on the browser using the command localhost:5000/graphql it shows an error
This site can’t be reached
I even tried to check for the ip address of docker container using "docker inspect" command and I've used that container ip address with the port still I'm getting the error. Can somebody please help me out on this. Any help would be much appreciated.
Additionally, it would also really helpful to know how to make the program "run monitor" to run on foreground using supervisor