docker-compose up failed - reading directory failed - docker

I have a docker image which I wanted to bring up to run test automatically, the scripts are located at /opt/robotframework/tests
Error occurred that docker cannot read the directory:
$ docker-compose up
Creating network "docker-robot-framework_default" with the default driver
Creating robot-runner ... done
Attaching to robot-runner
robot-runner | [ ERROR ] Reading directory '/opt/robotframework/tests' failed: PermissionError: [Errno 13] Permission denied: '/opt/robotframework/tests'
robot-runner |
robot-runner | Try --help for usage information.
robot-runner exited with code 252
docker-compose.yml
version: '3'
services:
robot-runner:
build:
context: .
dockerfile: /Dockerfile
container_name: robot-runner
image: ppodgorsek/robot-framework:latest
volumes:
- ./test:/opt/robotframework/tests
- ./test-audios:/opt/robotframework/test-audios
- ./output-local:/opt/robotframework/reports
environment:
PYTHONWARNINGS: "ignore:Unverified HTTPS request"
Dockerfile:
FROM fedora:36
MAINTAINER Paul Podgorsek <ppodgorsek#users.noreply.github.com>
LABEL description Robot Framework in Docker.
# Set the reports directory environment variable
ENV ROBOT_REPORTS_DIR /opt/robotframework/reports
# Set the tests directory environment variable
ENV ROBOT_TESTS_DIR /opt/robotframework/tests
# ENV ROBOT_TEST_AUDIOS_DIR /opt/robotframework/test-audios
# Set the working directory environment variable
ENV ROBOT_WORK_DIR /opt/robotframework/temp
# Setup X Window Virtual Framebuffer
ENV SCREEN_COLOUR_DEPTH 24
ENV SCREEN_HEIGHT 1080
ENV SCREEN_WIDTH 1920
# Setup the timezone to use, defaults to UTC
ENV TZ UTC
# Set number of threads for parallel execution
# By default, no parallelisation
ENV ROBOT_THREADS 1
# Define the default user who'll run the tests
ENV ROBOT_UID 1000
ENV ROBOT_GID 1000
# Dependency versions
ENV ALPINE_GLIBC 2.35-r0
ENV AWS_CLI_VERSION 1.22.87
ENV AXE_SELENIUM_LIBRARY_VERSION 2.1.6
ENV BROWSER_LIBRARY_VERSION 12.2.0
ENV CHROMIUM_VERSION 99.0
ENV DATABASE_LIBRARY_VERSION 1.2.4
ENV DATADRIVER_VERSION 1.6.0
ENV DATETIMETZ_VERSION 1.0.6
ENV FAKER_VERSION 5.0.0
ENV FIREFOX_VERSION 98.0
ENV FTP_LIBRARY_VERSION 1.9
ENV GECKO_DRIVER_VERSION v0.30.0
ENV IMAP_LIBRARY_VERSION 0.4.2
ENV PABOT_VERSION 2.5.2
ENV REQUESTS_VERSION 0.9.2
ENV ROBOT_FRAMEWORK_VERSION 5.0
ENV SELENIUM_LIBRARY_VERSION 6.0.0
ENV SSH_LIBRARY_VERSION 3.8.0
ENV XVFB_VERSION 1.20
# By default, no reports are uploaded to AWS S3
ENV AWS_UPLOAD_TO_S3 false
# Prepare binaries to be executed
COPY bin/chromedriver.sh /opt/robotframework/bin/chromedriver
COPY bin/chromium-browser.sh /opt/robotframework/bin/chromium-browser
COPY bin/run-tests-in-virtual-screen.sh /opt/robotframework/bin/
# COPY bin/mml_4_apr_2018_b_session3_2.wav /opt/robotframework/test-audios
# COPY bin/mml_4_apr_2018_b_session3_2.stm /opt/robotframework/test-audios
# Install system dependencies
RUN dnf upgrade -y --refresh \
&& dnf install -y \
chromedriver-${CHROMIUM_VERSION}* \
chromium-${CHROMIUM_VERSION}* \
firefox-${FIREFOX_VERSION}* \
npm \
nodejs \
python3-pip \
tzdata \
xorg-x11-server-Xvfb-${XVFB_VERSION}* \
&& dnf clean all
# FIXME: below is a workaround, as the path is ignored
RUN mv /usr/lib64/chromium-browser/chromium-browser /usr/lib64/chromium-browser/chromium-browser-original \
&& ln -sfv /opt/robotframework/bin/chromium-browser /usr/lib64/chromium-browser/chromium-browser
# Install Robot Framework and associated libraries
RUN pip3 install \
--no-cache-dir \
robotframework==$ROBOT_FRAMEWORK_VERSION \
robotframework-browser==$BROWSER_LIBRARY_VERSION \
robotframework-databaselibrary==$DATABASE_LIBRARY_VERSION \
robotframework-datadriver==$DATADRIVER_VERSION \
robotframework-datadriver[XLS] \
robotframework-datetime-tz==$DATETIMETZ_VERSION \
robotframework-faker==$FAKER_VERSION \
robotframework-ftplibrary==$FTP_LIBRARY_VERSION \
robotframework-imaplibrary2==$IMAP_LIBRARY_VERSION \
robotframework-pabot==$PABOT_VERSION \
robotframework-requests==$REQUESTS_VERSION \
robotframework-seleniumlibrary==$SELENIUM_LIBRARY_VERSION \
robotframework-sshlibrary==$SSH_LIBRARY_VERSION \
axe-selenium-python==$AXE_SELENIUM_LIBRARY_VERSION \
PyYAML \
# Install awscli to be able to upload test reports to AWS S3
awscli==$AWS_CLI_VERSION
# Gecko drivers
RUN dnf install -y \
wget \
# Download Gecko drivers directly from the GitHub repository
&& wget -q "https://github.com/mozilla/geckodriver/releases/download/$GECKO_DRIVER_VERSION/geckodriver-$GECKO_DRIVER_VERSION-linux64.tar.gz" \
&& tar xzf geckodriver-$GECKO_DRIVER_VERSION-linux64.tar.gz \
&& mkdir -p /opt/robotframework/drivers/ \
&& mv geckodriver /opt/robotframework/drivers/geckodriver \
&& rm geckodriver-$GECKO_DRIVER_VERSION-linux64.tar.gz \
&& dnf remove -y \
wget \
&& dnf clean all
# Install the Node dependencies for the Browser library
# FIXME: Playright currently doesn't support relying on system browsers, which is why the `--skip-browsers` parameter cannot be used here.
RUN rfbrowser init \
&& ln -sf /usr/lib64/libstdc++.so.6 /usr/local/lib/python3.10/site-packages/Browser/wrapper/node_modules/playwright-core/.local-browsers/firefox-1316/firefox/libstdc++.so.6
# Create the default report and work folders with the default user to avoid runtime issues
# These folders are writeable by anyone, to ensure the user can be changed on the command line.
RUN mkdir -p ${ROBOT_REPORTS_DIR} \
&& mkdir -p ${ROBOT_WORK_DIR} \
&& chown ${ROBOT_UID}:${ROBOT_GID} ${ROBOT_REPORTS_DIR} \
&& chown ${ROBOT_UID}:${ROBOT_GID} ${ROBOT_WORK_DIR} \
&& chmod ugo+w ${ROBOT_REPORTS_DIR} ${ROBOT_WORK_DIR}
# Allow any user to write logs
RUN chmod ugo+w /var/log \
&& chown ${ROBOT_UID}:${ROBOT_GID} /var/log
# Update system path
ENV PATH=/opt/robotframework/bin:/opt/robotframework/drivers:$PATH
# Set up a volume for the generated reports
VOLUME ${ROBOT_REPORTS_DIR}
USER ${ROBOT_UID}:${ROBOT_GID}
# A dedicated work folder to allow for the creation of temporary files
WORKDIR ${ROBOT_WORK_DIR}
# Execute all robot tests
CMD ["run-tests-in-virtual-screen.sh"]
local directories:
enter image description here

Basically the USER specified in dockerfile (USER ${ROBOT_UID}:${ROBOT_GID}) is used the container and has no access rights to the folder on your host. While you could use root in the container to "solve" the problem your container may get root on host. You should NEVER use root in a docker container.
To avoid the problem give the user (in your case 1000:1000) appropriate rights on the folder on host (./test) with setfacl. If the user is not present on host just add one with same UID/GID:
sudo addgroup robot --gid 1000
sudo adduser robot --ingroup robot --uid 1000
setfacl -R -m u:robot:rwx test

by adding user: root in docker-compose.yml. The user granted full access right to path.
version: '3'
services:
robot-runner:
build:
context: .
dockerfile: /Dockerfile
container_name: robot-runner
# image: ppodgorsek/robot-framework:latest
image: robot-runner:latest
user: root
volumes:
- ./BrowserTests:/opt/robotframework/tests
- ./output-local:/opt/robotframework/reports
environment:
PYTHONWARNINGS: "ignore:Unverified HTTPS request"
extra_hosts:
- "speech.sts:172.17.0.1"
- "speech.srs:172.17.0.1"
networks:
- sts_sts_network
networks:
sts_sts_network:
external: true

Related

Building a Dockerfile from inside Docker Compose

So I'm trying to follow these instructions:
https://github.com/open-forest/sendy
I'm using Portainer and trying to run a Sendy container (newsletter software). Instead of running a MySQL image with it, I'm just using my external managed database instead.
On my server I keep project data at: /var/docker/project-name. I use this structure for bind mounting if I need to bring data into the containers from the start.
So for this project in the project-name folder I have sendy-6.0.2.zip and this Dockerfile: (This file was provide via the instructions on the above link)
#
# Docker with Sendy Email Campaign Marketing
#
# Build:
# $ docker build -t sendy:latest --target sendy -f ./Dockerfile .
#
# Build w/ XDEBUG installed
# $ docker build -t sendy:debug-latest --target debug -f ./Dockerfile .
#
# Run:
# $ docker run --rm -d --env-file sendy.env sendy:latest
FROM php:7.4.8-apache as sendy
ARG SENDY_VER=6.0.2
ARG ARTIFACT_DIR=6.0.2
ENV SENDY_VERSION ${SENDY_VER}
RUN apt -qq update && apt -qq upgrade -y \
# Install unzip cron
&& apt -qq install -y unzip cron \
# Install php extension gettext
# Install php extension mysqli
&& docker-php-ext-install calendar gettext mysqli \
# Remove unused packages
&& apt autoremove -y
# Copy artifacts
COPY ./artifacts/${ARTIFACT_DIR}/ /tmp
# Install Sendy
RUN unzip /tmp/sendy-${SENDY_VER}.zip -d /tmp \
&& cp -r /tmp/includes/* /tmp/sendy/includes \
&& mkdir -p /tmp/sendy/uploads/csvs \
&& chmod -R 777 /tmp/sendy/uploads \
&& rm -rf /var/www/html \
&& mv /tmp/sendy /var/www/html \
&& chown -R www-data:www-data /var/www \
&& mv /usr/local/etc/php/php.ini-production /usr/local/etc/php/php.ini \
&& rm -rf /tmp/* \
&& echo "\nServerName \${SENDY_FQDN}" > /etc/apache2/conf-available/serverName.conf \
# Ensure X-Powered-By is always removed regardless of php.ini or other settings.
&& printf "\n\n# Ensure X-Powered-By is always removed regardless of php.ini or other settings.\n\
Header always unset \"X-Powered-By\"\n\
Header unset \"X-Powered-By\"\n" >> /var/www/html/.htaccess \
&& printf "[PHP]\nerror_reporting = E_ALL & ~E_NOTICE & ~E_STRICT & ~E_DEPRECATED\n" > /usr/local/etc/php/conf.d/error_reporting.ini
# Apache config
RUN a2enconf serverName
# Apache modules
RUN a2enmod rewrite headers
# Copy hello-cron file to the cron.d directory
COPY cron /etc/cron.d/cron
# Give execution rights on the cron job
RUN chmod 0644 /etc/cron.d/cron \
# Apply cron job
&& crontab /etc/cron.d/cron \
# Create the log file to be able to run tail
&& touch /var/log/cron.log
COPY artifacts/docker-entrypoint.sh /usr/local/bin/
ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"]
CMD ["apache2-foreground"]
#######################
# XDEBUG Installation
#######################
FROM sendy as debug
# Install xdebug extension
RUN pecl channel-update pecl.php.net \
&& pecl install xdebug \
&& docker-php-ext-enable xdebug \
&& rm -rf /tmp/pear
Here is my Docker Compose file:
version: '3.7'
services:
project-sendy:
container_name: project-sendy
image: sendy:6.0.2
build:
dockerfile: var/docker/project-sendy/Dockerfile
restart: unless-stopped
networks:
- proxy
- default
labels:
- "traefik.enable=true"
- "traefik.docker.network=proxy"
- "traefik.http.routers.project-secure.entrypoints=websecure"
- "traefik.http.routers.project-secure.rule=Host(`project.com`)"
environment:
SENDY_PROTOCOL: https
SENDY_FQDN: project.com
MYSQL_HOST: db-host-name-here
MYSQL_DATABASE: db-name-here
MYSQL_USER: db-user-name-here
MYSQL_PASSWORD: db-password-here
SENDY_DB_PORT: db-port-here
networks:
proxy:
external: true
When I try to deploy I get:
failed to deploy a stack: project-sendy Pulling project-sendy
Error could not find /data/compose/126/var/docker/project-sendy:
stat /data/compose/126/var/docker/project-sendy: no such file or directory
So here's what I've done.
I have the cron and artifacts folder on the same directory as the Dockerfile.
In the Dockerfile look for this line:
COPY artifacts/docker-entrypoint.sh /usr/local/bin/
Right below it put this line:
RUN chmod +x /usr/local/bin/docker-entrypoint.sh
Otherwise you will get this error:
Starting Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "/usr/local/bin/docker-entrypoint.sh": permission denied: unknown
Then build it with:
docker build -t sendy:6.0.2 .
Then your image will show up in portainer.
You can then remove the build section in your docker compose file and hit deploy. It now works for me.

how to make a dockerfile with only one container?

I have a yii1 application. And I have a dockerfile. And I had a docker-compose file.
But for the momemnt I only have one application. Because I have a remote database. So the database is not in a container.
So I have this dockerfile:
FROM php:7.3-apache
#COPY BaltimoreCyberTrustRoot.crt.pem /usr/local/share/ca-certificates/AzureDB.crt
# Copy virtual host into container
COPY 000-default.conf /etc/apache2/sites-available/000-default.conf
# Enable rewrite mode
RUN a2enmod rewrite
# Install necessary packages
RUN apt-get update && \
apt-get install \
libzip-dev \
wget \
git \
unzip \
-y --no-install-recommends
# Install PHP Extensions
RUN docker-php-ext-install zip pdo_mysql
# RUN pecl install -o -f xdebug-3.1.3 \
# && rm -rf /tmp/pear
# Copy composer installable
COPY ./install-composer.sh ./
# Copy php.ini
COPY ./php.ini /usr/local/etc/php/
#COPY BaltimoreCyberTrustRoot.crt.pem /var/www/html/
EXPOSE 80
# Cleanup packages and install composer
RUN apt-get purge -y g++ \
&& apt-get autoremove -y \
&& rm -r /var/lib/apt/lists/* \
&& rm -rf /tmp/* \
&& sh ./install-composer.sh \
&& rm ./install-composer.sh
# Change the current working directory
WORKDIR /var/www/html
# Change the owner of the container document root
RUN chown -R www-data:www-data /var/www
# Start Apache in foreground
CMD ["apache2-foreground"]
And I had this docker-compose file:
version: '3'
services:
web:
build: ./docker
container_name: dockeryiidisc
ports:
- 80:80
- 443:443
volumes:
- C:\xampp\htdocs\webScraper/docker:/etc/apache2/sites-enabled/
- C:\xampp\htdocs\webScraper:/var/www/html/
and that worked.
But so now I only want to use the dockerfile.
So I tried this:
docker build -t docker_webcrawler .
and this command:
docker run -d -p 80:80 --name cntr-apache docker_webcrawler
But if I then go to: http://localhost:80
I only see a empty directory:
Index of /
[ICO] Name Last modified Size Description
So what I have to change? That I only have to use the dockerfile?
Thank you
It looks like you're missing the volume mappings that you have in your docker-compose file. Try this
docker run -d -p 80:80 --name cntr-apache -v C:\xampp\htdocs\webScraper/docker:/etc/apache2/sites-enabled/ -v C:\xampp\htdocs\webScraper:/var/www/html/ docker_webcrawler

how to run docker to the same network with another docker container?

I have a website docker container (Docker-A) running on network sts_sts_network.
website URL : https://speech.sts:2000/
I then wrote a test scripts to run automation steps on the website.
The test will be executed in docker as well, by using the docker run command:
docker run -v `pwd`/reports:/opt/robotframework/reports:Z -v `pwd`/test:/opt/robotframework/tests:Z -e BROWSER=chrome ppodgorsek/robot-framework:latest
The test script failed due to
| FAIL |
Error: page.goto: net::ERR_NAME_NOT_RESOLVED at https://speech.sts:2000/
How do make the docker run to reach the website?
This is the dockerfile:
FROM fedora:36
MAINTAINER Paul Podgorsek <ppodgorsek#users.noreply.github.com>
LABEL description Robot Framework in Docker.
# Set the reports directory environment variable
ENV ROBOT_REPORTS_DIR /opt/robotframework/reports
# Set the tests directory environment variable
ENV ROBOT_TESTS_DIR /opt/robotframework/tests
# Set the working directory environment variable
ENV ROBOT_WORK_DIR /opt/robotframework/temp
# Setup X Window Virtual Framebuffer
ENV SCREEN_COLOUR_DEPTH 24
ENV SCREEN_HEIGHT 1080
ENV SCREEN_WIDTH 1920
# Setup the timezone to use, defaults to UTC
ENV TZ UTC
# Set number of threads for parallel execution
# By default, no parallelisation
ENV ROBOT_THREADS 1
# Define the default user who'll run the tests
ENV ROBOT_UID 1000
ENV ROBOT_GID 1000
# Dependency versions
ENV ALPINE_GLIBC 2.35-r0
ENV AWS_CLI_VERSION 1.22.87
ENV AXE_SELENIUM_LIBRARY_VERSION 2.1.6
ENV BROWSER_LIBRARY_VERSION 12.2.0
ENV CHROMIUM_VERSION 99.0
ENV DATABASE_LIBRARY_VERSION 1.2.4
ENV DATADRIVER_VERSION 1.6.0
ENV DATETIMETZ_VERSION 1.0.6
ENV FAKER_VERSION 5.0.0
ENV FIREFOX_VERSION 98.0
ENV FTP_LIBRARY_VERSION 1.9
ENV GECKO_DRIVER_VERSION v0.30.0
ENV IMAP_LIBRARY_VERSION 0.4.2
ENV PABOT_VERSION 2.5.2
ENV REQUESTS_VERSION 0.9.2
ENV ROBOT_FRAMEWORK_VERSION 5.0
ENV SELENIUM_LIBRARY_VERSION 6.0.0
ENV SSH_LIBRARY_VERSION 3.8.0
ENV XVFB_VERSION 1.20
# By default, no reports are uploaded to AWS S3
ENV AWS_UPLOAD_TO_S3 false
# Prepare binaries to be executed
COPY bin/chromedriver.sh /opt/robotframework/bin/chromedriver
COPY bin/chromium-browser.sh /opt/robotframework/bin/chromium-browser
COPY bin/run-tests-in-virtual-screen.sh /opt/robotframework/bin/
# Install system dependencies
RUN dnf upgrade -y --refresh \
&& dnf install -y \
chromedriver-${CHROMIUM_VERSION}* \
chromium-${CHROMIUM_VERSION}* \
firefox-${FIREFOX_VERSION}* \
npm \
nodejs \
python3-pip \
tzdata \
xorg-x11-server-Xvfb-${XVFB_VERSION}* \
&& dnf clean all
# FIXME: below is a workaround, as the path is ignored
RUN mv /usr/lib64/chromium-browser/chromium-browser /usr/lib64/chromium-browser/chromium-browser-original \
&& ln -sfv /opt/robotframework/bin/chromium-browser /usr/lib64/chromium-browser/chromium-browser
# Install Robot Framework and associated libraries
RUN pip3 install \
--no-cache-dir \
robotframework==$ROBOT_FRAMEWORK_VERSION \
robotframework-browser==$BROWSER_LIBRARY_VERSION \
robotframework-databaselibrary==$DATABASE_LIBRARY_VERSION \
robotframework-datadriver==$DATADRIVER_VERSION \
robotframework-datadriver[XLS] \
robotframework-datetime-tz==$DATETIMETZ_VERSION \
robotframework-faker==$FAKER_VERSION \
robotframework-ftplibrary==$FTP_LIBRARY_VERSION \
robotframework-imaplibrary2==$IMAP_LIBRARY_VERSION \
robotframework-pabot==$PABOT_VERSION \
robotframework-requests==$REQUESTS_VERSION \
robotframework-seleniumlibrary==$SELENIUM_LIBRARY_VERSION \
robotframework-sshlibrary==$SSH_LIBRARY_VERSION \
axe-selenium-python==$AXE_SELENIUM_LIBRARY_VERSION \
PyYAML \
# Install awscli to be able to upload test reports to AWS S3
awscli==$AWS_CLI_VERSION
# Gecko drivers
RUN dnf install -y \
wget \
# Download Gecko drivers directly from the GitHub repository
&& wget -q "https://github.com/mozilla/geckodriver/releases/download/$GECKO_DRIVER_VERSION/geckodriver-$GECKO_DRIVER_VERSION-linux64.tar.gz" \
&& tar xzf geckodriver-$GECKO_DRIVER_VERSION-linux64.tar.gz \
&& mkdir -p /opt/robotframework/drivers/ \
&& mv geckodriver /opt/robotframework/drivers/geckodriver \
&& rm geckodriver-$GECKO_DRIVER_VERSION-linux64.tar.gz \
&& dnf remove -y \
wget \
&& dnf clean all
# Install the Node dependencies for the Browser library
# FIXME: Playright currently doesn't support relying on system browsers, which is why the `--skip-browsers` parameter cannot be used here.
RUN rfbrowser init \
&& ln -sf /usr/lib64/libstdc++.so.6 /usr/local/lib/python3.10/site-packages/Browser/wrapper/node_modules/playwright-core/.local-browsers/firefox-1316/firefox/libstdc++.so.6
# Create the default report and work folders with the default user to avoid runtime issues
# These folders are writeable by anyone, to ensure the user can be changed on the command line.
RUN mkdir -p ${ROBOT_REPORTS_DIR} \
&& mkdir -p ${ROBOT_WORK_DIR} \
&& chown ${ROBOT_UID}:${ROBOT_GID} ${ROBOT_REPORTS_DIR} \
&& chown ${ROBOT_UID}:${ROBOT_GID} ${ROBOT_WORK_DIR} \
&& chmod ugo+w ${ROBOT_REPORTS_DIR} ${ROBOT_WORK_DIR}
# Allow any user to write logs
RUN chmod ugo+w /var/log \
&& chown ${ROBOT_UID}:${ROBOT_GID} /var/log
# Update system path
ENV PATH=/opt/robotframework/bin:/opt/robotframework/drivers:$PATH
# Set up a volume for the generated reports
VOLUME ${ROBOT_REPORTS_DIR}
USER ${ROBOT_UID}:${ROBOT_GID}
# A dedicated work folder to allow for the creation of temporary files
WORKDIR ${ROBOT_WORK_DIR}
# Execute all robot tests
CMD ["run-tests-in-virtual-screen.sh"]
You can create a Docker Network:
docker network create sts_sts_network
Then you modify your docker run command:
App1:
docker run -v ... --network sts_sts_network
App2:
docker run -v ... --network sts_sts_network
https://docs.docker.com/network/bridge/

VS code remote containers - connect to a database either on host machine or one in a separate docker-compose file

I have started using VS code remote containers, specifically the GO one thats generated for you via the options. It provides a devcontainer.json file and a Dockerfile which works fine.
My problem is trying to connect to a database from within this remote container.
I have tried spinning up a separate postgres database using docker-compose and have also tried connecting to a database installed on my base machine.
However each time I try to connect either via db, err := sqlx.Connect("postgres", dsn) or using soda soda migrate up if just hangs for a while before saying its unable to connect.
Has anyone got a solution to connecting to a DB?
This is my dockerfile for go:
#-------------------------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See https://go.microsoft.com/fwlink/?linkid=2090316 for license information.
#-------------------------------------------------------------------------------------------------------------
FROM golang:1
# This Dockerfile adds a non-root user with sudo access. Use the "remoteUser"
# property in devcontainer.json to use it. On Linux, the container user's GID/UIDs
# will be updated to match your local UID/GID (when using the dockerFile property).
# See https://aka.ms/vscode-remote/containers/non-root-user for details.
ARG USERNAME=vscode
ARG USER_UID=1000
ARG USER_GID=$USER_UID
# Configure apt, install packages and tools
RUN apt-get update \
&& export DEBIAN_FRONTEND=noninteractive \
&& apt-get -y install --no-install-recommends apt-utils dialog 2>&1 \
#
# Verify git, process tools, lsb-release (common in install instructions for CLIs) installed
&& apt-get -y install git openssh-client less iproute2 procps lsb-release \
#
# Build Go tools w/module support
&& mkdir -p /tmp/gotools \
&& cd /tmp/gotools \
&& GOPATH=/tmp/gotools GO111MODULE=on go get -v golang.org/x/tools/gopls#latest 2>&1 \
&& GOPATH=/tmp/gotools GO111MODULE=on go get -v \
honnef.co/go/tools/...#latest \
golang.org/x/tools/cmd/gorename#latest \
golang.org/x/tools/cmd/goimports#latest \
golang.org/x/tools/cmd/guru#latest \
golang.org/x/lint/golint#latest \
github.com/mdempsky/gocode#latest \
github.com/cweill/gotests/...#latest \
github.com/haya14busa/goplay/cmd/goplay#latest \
github.com/sqs/goreturns#latest \
github.com/josharian/impl#latest \
github.com/davidrjenni/reftools/cmd/fillstruct#latest \
github.com/uudashr/gopkgs/v2/cmd/gopkgs#latest \
github.com/ramya-rao-a/go-outline#latest \
github.com/acroca/go-symbols#latest \
github.com/godoctor/godoctor#latest \
github.com/rogpeppe/godef#latest \
github.com/zmb3/gogetdoc#latest \
github.com/fatih/gomodifytags#latest \
github.com/mgechev/revive#latest \
github.com/go-delve/delve/cmd/dlv#latest 2>&1 \
#
# Build Go tools w/o module support
&& GOPATH=/tmp/gotools go get -v github.com/alecthomas/gometalinter 2>&1 \
#
# Build gocode-gomod
&& GOPATH=/tmp/gotools go get -x -d github.com/stamblerre/gocode 2>&1 \
&& GOPATH=/tmp/gotools go build -o gocode-gomod github.com/stamblerre/gocode \
#
# Install Go tools
&& mv /tmp/gotools/bin/* /usr/local/bin/ \
&& mv gocode-gomod /usr/local/bin/ \
#
# Install golangci-lint
&& curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b /usr/local/bin 2>&1 \
#
# Create a non-root user to use if preferred - see https://aka.ms/vscode-remote/containers/non-root-user.
&& groupadd --gid $USER_GID $USERNAME \
&& useradd -s /bin/bash --uid $USER_UID --gid $USER_GID -m $USERNAME \
# [Optional] Add sudo support
&& apt-get install -y sudo \
&& echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME \
&& chmod 0440 /etc/sudoers.d/$USERNAME \
#
# Clean up
&& apt-get autoremove -y \
&& apt-get clean -y \
&& rm -rf /var/lib/apt/lists/* /tmp/gotools
# Update this to "on" or "off" as appropriate
ENV GO111MODULE=auto
This is the devcontainer.json file:
// For format details, see https://aka.ms/vscode-remote/devcontainer.json or this file's README at:
// https://github.com/microsoft/vscode-dev-containers/tree/v0.122.1/containers/go
{
"name": "Go",
"dockerFile": "Dockerfile",
"runArgs": [ "--cap-add=SYS_PTRACE", "--security-opt", "seccomp=unconfined" ],
// Set *default* container specific settings.json values on container create.
"settings": {
"terminal.integrated.shell.linux": "/bin/bash",
"go.gopath": "/go"
},
// Add the IDs of extensions you want installed when the container is created.
"extensions": [
"golang.Go"
],
// Use 'forwardPorts' to make a list of ports inside the container available locally.
"forwardPorts": [3000],
// Use 'postCreateCommand' to run commands after the container is created.
// "postCreateCommand": "go version",
// Uncomment to connect as a non-root user. See https://aka.ms/vscode-remote/containers/non-root.
// "remoteUser": "vscode"
}
This is the docker-compose file I am using for the DB
version: "3.1"
services:
db:
image: postgres:10.10
environment:
POSTGRES_PASSWORD: pass
POSTGRES_USER: user
POSTGRES_DB: my_db
container_name: test_db
ports:
- 1234:5432
restart: "no"
volumes:
- ../postgres-data:/var/lib/postgresql/data
Soda uses a .yml file to connect to the database and its as follows:
development:
dialect: postgres
database: my_db
user: user
password: pass
host: 174.24.0.1
port: 1234
Thanks in advance
Great answer here if anyone is still looking:
If your database is running on host machine, you can use this:
development:
dialect: postgres
database: my_db
user: user
password: pass
host: host.docker.internal
port: 1234

Can't build openjdk:8-jdk image directly

I'm slowly making my way through the Riot Taking Control of your Docker Image tutorial http://engineering.riotgames.com/news/taking-control-your-docker-image. This tutorial is a little old, so there are some definite changes to how the end file looks. After hitting several walls I decided to work in the opposite order of the tutorial. I successfully folded the official jenkinsci image into my personal Dockerfile, starting with FROM: openjdk:8-dk. But when I try to fold in the openjdk:8-dk file into my personal image I receive the following error
E: Version '8u102-b14.1-1~bpo8+1' for 'openjdk-8-jdk' was not found
ERROR: Service 'jenkinsmaster' failed to build: The command '/bin/sh
-c set -x && apt-get update && apt-get install -y openjdk-8-jdk="$JAVA_DEBIAN_VERSION"
ca-certificates-java="$CA_CERTIFICATES_JAVA_VERSION" && rm -rf
/var/lib/apt/lists/* && [ "$JAVA_HOME" = "$(docker-java-home)" ]'
returned a non-zero code: 100 Cosettes-MacBook-Pro:docker-test
Cosette$
I'm receiving this error even when I gave up and directly copied and pasted the openjdk:8-jdk Dockerfile into my own. My end goal is to bring my personal Dockerfile down to the point that it starts FROM debian-jessie. Any help would be appreciated.
My Dockerfile:
FROM buildpack-deps:jessie-scm
# A few problems with compiling Java from source:
# 1. Oracle. Licensing prevents us from redistributing the official JDK.
# 2. Compiling OpenJDK also requires the JDK to be installed, and it gets
# really hairy.
RUN apt-get update && apt-get install -y --no-install-recommends \
bzip2 \
unzip \
xz-utils \
&& rm -rf /var/lib/apt/lists/*
RUN echo 'deb http://deb.debian.org/debian jessie-backports main' > /etc/apt/sources.list.d/jessie-backports.list
# Default to UTF-8 file.encoding
ENV LANG C.UTF-8
# add a simple script that can auto-detect the appropriate JAVA_HOME value
# based on whether the JDK or only the JRE is installed
RUN { \
echo '#!/bin/sh'; \
echo 'set -e'; \
echo; \
echo 'dirname "$(dirname "$(readlink -f "$(which javac || which java)")")"'; \
} > /usr/local/bin/docker-java-home \
&& chmod +x /usr/local/bin/docker-java-home
ENV JAVA_HOME /usr/lib/jvm/java-8-openjdk-amd64
ENV JAVA_VERSION 8u102
ENV JAVA_DEBIAN_VERSION 8u102-b14.1-1~bpo8+1
# see https://bugs.debian.org/775775
# and https://github.com/docker-library/java/issues/19#issuecomment-70546872
ENV CA_CERTIFICATES_JAVA_VERSION 20140324
RUN set -x \
&& apt-get update \
&& apt-get install -y \
openjdk-8-jdk="$JAVA_DEBIAN_VERSION" \
ca-certificates-java="$CA_CERTIFICATES_JAVA_VERSION" \
&& rm -rf /var/lib/apt/lists/* \
&& [ "$JAVA_HOME" = "$(docker-java-home)" ]
# see CA_CERTIFICATES_JAVA_VERSION notes above
RUN /var/lib/dpkg/info/ca-certificates-java.postinst configure
# Jenkins Specifics
# install Tini
ENV TINI_VERSION 0.9.0
ENV TINI_SHA fa23d1e20732501c3bb8eeeca423c89ac80ed452
# Use tini as subreaper in Docker container to adopt zombie processes
RUN curl -fsSL https://github.com/krallin/tini/releases/download/v${TINI_VERSION}/tini-static -o /bin/tini && chmod +x /bin/tini \
&& echo "$TINI_SHA /bin/tini" | sha1sum -c -
# Set Jenkins Environmental Variables
ENV JENKINS_HOME /var/jenkins_home
ENV JENKINS_SLAVE_AGENT_PORT 50000
# jenkins version being bundled in this docker image
ARG JENKINS_VERSION
ENV JENKINS_VERSION ${JENKINS_VERSION:-2.19.1}
# jenkins.war checksum, download will be validated using it
ARG JENKINS_SHA=dc28b91e553c1cd42cc30bd75d0f651671e6de0b
ENV JENKINS_UC https://updates.jenkins.io
ENV COPY_REFERENCE_FILE_LOG $JENKINS_HOME/copy_reference_file.log
ENV JAVA_OPTS="-Xmx8192m"
ENV JENKINS_OPTS="--handlerCountMax=300 --logfile=/var/log/jenkins/jenkins.log --webroot=/var/cache/jenkins/war"
# Can be used to customize where jenkins.war get downloaded from
ARG JENKINS_URL=http://repo.jenkins-ci.org/public/org/jenkins-ci/main/jenkins-war/${JENKINS_VERSION}/jenkins-war-${JENKINS_VERSION}.war
ARG user=jenkins
ARG group=jenkins
ARG uid=1000
ARG gid=1000
# Jenkins is run with user `jenkins`, uid = 1000. If you bind mount a volume from the host or a data
# container, ensure you use the same uid.
RUN groupadd -g ${gid} ${group} \
&& useradd -d "$JENKINS_HOME" -u ${uid} -g ${gid} -m -s /bin/bash ${user}
# Jenkins home directory is a volume, so configuration and build history
# can be persisted and survive image upgrades
VOLUME /var/jenkins_home
# `/usr/share/jenkins/ref/` contains all reference configuration we want
# to set on a fresh new installation. Use it to bundle additional plugins
# or config file with your custom jenkins Docker image.
RUN mkdir -p /usr/share/jenkins/ref/init.groovy.d
# Install Jenkins. Could use ADD but this one does not check Last-Modified header neither does it
# allow to control checksum. see https://github.com/docker/docker/issues/8331
RUN curl -fsSL ${JENKINS_URL} -o /usr/share/jenkins/jenkins.war \
&& echo "${JENKINS_SHA} /usr/share/jenkins/jenkins.war" | sha1sum -c -
# Prep Jenkins Directories
USER root
RUN chown -R ${user} "$JENKINS_HOME" /usr/share/jenkins/ref
RUN mkdir /var/log/jenkins
RUN mkdir /var/cache/jenkins
RUN chown -R ${group}:${user} /var/log/jenkins
RUN chown -R ${group}:${user} /var/cache/jenkins
# Expose ports for web (8080) & node (50000) agents
EXPOSE 8080
EXPOSE 50000
# Copy in local config filesfiles
COPY init.groovy /usr/share/jenkins/ref/init.groovy.d/tcp-slave-agent-port.groovy
COPY jenkins-support /usr/local/bin/jenkins-support
COPY jenkins.sh /usr/local/bin/jenkins.sh
# NOTE : Just set pluginID to download latest version of plugin.
# NOTE : All plugins need to be listed as there is no transitive dependency resolution.
# from a derived Dockerfile, can use `RUN plugins.sh active.txt` to setup
# /usr/share/jenkins/ref/plugins from a support bundle
COPY plugins.sh /usr/local/bin/plugins.sh
RUN chmod +x /usr/local/bin/plugins.sh
RUN chmod +x /usr/local/bin/jenkins.sh
# Switch to the jenkins user
USER ${user}
# Tini as the entry point to manage zombie processes
ENTRYPOINT ["/bin/tini", "--", "/usr/local/bin/jenkins.sh"]
Try a JAVA_DEBIAN_VERSION of 8u111-b14-2~bpo8+1
Here's what happens: when you build the docker file, docker tries to execute all the lines in the dockerfile. One of those is this apt command: apt-get install -y openjdk-8-jdk="$JAVA_DEBIAN_VERSION". This comand says "Install OpenJDK version $JAVA_DEBIAN_VERSION, exactly. Nothing else.". This version is no longer available in Debian repositories, so it can't be apt-get installed! I believe this happens with all packages in official mirrors: if a new version of the package is released, the older version is no longer around to be installed.
If you want to access older Debian packages, you can use something like http://snapshot.debian.org/. The older OpenJDK package has known security vulnerabilities. I recommend using the latest version.
You can use the latest version by leaving out the explicit version in the apt-get command. On the other hand, this will make your image less reproducible: building the image today may get you u111, building it tomorrow may get you u112.
As for why the instructions worked in the other Dockerfile, I think the reason is that at the time the other Dockerfile was built, the package was available. So docker could apt-get install it. Docker then built the image containing the (older) OpenJDK. That image is a binary, so you can install it, or use it in FROM without any issues. But you can't reproduce the image: if you were to try and build the same image yourself, you would run into the same errors.
This also brings up an issue about security updates: since docker images are effectively static binaries (built once, bundle in all dependencies), they don't get security updates once built. You need to keep track of any security updates affecting your docker images and rebuild any affected docker images.

Resources