How to continuously copy files into docker - docker

I am a docker newbie and i can't rly figure out how the changes that will be made to my working directory will be continuously copied to the docker container. Is there a command that copies all my changes to the docker container all the time ?
Edit : i added docker file and docker compose
My docker file
FROM scratch
ADD centos-7-x86_64-docker.tar.xz /
LABEL \
org.label-schema.schema-version="1.0" \
org.label-schema.name="CentOS Base Image" \
org.label-schema.vendor="CentOS" \
org.label-schema.license="GPLv2" \
org.label-schema.build-date="20201113" \
org.opencontainers.image.title="CentOS Base Image" \
org.opencontainers.image.vendor="CentOS" \
org.opencontainers.image.licenses="GPL-2.0-only" \
org.opencontainers.image.created="2020-11-13 00:00:00+00:00"
RUN yum clean all && yum update -y && yum -y upgrade
RUN yum groupinstall "Development Tools" -y
RUN yum install -y wget gettext-devel curl-devel openssl-devel perl-devel perl-CPAN zlib-devel && wget https://github.com/git/git/archive/v2.26.2.tar.gz\
&& tar -xvzf v2.26.2.tar.gz && cd git-2.26.2 && make configure && ./configure --prefix=/usr/local && make install
# RUN mkdir -p /root/.ssh && \
# chmod 0700 /root/.ssh && \
# ssh-keyscan github.com > /root/.ssh/known_hosts
# RUN ssh-keygen -q -t rsa -N '' -f /id_rsa
# RUN echo "$ssh_prv_key" > /root/.ssh/id_rsa && \
# echo "$ssh_pub_key" > /root/.ssh/id_rsa.pub && \
# chmod 600 /root/.ssh/id_rsa && \
# chmod 600 /root/.ssh/id_rsa.pub
RUN ls
RUN cd / && git clone https://github.com/odoo/odoo.git \
&& cd odoo \
&& git fetch \
&& git checkout 9.0
RUN yum install python-devel libxml2-devel libxslt-dev openldap-devel libtiff-devel libjpeg-devel libzip-devel freetype-devel lcms2-devel \
libwebp-devel tcl-devel tk-devel python-pip nodejs
RUN pip install setuptools==1.4.1 beautifulsoup4==4.9.3 pillow openpyxl==2.6.4 luhn gmp-devel paramiko==1.7.7.2 python2-secrets cffi pysftp==0.2.8
RUN pip install -r requirements.txt
RUN npm install -g less
CMD ["/bin/bash","git"]
My docker-compose
version: '3.3'
services:
app: &app
build:
context: .
dockerfile: ./docker/app/Dockerfile
container_name: app
tty: true
db:
image: postgres:9.2.18
environment:
- POSTGRES_DB=test
ports:
- 5432:5432
volumes:
- ./docker/db/pg-data:/var/lib/postgresql/data
odoo:
<<: *app
command: python odoo.py -w odoo -r odoo
ports:
- '8069:8069'
depends_on:
- db

If I understand correctly you want to mount a path from the host into a container which can be done using volumes. Something like this would keep the folders in sync which can be useful for development
docker run -v /path/to/local/folder:/path/in/container busybox

Related

Generating PHP library with Dockerized gRPC

I'm trying to build a gRPC PHP Client and gRPC NodeJs Server in docker. But the problem is I can't install protoc-gen-php-grpc to my docker server. When I try to run this run this makefile:
proto_from_within_container:
# PHP
protoc /var/www/protos/smellycat.proto \
--php_out=/var/www/php-client/src \
$(: 👇 generate server interface) \
--php-grpc_out=/var/www/php-client/src \
$(: 👇 generates the client code) \
--grpc_out=/var/www/php-client/src \
--plugin=protoc-gen-grpc=/protobuf/grpc/bins/opt/grpc_php_plugin \
--proto_path /var/www/protos
proto:
powershell rm -r -fo php-client/src -ErrorAction SilentlyContinue
powershell New-Item -ItemType Directory -Path php-client/src -Force -ErrorAction SilentlyContinue
docker-compose run grpc-server make proto_from_within_container
With this command: make proto
Getting this error message after docker containers builded:
protoc /var/www/protos/smellycat.proto \
--php_out=/var/www/php-client/src \
\
--php-grpc_out=/var/www/php-client/src \
\
--grpc_out=/var/www/php-client/src \
--plugin=protoc-gen-grpc=/protobuf/grpc/bins/opt/grpc_php_plugin \
--proto_path /var/www/protos
protoc-gen-php-grpc: program not found or is not executable
Please specify a program using absolute path or make sure the program is available in your PATH system variable
--php-grpc_out: protoc-gen-php-grpc: Plugin failed with status code 1.
Makefile:4: recipe for target 'proto_from_within_container' failed
make: *** [proto_from_within_container] Error 1
This is my docker-compose file
version: "3"
services:
grpc-server:
container_name: grpc-server
build:
context: .
dockerfile: Dockerfile-server
working_dir: /var/www
volumes:
- .:/var/www
grpc-client:
image: php:7.4-cli
container_name: grpc-client
build:
context: .
dockerfile: Dockerfile-client
working_dir: /var/www
volumes:
- .:/var/www
command: bash -c [php php_client.php && composer install]
And this is my grpc-server docker file:
FROM node:latest
ENV DEBIAN_FRONTEND=noninteractive
#Versions
ARG PROTOBUF_VERSION=3.14.0
ARG PHP_GRPC_VERSION=1.34.0
# Utils
RUN apt-get update -yqq \
&& apt-get install -yqq wget unzip zlib1g-dev git autoconf libtool automake build-essential software-properties-common curl zip \
&& rm -rf /var/lib/apt/lists/*
# Protobuf
RUN mkdir -p /protobuf
RUN cd /protobuf \
&& wget https://github.com/protocolbuffers/protobuf/releases/download/v${PROTOBUF_VERSION}/protoc-${PROTOBUF_VERSION}-linux-x86_64.zip -O protobuf.zip \
&& unzip protobuf.zip && rm protobuf.zip
# grpc PHP (generate client)
RUN apt-get update -yqq && apt-get upgrade -yqq
RUN apt-get install php php-dev php-pear phpunit zlib1g-dev -yqq
RUN pecl install grpc-${PHP_GRPC_VERSION}
RUN cd /protobuf && git clone -b v${PHP_GRPC_VERSION} https://github.com/grpc/grpc \
&& cd /protobuf/grpc && git submodule update --init
RUN cd /protobuf/grpc && make grpc_php_plugin
ENV PATH "/protobuf/bin:${PATH}"
ENV PATH "/protobuf/grpc/bins/opt:${PATH}"
# NPM Installation
WORKDIR /var/www
COPY . /var/www
RUN npm install
CMD ["node", "server.js"]
Do you have any advice?
After a lot of search and readings, I finally managed to build a full application that communicates with each other.
The problem was at the Makefile, at this step:
--plugin=protoc-gen-grpc=/protobuf/grpc/bins/opt/grpc_php_plugin
I was assigning the wrong path for grpc_php_plugin.
There is my new dockerfile:
FROM php:7.4-cli
# Environment variables
ENV DEBIAN_FRONTEND=noninteractive
# Utils
RUN apt-get update -yqq && \
apt-get upgrade -yqq && \
apt-get install -y unzip build-essential git software-properties-common curl pkg-config zip zlib1g-dev
# Composer installation
COPY --from=composer:latest /usr/bin/composer /usr/local/bin/composer
# Install grpc and probuf with pecl
RUN pecl install grpc && pecl install protobuf
# Enable grpc and protobuf extensions in php.ini file
RUN echo starting && \
docker-php-ext-enable grpc && \
docker-php-ext-enable protobuf
# Install cmake
RUN apt-get update -yqq && apt-get -y install cmake
# Install grpc_php_plugin and protoc
RUN git clone -b v1.36.2 https://github.com/grpc/grpc && \
cd grpc && git submodule update --init && \
mkdir cmake/build && cd cmake/build && \
cmake ../.. && make protoc grpc_php_plugin
# Setting node, protoc and grpc_php_plugin paths
ENV PATH "/grpc/cmake/build:${PATH}"
ENV PATH "/grpc/cmake/build/third_party/protobuf:${PATH}"
# Moving client folder to vm
WORKDIR /var/www
COPY ./client /var/www
# Packages
RUN composer install
# Generate php libraries from proto file
RUN make proto
CMD [ "php", "./handler.php" ]
For my full application, click.

Ionic serve hot reload with docker not working

I had an ionic app which I have dockerized. The build and up commands are successful and I can access the app at http://localhost:8100. However, hot reload doesn't work. Whenever I edit some file, those changes are nor reflected until redo an ionic serve, but if I rename some file inside the container, the ionic rebuild the app automatically.
Dockerfile
FROM ubuntu:18.04
ENV DEBIAN_FRONTEND=noninteractiv \
TERM=xterm \
# https://cordova.apache.org/docs/en/8.x/guide/platforms/android/#installing-the-requirements
JAVA_VERSION=8 \
# https://github.com/gradle/gradle/releases
GRADLE_VERSION=5.2.1 \
# https://developer.android.com/studio/releases/sdk-tools V26.1.1
ANDROID_SDK_URL="https://dl.google.com/android/repository/sdk-tools-linux-4333796.zip" \
# https://developer.android.com/studio/releases/build-tools
ANDROID_BUILD_TOOLS_VERSION=28.0.3 \
# https://source.android.com/setup/start/build-numbers
ANDROID_APIS="android-28" \
ANDROID_HOME="/opt/android" \
# https://nodejs.org/en/
NODE_VERSION=11.10.0 \
# https://www.npmjs.com/package/cordova
CORDOVA_VERSION=8.1.2 \
# https://www.npmjs.com/package/ionic
IONIC_VERSION=4.10.3
# dependences
WORKDIR /tmp
RUN mkdir /app && \
buildDeps='software-properties-common'; \
set -x && \
dpkg --add-architecture i386 && \
apt-get -qq update && \
apt-get -qq install -y $buildDeps curl git ca-certificates bzip2 openssh-client unzip wget libncurses5:i386 libstdc++6:i386 zlib1g:i386 --no-install-recommends
# java # use WebUpd8 PPA
RUN add-apt-repository ppa:webupd8team/java -y && \
apt-get -qq update -y && \
# automatically accept the Oracle license
echo oracle-java"$JAVA_VERSION"-installer shared/accepted-oracle-license-v1-1 select true | /usr/bin/debconf-set-selections && \
apt-get -qq install -y oracle-java"$JAVA_VERSION"-installer oracle-java"$JAVA_VERSION"-set-default
ENV JAVA_HOME=/usr/lib/jvm/java-"$JAVA_VERSION"-oracle
# gradle
RUN wget https://services.gradle.org/distributions/gradle-"$GRADLE_VERSION"-bin.zip && \
mkdir /opt/gradle && \
unzip -d /opt/gradle gradle-"$GRADLE_VERSION"-bin.zip
ENV PATH=$PATH:/opt/gradle/gradle-"$GRADLE_VERSION"/bin
# android
WORKDIR /opt/android
ENV PATH $PATH:"$ANDROID_HOME"/platform-tools:"$ANDROID_HOME"/tools:"$ANDROID_HOME"/build-tools/"$ANDROID_BUILD_TOOLS_VERSION"
RUN wget -O tools.zip "$ANDROID_SDK_URL" && \
unzip tools.zip && \
rm tools.zip && \
# echo y | android update sdk -a -u -t platform-tools,"$ANDROID_APIS",build-tools-"$ANDROID_BUILD_TOOLS_VERSION" && \
yes Y | "$ANDROID_HOME"/tools/bin/sdkmanager "build-tools;$ANDROID_BUILD_TOOLS_VERSION" "platforms;$ANDROID_APIS" "platform-tools" && \
chmod a+x -R "$ANDROID_HOME" && \
chown -R root:root "$ANDROID_HOME"
# node
WORKDIR /opt/node
RUN curl -sL https://nodejs.org/dist/v"$NODE_VERSION"/node-v"$NODE_VERSION"-linux-x64.tar.gz | tar xz --strip-components=1
ENV PATH=$PATH:/opt/node/bin
# ionic & cordova
WORKDIR /tmp
RUN npm i -g ionic#"$IONIC_VERSION" cordova#"$CORDOVA_VERSION" && \
ionic config set -g daemon.updates false && \
cordova telemetry off
# clean up
RUN rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* && \
apt-get purge -y --auto-remove $buildDeps && \
apt-get autoremove -y && \
apt-get clean
# app
WORKDIR /app
EXPOSE 8100 35729
docker-compose
version: '3'
networks:
default:
driver: bridge
ipam:
config:
- subnet: 172.18.19.0/24
services:
ionic:
container_name: Ionic
image: ionic
restart: 'no'
networks:
default:
ports:
- "8100:8100"
- "35729:35729"
volumes:
- w:\projects\test:/app
command: "ionic serve"
Is the problem something I did? or will it be a docker / ionic bug?
I found this on github. you can try to put this in /etc/sysctl.conf
fs.inotify.max_user_watches=524288
and then reboot or
sudo sysctl --system

docker compose not finding file in gitlab job

I'm running a gitlab-ci.yml job with a gitlab-runner which is using the docker executor with the docker.sock mounted from the host (set in the config file).
docker composes run.sh file located here: https://github.com/docker/compose/blob/master/script/run/run.sh doesn't seem to find the docker-compose.lint.yml file I need it to run.
Part of the gitlab-ci.yml file:
Lint from image:
<<: *temp
stage: Lint
script:
- ls -a
- cat docker-compose.lint.yml
- . docker-compose_run.sh -f docker-compose.lint.yml up --remove-orphans --force-recreate --abort-on-container-exit
Dockerfile of the image that the gitlab-ci.yml file is using:
FROM debian:stretch
# Set Bash as default shell
RUN rm /bin/sh && \
ln --symbolic /bin/bash /bin/sh
# apt-get
RUN apt-get update && \
apt-get install -y \
# Install SSH
openssh-client \
openssh-server \
openssl \
# Install cURL
curl \
# Install git
git \
# Install locales
locales \
# Install Python
python3 \
python-dev \
python3-pip \
# Build stuff
build-essential \
libssl-dev \
libffi-dev \
apt-transport-https \
ca-certificates \
curl \
gnupg2 \
software-properties-common
# Install Docker
# Add Docker’s official GPG key
RUN curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo "$ID")/gpg | apt-key add - && \
# Set up the stable repository
add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID") \
$(lsb_release -cs) \
stable" && \
# Update package index
apt-get update && \
# Install Docker CE
apt-get install -y docker-ce
# pip
RUN pip3 install \
# Install Docker Compose
docker-compose
# Set locale
RUN sed --in-place '/en_US.UTF-8/s/^# //' /etc/locale.gen && \
locale-gen && \
# Set system locale (add line)
echo "export LANG=en_US.UTF-8" >> /etc/profile && \
# Set system timezone (add line)
echo "export TZ=UTC" >> /etc/profile
Job output:
...more stuff above here
$ ls -a
.
..
.clocignore
.dockerignore
.eslintignore
.eslintrc.json
.git
.gitignore
.gitlab-ci.yml
.nvmrc
.nyc_output
Dockerfile
README.md
__.gitlab-ci.yml
bin
build.sh
build_and_test.sh
coverage
docker-compose.lint.yml
docker-compose.test.yml
docker-compose_run.sh
lib
package-lock.json
package.json
stop.sh
test
test.sh
wait-for-vertica.sh
$ cat docker-compose.lint.yml
version: "3"
services:
ei:
image: lint:1
environment:
- NODE_ENV=test
entrypoint: ["npm", "run", "lint"]
$ . docker-compose_run.sh -f docker-compose.lint.yml up --remove-orphans --force-recreate --abort-on-container-exit
.IOError: [Errno 2] No such file or directory: u'./docker-compose.lint.yml'
ERROR: Job failed: exit code 1
Since this question was the top google result for my search:
executor failed running [/bin/sh -c apt-get install git-all]: exit code: 1
To fix my problem I added -y to the apt-get command, that is changed this:
apt-get install git-all
to this
apt-get install git-all -y

Containerization of Node-Red failing: cannot find module 'express'

I am very new to Docker.
I am getting an error saying "cannot find module 'express'" while trying to containerize simple node-red application. The details are as follows:
Base machine
OS -Debian 9 (stretch) 64-bit
RAM -8 gb
GNOME - 3.22.2
Env - Oracle Virtual Box
Node-Red source
https://github.com/node-red/node-red.git
Docker Version
17.12.0-ce, build c97c6d6
docker-compose -v
1.20.1, build 5d8c71b
Docker File
FROM debian:stretch-slim
RUN useradd -c 'Node-Red user' -m -d /home/nodered -s /bin/bash nodered
RUN chown -R nodered.nodered /home/nodered
RUN echo "Acquire::http::Proxy \"http://xxxx:yyyy";" > /etc/apt/apt.conf.d/01turnkey \
&& echo "Acquire::https::Proxy \"http://xxxx.yyyy";" >> /etc/apt/apt.conf.d/01turnkey
ENV http_proxy="http://xxxx:yyyy \
https_proxy="http://xxxx:yyyy"
USER root
RUN apt-get update && apt-get -y install --no-install-recommends \
ca-certificates \
apt-utils \
curl \
sudo \
git \
python \
make \
g++ \
gnupg2
RUN mkdir -p /home/nodered/shaan-node-red && chown -R nodered.nodered /home/nodered/shaan-node-red
ENV HOME /home/nodered/shaan-node-red
WORKDIR /home/nodered/shaan-node-red
RUN ls -la
RUN env
USER root
RUN echo "nodered ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/nodered && \
chmod 0440 /etc/sudoers.d/nodered
RUN curl -sL https://deb.nodesource.com/setup_9.x | bash -
RUN apt-get -y install nodejs
RUN rm -rf node-v9.x
RUN node -v (v9.9.0) && npm -v (5.6.0)
RUN npm config set proxy "http://xxxx:yyyy" \
npm config set http-proxy "http://xxxx:yyyy"
COPY . /home/nodered/shaan-node-red
RUN cd /home/nodered/shaan-node-red && ls -la && npm install
RUN npm run build && ls -la
RUN cd /home/nodered/shaan-node-red/node_modules/ && git clone https://github.com/netsmarttech/node-red-contrib-s7.git && ls -la | grep s7 && cd ./node-red-contrib-s7 && npm install
RUN ls -la /home/nodered/shaan-node-red/node_modules
ENTRYPOINT ["sh","entrypoint.sh"]
entrypoint.sh
node /home/nodered/shaan-node-red/red.js
Docker-compose.yml
version: '2.0'
services:
web:
image: shaan-node-red
build: .
volumes:
- .:/home/nodered/shaan-node-red
ports:
- "1880:1880"
- "5858:5858"
network_mode: host
Building with command:
docker-compose up
Error description
Note
Not getting any error while building same node-red at the base machine.

Can not run 'varnishadm' inside docker container started with varnishd

I am running docker (via docker-compose) and can't run varnishadm from within the container. The error produced is:
Cannot open /var/lib/varnish/4f0dab1efca3/_.vsm: No such file or directory
Could not open shared memory
I have tried searching on the 'shared memory' issue and _.vsm with no luck. It seems that the _.vsm is not created at all and /var/lib/varnish/ inside the container is empty.
I have tried a variety of -T settings without any luck.
Why run varnishadm?
The root of why I need to run varnishadm is to reload varnish while saving the cache. My backup backup backup option is to set up varnish as a service. We are on an old version of Varnish for the time being.
How am I starting docker?
CMD varnishd -F -f /etc/varnish/varnish.vcl \
-s malloc,1G \
-a :80
Full Dockerfile
FROM ubuntu:12.04
RUN apt-get update \
&& apt-get upgrade -y \
&& apt-get install wget dtrx varnish -y \
&& apt-get install pkg-config autoconf autoconf-archive automake libtool python-docutils libpcre3 libpcre3-dev xsltproc make -y \ && rm -rf /var/lib/apt/lists/*
RUN export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig/
RUN wget https://github.com/varnishcache/varnish-cache/archive/varnish-
3.0.2.tar.gz --no-check-certificate \
&& dtrx -n varnish-3.0.2.tar.gz
WORKDIR /varnish-3.0.2/varnish-cache-varnish-3.0.2/
RUN cd /varnish-3.0.2/varnish-cache-varnish-3.0.2/ && ./autogen.sh &&
cd /varnish-3.0.2/varnish-cache-varnish-3.0.2/ && ./configure && make install
RUN cd / && wget --no-check-certificate https://github.com/Dridi/libvmod-querystring/archive/v0.3.tar.gz && dtrx -n ./v0.3.tar.gz
WORKDIR /v0.3/libvmod-querystring-0.3
RUN ./autogen.sh && ./configure VARNISHSRC=/varnish-3.0.2/varnish-cache-varnish-3.0.2/ && make install
RUN cp /usr/local/lib/varnish/vmods/* /usr/lib/varnish/vmods/
WORKDIR /etc/varnish/
CMD varnishd -F -f /etc/varnish/varnish.vcl \
-s malloc,1G \
-a :80
EXPOSE 80
Full docker-compose
version: "3"
services:
varnish:
build: ./
ports:
- "8000:80"
volumes:
- ./default.vcl:/etc/varnish/varnish.vcl
- ./devicedetect.vcl:/etc/varnish/devicedetect.vcl
restart: unless-stopped

Resources