graphql and serverless offline running in docker - docker

I have this docker file:
FROM node:6-slim
RUN apt-get update \
&& apt-get install -y python2.7 make g++ git bzip2 libfreetype6 libfontconfig1 \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* \
&& ln -s /usr/bin/python2.7 /usr/bin/python
RUN cd $(npm root -g)/npm \
&& npm install serverless -g\
&& npm install fs-extra \
&& sed -i -e s/graceful-fs/fs-extra/ -e s/fs\.move/fs.rename/ ./lib/utils/rename.js
ADD package.json /tmp/package.json
CMD [REV=$(git rev-list -n 1 HEAD 'package.json');]
CMD [STAMP=$(git show --pretty=format:%ai --abbrev-commit "$REV" | head -n 1);]
CMD [touch -d "$STAMP" package.json;]
RUN cd /tmp && npm install --quiet --production \
&& npm rebuild bcrypt \
&& rm -Rf node_modules/ffprobe-static/bin/darwin \
&& rm -Rf node_modules/ffprobe-static/bin/win32
RUN mkdir -p /usr/src/app && cp -a /tmp/node_modules /usr/src/app
WORKDIR /usr/src/app
ADD . /usr/src/app
RUN cd /usr/src/app \
&& git gc --aggressive --prune=all
ENV PORT 9999
EXPOSE 9999
CMD [ "npm", "run", "offline" ]
When I run it, all works without problems, but when I try to access the docker instance:
docker inspect --format '{{ .NetworkSettings.IPAddress }}' serverless
172.17.0.3
When I go to 172.17.0.3:9999 I am not able to connect to the instance.
Whereas if I run my application from my local machine, localhost:9999 responds.
Here is a copy of my serverless.yml:
service: serverless-graphql
plugins:
- serverless-offline
provider:
name: aws
runtime: nodejs4.3
timeout: 30
stage: dev
# you can add statements to the Lambda function's IAM Role here
iamRoleStatements:
- Effect: Allow
Action:
- dynamodb:*
Resource: "*"
- Effect: Allow
Action:
- s3:*
Resource: "*"
# you can define service wide environment variables here
environment:
database: mongo
MONGO_URI: "mongodb://docker-mongo:27017/test-db"
functions:
graphql:
handler: lib/handler.graphql
timeout: 12
events:
- http:
path: graphql
method: post
cors: true
stripewebhook:
handler: lib/handler.stripewebhook
events:
- http:
path: stripewebhook
method: post
cors: true
Any advice is much appreciated.

Related

Cannot start service postgres: driver failed programming external connectivity on endpoint

I am tryning to dockerize my rails application and I've already tried everything including restarting my postgres, restarting docker, removing all containers and images, removing networks, volumes, literally nuking everything relatied to docker but nothing helped.
When I do docker-compose up I get an error
ERROR: for rails-rest-api_postgres_1 Cannot start service postgres: driver failed programming external connectivity on endpoint rails-rest-api_postgres_1 (6211b1f9b1dc688472760c389cc0684542c89cf8afee7dce7c90032f65c58511): Error starting userland proxy: listen tcp4 0.0.0.0:5432: bind: address already in use
lsof -i :5432
doesn't return anything
I've literally everything from this post https://github.com/docker/compose/issues/4126 and nothing helped.
This is my docker-compose.yml
version: '3.5'
services:
postgres:
image: postgres
environment:
POSTGRES_PASSWORD: postgres
ports:
- '5432:5432'
volumes:
- ./tmp/db:/var/lib/postgresql/data
environment:
POSTGRES_PASSWORD: password
web:
build: .
command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
volumes:
- .:/myapp
ports:
- "3000:3000"
depends_on:
- postgres
Dockerfile
FROM ruby:2.7.4-alpine as base
RUN echo "#edge http://dl-cdn.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories
RUN apk update && apk upgrade && apk add build-base && apk --no-cache add \
tzdata \
bash \
git \
libstdc++ \
ca-certificates \
libffi-dev \
postgresql-dev \
postgresql-client \
linux-headers \
libpq \
openssh \
file \
libxml2-dev \
curl \
gmp-dev \
musl \
gcompat \
aws-cli#edge \
shared-mime-info \
libucontext-dev \
&& echo ‘gem: --no-document’ > /etc/gemrc
ARG IMAGE_TAG
ENV IMAGE_TAG=$IMAGE_TAG
RUN mkdir -p /app/vendor/gems
WORKDIR /app
COPY Gemfile /app/Gemfile
COPY Gemfile.lock /app/Gemfile.lock
RUN gem install bundler:2.2.24
RUN bundle config set --local deployment 'true'
RUN bundle config set --local without 'development test'
RUN bundle config --delete without
RUN bundle config --delete with
RUN bundle install
COPY . /app
RUN RAILS_ENV=staging \
DATABASE_URL=postgres:null \
SECRET_KEY_BASE=blah
ENTRYPOINT ["bundle", "exec"]
CMD ["rails", "server", "-b", "0.0.0.0"]
# Deploy Image
FROM base as deploy
RUN rm -rf /root/.ssh
# Dev Image
FROM base as dev
RUN bundle install --with development
RUN apk update && apk upgrade && apk --no-cache add \
curl-dev \
postgresql \
&& echo ‘gem: --no-document’ > /etc/gemrc
RUN rm -rf /root/.ssh
# CI Image
FROM base as ci
RUN apk update && apk upgrade && apk --no-cache add \
curl-dev \
postgresql \
&& echo ‘gem: --no-document’ > /etc/gemrc
RUN bundle install \
--with test \
--deployment
RUN rm -rf /root/.ssh
COPY config/database.ci.yml config/database.yml
I have fix this issue.
sudo services postgresql stop # or sudo systemctl stop postgresql
docker-compose up
It works for you

How to continuously copy files into docker

I am a docker newbie and i can't rly figure out how the changes that will be made to my working directory will be continuously copied to the docker container. Is there a command that copies all my changes to the docker container all the time ?
Edit : i added docker file and docker compose
My docker file
FROM scratch
ADD centos-7-x86_64-docker.tar.xz /
LABEL \
org.label-schema.schema-version="1.0" \
org.label-schema.name="CentOS Base Image" \
org.label-schema.vendor="CentOS" \
org.label-schema.license="GPLv2" \
org.label-schema.build-date="20201113" \
org.opencontainers.image.title="CentOS Base Image" \
org.opencontainers.image.vendor="CentOS" \
org.opencontainers.image.licenses="GPL-2.0-only" \
org.opencontainers.image.created="2020-11-13 00:00:00+00:00"
RUN yum clean all && yum update -y && yum -y upgrade
RUN yum groupinstall "Development Tools" -y
RUN yum install -y wget gettext-devel curl-devel openssl-devel perl-devel perl-CPAN zlib-devel && wget https://github.com/git/git/archive/v2.26.2.tar.gz\
&& tar -xvzf v2.26.2.tar.gz && cd git-2.26.2 && make configure && ./configure --prefix=/usr/local && make install
# RUN mkdir -p /root/.ssh && \
# chmod 0700 /root/.ssh && \
# ssh-keyscan github.com > /root/.ssh/known_hosts
# RUN ssh-keygen -q -t rsa -N '' -f /id_rsa
# RUN echo "$ssh_prv_key" > /root/.ssh/id_rsa && \
# echo "$ssh_pub_key" > /root/.ssh/id_rsa.pub && \
# chmod 600 /root/.ssh/id_rsa && \
# chmod 600 /root/.ssh/id_rsa.pub
RUN ls
RUN cd / && git clone https://github.com/odoo/odoo.git \
&& cd odoo \
&& git fetch \
&& git checkout 9.0
RUN yum install python-devel libxml2-devel libxslt-dev openldap-devel libtiff-devel libjpeg-devel libzip-devel freetype-devel lcms2-devel \
libwebp-devel tcl-devel tk-devel python-pip nodejs
RUN pip install setuptools==1.4.1 beautifulsoup4==4.9.3 pillow openpyxl==2.6.4 luhn gmp-devel paramiko==1.7.7.2 python2-secrets cffi pysftp==0.2.8
RUN pip install -r requirements.txt
RUN npm install -g less
CMD ["/bin/bash","git"]
My docker-compose
version: '3.3'
services:
app: &app
build:
context: .
dockerfile: ./docker/app/Dockerfile
container_name: app
tty: true
db:
image: postgres:9.2.18
environment:
- POSTGRES_DB=test
ports:
- 5432:5432
volumes:
- ./docker/db/pg-data:/var/lib/postgresql/data
odoo:
<<: *app
command: python odoo.py -w odoo -r odoo
ports:
- '8069:8069'
depends_on:
- db
If I understand correctly you want to mount a path from the host into a container which can be done using volumes. Something like this would keep the folders in sync which can be useful for development
docker run -v /path/to/local/folder:/path/in/container busybox

How to access remote debugging page for dockerized Chromium launch by Puppeteer?

When the chromium succeed to launch, its Debugging WebSocket URL should be like ws://127.0.0.1:9222/devtools/browser/ec261e61-0e52-4016-a5d7-d541e82ecb0a.
127.0.0.1:9222 should be able to browse by Chrome to inspect the headless Chromium. However, I cannot access the remote debugger URL by Chrome after I dockerize my application.
launchOption for launching chromium by Puppeteer:
{
"args": [
"--remote-debugging-port=9222",
"--window-size=1920,1080",
"--mute-audio",
"--disable-notifications",
"--force-device-scale-factor=0.8",
"--no-sandbox",
"--disable-setuid-sandbox"
],
"defaultViewport": {
"height": 1080,
"width": 1920
},
"headless": true
}
Dockerfile:
FROM node:10.16.3-slim
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
&& apt-get update \
&& apt-get install -y google-chrome-unstable fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst fonts-freefont-ttf \
--no-install-recommends \
&& rm -rf /var/lib/apt/lists/* \
&& wget --quiet https://raw.githubusercontent.com/vishnubob/wait-for-it/master/wait-for-it.sh -O /usr/sbin/wait-for-it.sh \
&& chmod +x /usr/sbin/wait-for-it.sh
WORKDIR /usr/app
COPY ./ ./
VOLUME ["......." ]
RUN groupadd -r pptruser && useradd -r -g pptruser -G audio,video pptruser \
&& mkdir -p /home/pptruser/Downloads \
&& chown -R pptruser:pptruser /home/pptruser \
&& chown -R pptruser:pptruser /usr/app \
&& npm install
USER pptruser
CMD npm run start
EXPOSE 3000 9222
Run the new container by :
docker run \
-p 3000:3000 \
-p 9222:9222 \
pptr
Port 9222 should be accessible in my host machine. But Chrome shows the error ERR_EMPTY_RESPONSE when I browse 127.0.0.1:9222 and DOCKER-INTERNAL-IP:9222 will timeout.
I managed to make this work with puppeteer using the following Dockerfile, docker run and puppeteer config:
FROM ubuntu:18.04
RUN apt update \
&& apt install -y \
curl \
wget \
gnupg \
gcc \
g++ \
make \
&& curl -sL https://deb.nodesource.com/setup_10.x | bash - \
&& apt install -y nodejs \
&& rm -rf /var/lib/apt/lists/*
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
&& apt-get update \
&& apt-get install -y google-chrome-unstable fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst fonts-freefont-ttf \
--no-install-recommends \
&& rm -rf /var/lib/apt/lists/*
RUN groupadd -r pptruser && useradd -r -g pptruser -G audio,video pptruser \
&& mkdir -p /home/pptruser/Downloads \
&& chown -R pptruser:pptruser /home/pptruser
ADD . /app
WORKDIR /app
RUN chown -R pptruser:pptruser /app
RUN rm -rf node_modules
RUN rm -rf build/*
USER pptruser
RUN npm install --dev
RUN chmod +x /app/entrypoint.sh
ENTRYPOINT /app/entrypoint.sh
Docker run:
docker run -p 9223:9222 -it myimage
Puppeteer launch:
this.browser = await puppeteer.launch(
{
headless: true,
args: [
'--remote-debugging-port=9222',
'--remote-debugging-address=0.0.0.0',
'--no-sandbox'
]
}
);
The entrypoint just launches the platform like: node build/main.js
After that I just had to connect to localhost:9223 on Chrome to see the browser. Hope it helps!
I know there is already an accepted answer, but let me add onto this in hopes to greatly reduce your image size. One shouldn't add too many extras into the Dockerfile if one can help it. But ultimately, adding --remote-debugging-port=9222 and --remote-debugging-address=0.0.0.0 will allow you to access it.
Dockerfile
FROM ubuntu:latest
LABEL Full Name <email#email.com> https://yourwebsite.com
WORKDIR /home/
COPY wrapper-script.sh wrapper-script.sh
# install chromium-browser and cleanup.
RUN apt update && apt install chromium-browser --no-install-recommends -y && apt autoremove && apt clean && apt autoclean && rm -rf /var/lib/apt/lists/*
# Run your commands and add environment variables from your compose file.
CMD ["sh", "wrapper-script.sh"]
I use a wrapper script so that I can include environment variables here. You can see URL and USERNAME set so that I can configure them from the compose file. Of course, i'm sure there is a better way to do this, but I do this so that I can scale my containers horizontally with ease.
wrapper-script.sh
#!/bin/bash
# Start the process
chromium-browser --headless --disable-gpu --no-sandbox --remote-debugging-port=9222 --remote-debugging-address=0.0.0.0 ${URL}${USERNAME}
status=$?
if [ $status -ne 0 ]; then
echo "Failed to start chromium-browser: $status"
exit $status
fi
# Naive check runs checks once a minute to see if either of the processes exited.
# This illustrates part of the heavy lifting you need to do if you want to run
# more than one service in a container. The container exits with an error
# if it detects that either of the processes has exited.
# Otherwise it loops forever, waking up every 60 seconds
while sleep 60; do
ps aux |grep chromium-browser | grep -q -v grep
PROCESS_1_STATUS=$?
# If the greps above find anything, they exit with 0 status
# If they are not both 0, then something is wrong
if [ $PROCESS_1_STATUS -ne 0 ]; then
echo "One of the processes has already exited."
exit 1
fi
done
Lastly, I have the docker-compose file. This is where I define all my settings so that I can configure my wrapper-script.sh with what I need and scale horizontally. Notice the environment section of the docker-compose file. USERNAME and URL are environment variables, and they can be called from the wrapper script.
docker-compose.yml
version: '3.7'
services:
chrome:
command: [ 'sh', 'wrapper-script.sh' ]
image: headless-chrome
build:
context: .
dockerfile: Dockerfile
environment:
- USERNAME=eaglejs
- URL=https://teamtreehouse.com/
ports:
- 9222:9222
If you are wondering what my folder structure looks like. all three files are at the root of the folder. For example:
My_Docker_Repo:
Dockerfile
docker-compose.yml
wrapper-script.sh
After that is all said and done, I simply run docker-compose up and I have one container running. Right now, using the ports section, you'll have to do something to scale that as well. if you were to run docker-compose up --scale chrome=5 your ports will clash, but let me know if you want to try that and i'll see what I can do for scaling, but other than that, if it is for testing, this should work well the way it is. :) Happy coding!
eaglejs

Containerization of Node-Red failing: cannot find module 'express'

I am very new to Docker.
I am getting an error saying "cannot find module 'express'" while trying to containerize simple node-red application. The details are as follows:
Base machine
OS -Debian 9 (stretch) 64-bit
RAM -8 gb
GNOME - 3.22.2
Env - Oracle Virtual Box
Node-Red source
https://github.com/node-red/node-red.git
Docker Version
17.12.0-ce, build c97c6d6
docker-compose -v
1.20.1, build 5d8c71b
Docker File
FROM debian:stretch-slim
RUN useradd -c 'Node-Red user' -m -d /home/nodered -s /bin/bash nodered
RUN chown -R nodered.nodered /home/nodered
RUN echo "Acquire::http::Proxy \"http://xxxx:yyyy";" > /etc/apt/apt.conf.d/01turnkey \
&& echo "Acquire::https::Proxy \"http://xxxx.yyyy";" >> /etc/apt/apt.conf.d/01turnkey
ENV http_proxy="http://xxxx:yyyy \
https_proxy="http://xxxx:yyyy"
USER root
RUN apt-get update && apt-get -y install --no-install-recommends \
ca-certificates \
apt-utils \
curl \
sudo \
git \
python \
make \
g++ \
gnupg2
RUN mkdir -p /home/nodered/shaan-node-red && chown -R nodered.nodered /home/nodered/shaan-node-red
ENV HOME /home/nodered/shaan-node-red
WORKDIR /home/nodered/shaan-node-red
RUN ls -la
RUN env
USER root
RUN echo "nodered ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/nodered && \
chmod 0440 /etc/sudoers.d/nodered
RUN curl -sL https://deb.nodesource.com/setup_9.x | bash -
RUN apt-get -y install nodejs
RUN rm -rf node-v9.x
RUN node -v (v9.9.0) && npm -v (5.6.0)
RUN npm config set proxy "http://xxxx:yyyy" \
npm config set http-proxy "http://xxxx:yyyy"
COPY . /home/nodered/shaan-node-red
RUN cd /home/nodered/shaan-node-red && ls -la && npm install
RUN npm run build && ls -la
RUN cd /home/nodered/shaan-node-red/node_modules/ && git clone https://github.com/netsmarttech/node-red-contrib-s7.git && ls -la | grep s7 && cd ./node-red-contrib-s7 && npm install
RUN ls -la /home/nodered/shaan-node-red/node_modules
ENTRYPOINT ["sh","entrypoint.sh"]
entrypoint.sh
node /home/nodered/shaan-node-red/red.js
Docker-compose.yml
version: '2.0'
services:
web:
image: shaan-node-red
build: .
volumes:
- .:/home/nodered/shaan-node-red
ports:
- "1880:1880"
- "5858:5858"
network_mode: host
Building with command:
docker-compose up
Error description
Note
Not getting any error while building same node-red at the base machine.

Can not run 'varnishadm' inside docker container started with varnishd

I am running docker (via docker-compose) and can't run varnishadm from within the container. The error produced is:
Cannot open /var/lib/varnish/4f0dab1efca3/_.vsm: No such file or directory
Could not open shared memory
I have tried searching on the 'shared memory' issue and _.vsm with no luck. It seems that the _.vsm is not created at all and /var/lib/varnish/ inside the container is empty.
I have tried a variety of -T settings without any luck.
Why run varnishadm?
The root of why I need to run varnishadm is to reload varnish while saving the cache. My backup backup backup option is to set up varnish as a service. We are on an old version of Varnish for the time being.
How am I starting docker?
CMD varnishd -F -f /etc/varnish/varnish.vcl \
-s malloc,1G \
-a :80
Full Dockerfile
FROM ubuntu:12.04
RUN apt-get update \
&& apt-get upgrade -y \
&& apt-get install wget dtrx varnish -y \
&& apt-get install pkg-config autoconf autoconf-archive automake libtool python-docutils libpcre3 libpcre3-dev xsltproc make -y \ && rm -rf /var/lib/apt/lists/*
RUN export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig/
RUN wget https://github.com/varnishcache/varnish-cache/archive/varnish-
3.0.2.tar.gz --no-check-certificate \
&& dtrx -n varnish-3.0.2.tar.gz
WORKDIR /varnish-3.0.2/varnish-cache-varnish-3.0.2/
RUN cd /varnish-3.0.2/varnish-cache-varnish-3.0.2/ && ./autogen.sh &&
cd /varnish-3.0.2/varnish-cache-varnish-3.0.2/ && ./configure && make install
RUN cd / && wget --no-check-certificate https://github.com/Dridi/libvmod-querystring/archive/v0.3.tar.gz && dtrx -n ./v0.3.tar.gz
WORKDIR /v0.3/libvmod-querystring-0.3
RUN ./autogen.sh && ./configure VARNISHSRC=/varnish-3.0.2/varnish-cache-varnish-3.0.2/ && make install
RUN cp /usr/local/lib/varnish/vmods/* /usr/lib/varnish/vmods/
WORKDIR /etc/varnish/
CMD varnishd -F -f /etc/varnish/varnish.vcl \
-s malloc,1G \
-a :80
EXPOSE 80
Full docker-compose
version: "3"
services:
varnish:
build: ./
ports:
- "8000:80"
volumes:
- ./default.vcl:/etc/varnish/varnish.vcl
- ./devicedetect.vcl:/etc/varnish/devicedetect.vcl
restart: unless-stopped

Resources