I am tryning to dockerize my rails application and I've already tried everything including restarting my postgres, restarting docker, removing all containers and images, removing networks, volumes, literally nuking everything relatied to docker but nothing helped.
When I do docker-compose up I get an error
ERROR: for rails-rest-api_postgres_1 Cannot start service postgres: driver failed programming external connectivity on endpoint rails-rest-api_postgres_1 (6211b1f9b1dc688472760c389cc0684542c89cf8afee7dce7c90032f65c58511): Error starting userland proxy: listen tcp4 0.0.0.0:5432: bind: address already in use
lsof -i :5432
doesn't return anything
I've literally everything from this post https://github.com/docker/compose/issues/4126 and nothing helped.
This is my docker-compose.yml
version: '3.5'
services:
postgres:
image: postgres
environment:
POSTGRES_PASSWORD: postgres
ports:
- '5432:5432'
volumes:
- ./tmp/db:/var/lib/postgresql/data
environment:
POSTGRES_PASSWORD: password
web:
build: .
command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
volumes:
- .:/myapp
ports:
- "3000:3000"
depends_on:
- postgres
Dockerfile
FROM ruby:2.7.4-alpine as base
RUN echo "#edge http://dl-cdn.alpinelinux.org/alpine/edge/community" >> /etc/apk/repositories
RUN apk update && apk upgrade && apk add build-base && apk --no-cache add \
tzdata \
bash \
git \
libstdc++ \
ca-certificates \
libffi-dev \
postgresql-dev \
postgresql-client \
linux-headers \
libpq \
openssh \
file \
libxml2-dev \
curl \
gmp-dev \
musl \
gcompat \
aws-cli#edge \
shared-mime-info \
libucontext-dev \
&& echo ‘gem: --no-document’ > /etc/gemrc
ARG IMAGE_TAG
ENV IMAGE_TAG=$IMAGE_TAG
RUN mkdir -p /app/vendor/gems
WORKDIR /app
COPY Gemfile /app/Gemfile
COPY Gemfile.lock /app/Gemfile.lock
RUN gem install bundler:2.2.24
RUN bundle config set --local deployment 'true'
RUN bundle config set --local without 'development test'
RUN bundle config --delete without
RUN bundle config --delete with
RUN bundle install
COPY . /app
RUN RAILS_ENV=staging \
DATABASE_URL=postgres:null \
SECRET_KEY_BASE=blah
ENTRYPOINT ["bundle", "exec"]
CMD ["rails", "server", "-b", "0.0.0.0"]
# Deploy Image
FROM base as deploy
RUN rm -rf /root/.ssh
# Dev Image
FROM base as dev
RUN bundle install --with development
RUN apk update && apk upgrade && apk --no-cache add \
curl-dev \
postgresql \
&& echo ‘gem: --no-document’ > /etc/gemrc
RUN rm -rf /root/.ssh
# CI Image
FROM base as ci
RUN apk update && apk upgrade && apk --no-cache add \
curl-dev \
postgresql \
&& echo ‘gem: --no-document’ > /etc/gemrc
RUN bundle install \
--with test \
--deployment
RUN rm -rf /root/.ssh
COPY config/database.ci.yml config/database.yml
I have fix this issue.
sudo services postgresql stop # or sudo systemctl stop postgresql
docker-compose up
It works for you
Related
I am a docker newbie and i can't rly figure out how the changes that will be made to my working directory will be continuously copied to the docker container. Is there a command that copies all my changes to the docker container all the time ?
Edit : i added docker file and docker compose
My docker file
FROM scratch
ADD centos-7-x86_64-docker.tar.xz /
LABEL \
org.label-schema.schema-version="1.0" \
org.label-schema.name="CentOS Base Image" \
org.label-schema.vendor="CentOS" \
org.label-schema.license="GPLv2" \
org.label-schema.build-date="20201113" \
org.opencontainers.image.title="CentOS Base Image" \
org.opencontainers.image.vendor="CentOS" \
org.opencontainers.image.licenses="GPL-2.0-only" \
org.opencontainers.image.created="2020-11-13 00:00:00+00:00"
RUN yum clean all && yum update -y && yum -y upgrade
RUN yum groupinstall "Development Tools" -y
RUN yum install -y wget gettext-devel curl-devel openssl-devel perl-devel perl-CPAN zlib-devel && wget https://github.com/git/git/archive/v2.26.2.tar.gz\
&& tar -xvzf v2.26.2.tar.gz && cd git-2.26.2 && make configure && ./configure --prefix=/usr/local && make install
# RUN mkdir -p /root/.ssh && \
# chmod 0700 /root/.ssh && \
# ssh-keyscan github.com > /root/.ssh/known_hosts
# RUN ssh-keygen -q -t rsa -N '' -f /id_rsa
# RUN echo "$ssh_prv_key" > /root/.ssh/id_rsa && \
# echo "$ssh_pub_key" > /root/.ssh/id_rsa.pub && \
# chmod 600 /root/.ssh/id_rsa && \
# chmod 600 /root/.ssh/id_rsa.pub
RUN ls
RUN cd / && git clone https://github.com/odoo/odoo.git \
&& cd odoo \
&& git fetch \
&& git checkout 9.0
RUN yum install python-devel libxml2-devel libxslt-dev openldap-devel libtiff-devel libjpeg-devel libzip-devel freetype-devel lcms2-devel \
libwebp-devel tcl-devel tk-devel python-pip nodejs
RUN pip install setuptools==1.4.1 beautifulsoup4==4.9.3 pillow openpyxl==2.6.4 luhn gmp-devel paramiko==1.7.7.2 python2-secrets cffi pysftp==0.2.8
RUN pip install -r requirements.txt
RUN npm install -g less
CMD ["/bin/bash","git"]
My docker-compose
version: '3.3'
services:
app: &app
build:
context: .
dockerfile: ./docker/app/Dockerfile
container_name: app
tty: true
db:
image: postgres:9.2.18
environment:
- POSTGRES_DB=test
ports:
- 5432:5432
volumes:
- ./docker/db/pg-data:/var/lib/postgresql/data
odoo:
<<: *app
command: python odoo.py -w odoo -r odoo
ports:
- '8069:8069'
depends_on:
- db
If I understand correctly you want to mount a path from the host into a container which can be done using volumes. Something like this would keep the folders in sync which can be useful for development
docker run -v /path/to/local/folder:/path/in/container busybox
I am new to docker and was trying to integrate docker with my rails app. I was following this doc
I was able to setup the services but I am facing a problem.
Whenever I add a Gem to my Gemfile I get error in docker-compose up saying
sidekiq_1 | bundler: failed to load command: sidekiq (/usr/local/bundle/bin/sidekiq)
sidekiq_1 | /usr/local/bundle/gems/bundler-2.2.3/lib/bundler/resolver.rb:309:in `block in verify_gemfile_dependencies_are_found!': Could not find gem 'awesome_print' in any of the gem sources listed in your Gemfile. (Bundler::GemNotFound)
Dockerfile
FROM ruby:3.0.0-alpine3.12
ENV BUNDLER_VERSION=2.2.3
RUN apk add --update --no-cache \
binutils-gold \
build-base \
curl py-pip \
curl \
file \
g++ \
gcc \
git \
less \
libstdc++ \
libffi-dev \
libc-dev \
linux-headers \
libxml2-dev \
libxslt-dev \
libgcrypt-dev \
make \
netcat-openbsd \
nodejs \
openssl \
pkgconfig \
python3 \
tzdata \
yarn
RUN gem install bundler -v 2.2.3
WORKDIR /app
COPY Gemfile Gemfile.lock ./
RUN bundle config build.nokogiri --use-system-libraries
RUN bundle check || bundle install
COPY package.json yarn.lock ./
RUN yarn install --check-files
COPY . ./
ENTRYPOINT ["./entrypoints/docker-entrypoint.sh"]
docker-compose.yml
version: '3.6'
services:
app:
build:
context: .
dockerfile: Dockerfile
container_name: sna-main
depends_on:
- redis
ports:
- "3000:3000"
volumes:
- .:/app
- gem_cache:/usr/local/bundle/gems
- node_modules:/app/node_modules
environment:
RAILS_ENV: development
redis:
image: redis:5.0.7
sidekiq:
build:
context: .
dockerfile: Dockerfile
depends_on:
- app
- redis
volumes:
- .:/app
- gem_cache:/usr/local/bundle/gems
- node_modules:/app/node_modules
environment:
RAILS_ENV: development
entrypoint: ./entrypoints/sidekiq-entrypoint.sh
volumes:
gem_cache:
node_modules:
It would be great if someone can help me out to understand how to run bundle install every time there is a change in Gemfile.
From the doc I have understood that because we are caching the Gems so we need to remove the volume explicitly if we add/remove a new gem.
I did remove the volumes by running docker-compose down -v.
It removed the volumes but still getting the same error.
There's a chance that this is because docker-compose cached your images; you need to re-build with docker-compose up --build to reflect the image changes in docker-compose.
I got it work by adding bundle install in my entrypoint file
----------------./entrypoints/docker-entrypoint.sh-----------------
#!/bin/sh
set -e
if [ -f tmp/pids/server.pid ]; then
rm tmp/pids/server.pid
fi
bundle install
bundle exec rails s -b 0.0.0.0
----------------./entrypoints/sidekiq-entrypoint.sh-----------------
#!/bin/sh
set -e
if [ -f tmp/pids/server.pid ]; then
rm tmp/pids/server.pid
fi
bundle install && bundle exec sidekiq -C config/sidekiq.yml
when I start my container with docker-compose up my app eventually gets stuck and repeats itself over and over with these lines:
I, [2019-12-09T15:19:35.396147 #342] INFO -- : Refreshing Gem list
E, [2019-12-09T15:19:47.019123 #342] ERROR -- : No such file or directory # rb_sysopen - /usr/src/app/config/environments/development.rb (Errno::ENOENT)
.
.
.
The thing is, when I open the container bash and use ls, I can see that the file is in the path that he can't seem to find.
Any advice on how to fix this problem? Thank you!
EDIT: Dockerfile
FROM ruby:2.6.3-alpine3.9
ENV APP_HOME /usr/src/app
ENV CONTAINERNAME_ENVIRONMENT docker
EXPOSE 80
WORKDIR $APP_HOME
COPY Gemfile* $APP_HOME/
# general dependencies
RUN apk update
RUN set -ex \
&& apk add --no-cache libpq imagemagick nodejs bash gcompat git
# poltergeist, see https://github.com/Overbryd/docker-phantomjs-alpine/releases
ENV PHANHOME /usr/share
RUN apk add --no-cache fontconfig curl && \
mkdir -p $PHANHOME \
&& curl -L https://github.com/Overbryd/docker-phantomjs-alpine/releases/download/2.11/phantomjs-alpine-x86_64.tar.bz2 | tar xj -C $PHANHOME \
&& ln -s $PHANHOME/phantomjs/phantomjs /usr/bin/phantomjs \
&& phantomjs --version
# build dependencies
RUN set -ex \
&& apk add --no-cache --virtual builddependencies \
linux-headers \
libpq \
tzdata \
build-base \
postgresql-dev \
imagemagick-dev \
&& bundle install \
&& apk del builddependencies
CMD ["bundle", "exec", "unicorn", "--port", "80"]
EDIT: docker-compose.yml
version: '3'
services:
postgresql:
image: postgres:9.6.2-alpine
container_name: postgresql-dev
environment:
- APPLICATION=myapp
- POSTGRES_PASSWORD=myapp
- POSTGRES_USER=my_app
- POSTGRES_DB=myapp
ports:
- 5432:5432
imimap:
build: .
image: myapp:dev
container_name: myapp-dev
environment:
- APPLICATION=myapp
- POSTGRES_PASSWORD=myapp
- POSTGRES_USER=my_app
- POSTGRES_DB=myapp
- RAILS_MASTER_KEY=${RAILS_MASTER_KEY}
- LDAP=${LDAP}
depends_on:
- postgresql
volumes:
- ./logc:/usr/src/app/log
- .:/usr/src/app
links:
- postgresql
ports:
- 8080:80
#entrypoint:
# - ci-cd/stall.sh
Changes in my requirements.txt are not being reflected when I run:
docker-compose -f docker-compose-dev.yml up -d
docker-compose-dev.yml
version: '3.6'
services:
web:
build:
context: ./services/web
dockerfile: Dockerfile-dev
volumes:
- './services/web:/usr/src/app'
ports:
- 5001:5000
environment:
- FLASK_ENV=development
- APP_SETTINGS=project.config.DevelopmentConfig
- DATABASE_URL=postgres://postgres:postgres#web-db:5432/web_dev
- DATABASE_TEST_URL=postgres://postgres:postgres#web-db:5432/web_test
depends_on:
- web-db
web-db:
build:
context: ./services/web/project/db
dockerfile: Dockerfile
ports:
- 5435:5432
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
nginx:
build:
context: ./services/nginx
dockerfile: Dockerfile-dev
restart: always
ports:
- 80:80
depends_on:
- web
- client
client:
build:
context: ./services/client
dockerfile: Dockerfile-dev
volumes:
- './services/client:/usr/src/app'
- '/usr/src/app/node_modules'
ports:
- 3007:3000
environment:
- NODE_ENV=development
- REACT_APP_WEB_SERVICE_URL=${REACT_APP_WEB_SERVICE_URL}
depends_on:
- web
Dockerfile-dev
# base image
FROM python:3.6-alpine
# install dependencies
RUN apk update && \
apk add --virtual build-deps gcc python-dev musl-dev && \
apk add libffi-dev && \
apk add postgresql-dev && \
apk add netcat-openbsd && \
apk add bind-tools && \
apk add --update --no-cache g++ libxslt-dev && \
apk add jpeg-dev zlib-dev
ENV PACKAGES="\
dumb-init \
musl \
libc6-compat \
linux-headers \
build-base \
bash \
git \
ca-certificates \
freetype \
libgfortran \
libgcc \
libstdc++ \
openblas \
tcl \
tk \
libssl1.0 \
"
ENV PYTHON_PACKAGES="\
numpy \
matplotlib \
scipy \
scikit-learn \
nltk \
"
RUN apk add --no-cache --virtual build-dependencies python3 \
&& apk add --virtual build-runtime \
build-base python3-dev openblas-dev freetype-dev pkgconfig gfortran \
&& ln -s /usr/include/locale.h /usr/include/xlocale.h \
&& python3 -m ensurepip \
&& rm -r /usr/lib/python*/ensurepip \
&& pip3 install --upgrade pip setuptools \
&& ln -sf /usr/bin/python3 /usr/bin/python \
&& ln -sf pip3 /usr/bin/pip \
&& rm -r /root/.cache \
&& pip install --no-cache-dir $PYTHON_PACKAGES \
&& pip3 install 'pandas<0.21.0' \
&& apk del build-runtime \
&& apk add --no-cache --virtual build-dependencies $PACKAGES \
&& rm -rf /var/cache/apk/*
# set working directory
WORKDIR /usr/src/app
# add and install requirements
COPY ./requirements.txt /usr/src/app/requirements.txt # <--- refer to EDIT
RUN pip install -r requirements.txt
# add entrypoint.sh
COPY ./entrypoint.sh /usr/src/app/entrypoint.sh
RUN chmod +x /usr/src/app/entrypoint.sh
# add app
COPY . /usr/src/app
# run server
CMD ["/usr/src/app/entrypoint.sh"]
what am I missing?
EDIT
Like the accepted answer in [Docker how to run pip requirements.txt only if there was a change?, I'm already copying the requirements.txt file in a separate build step before adding the entire application into the image, but it does not seem to work.
I think the problem likely is that $ docker-compose up alone will not rebuild your images if you make changes. In order to get docker-compose to include your changes to your requirements.txt you will need to pass the --build flag to docker-compose.
I.e instead run:
docker-compose -f docker-compose-dev.yml up --build -d
Which will force a docker-compose rebuild the image. However this will rebuild all images in the docker-compose file which may or may not be desired.
If you only want to rebuild the image of a single service you can first run docker-compose -f docker-compose-dev.yml build web, then afterwards just run your original docker-compose command.
More info on the build command here.
Try to install requirements from the copied file
https://docs.docker.com/develop/develop-images/dockerfile_best-practices/
It is an example of their Dockerfile
COPY requirements.txt /tmp/
RUN pip install --requirement /tmp/requirements.txt
This is what you have
RUN pip install -r requirements.txt
Then after you have changed your docker file, you have to stop your container, remove your image, build a new one, and run container from it.
Stop container and remove the image.
docker-compose down
docker-compose --rmi all
--rmi all - removes all images. You might want to use --rmi IMAGE_NAME
And to start it (if you use not default parameters, change these commands with your arguments).
docker-compose up
Update
In case you have running docker and you do not want to stop it and rebuild an image (if you just want to install a package or run some commands or even start a new application), you can connect the container from your local machine and run command line commands.
docker exec -it [CONTAINER_ID] bash
To get [CONTAINER_ID], run
docker ps
Note docker-compose ps will give you containers names, but you need container id to ssh the container.
I am running docker (via docker-compose) and can't run varnishadm from within the container. The error produced is:
Cannot open /var/lib/varnish/4f0dab1efca3/_.vsm: No such file or directory
Could not open shared memory
I have tried searching on the 'shared memory' issue and _.vsm with no luck. It seems that the _.vsm is not created at all and /var/lib/varnish/ inside the container is empty.
I have tried a variety of -T settings without any luck.
Why run varnishadm?
The root of why I need to run varnishadm is to reload varnish while saving the cache. My backup backup backup option is to set up varnish as a service. We are on an old version of Varnish for the time being.
How am I starting docker?
CMD varnishd -F -f /etc/varnish/varnish.vcl \
-s malloc,1G \
-a :80
Full Dockerfile
FROM ubuntu:12.04
RUN apt-get update \
&& apt-get upgrade -y \
&& apt-get install wget dtrx varnish -y \
&& apt-get install pkg-config autoconf autoconf-archive automake libtool python-docutils libpcre3 libpcre3-dev xsltproc make -y \ && rm -rf /var/lib/apt/lists/*
RUN export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig/
RUN wget https://github.com/varnishcache/varnish-cache/archive/varnish-
3.0.2.tar.gz --no-check-certificate \
&& dtrx -n varnish-3.0.2.tar.gz
WORKDIR /varnish-3.0.2/varnish-cache-varnish-3.0.2/
RUN cd /varnish-3.0.2/varnish-cache-varnish-3.0.2/ && ./autogen.sh &&
cd /varnish-3.0.2/varnish-cache-varnish-3.0.2/ && ./configure && make install
RUN cd / && wget --no-check-certificate https://github.com/Dridi/libvmod-querystring/archive/v0.3.tar.gz && dtrx -n ./v0.3.tar.gz
WORKDIR /v0.3/libvmod-querystring-0.3
RUN ./autogen.sh && ./configure VARNISHSRC=/varnish-3.0.2/varnish-cache-varnish-3.0.2/ && make install
RUN cp /usr/local/lib/varnish/vmods/* /usr/lib/varnish/vmods/
WORKDIR /etc/varnish/
CMD varnishd -F -f /etc/varnish/varnish.vcl \
-s malloc,1G \
-a :80
EXPOSE 80
Full docker-compose
version: "3"
services:
varnish:
build: ./
ports:
- "8000:80"
volumes:
- ./default.vcl:/etc/varnish/varnish.vcl
- ./devicedetect.vcl:/etc/varnish/devicedetect.vcl
restart: unless-stopped