i am a newby in docker and I need an installation based on ubuntu of logstash.
I try the official logstash image and without success I can't run it so I decided to build my own installation based on my needs.
It works well but takes a lot of time to build it.
I wonder how can I improve (speed up) my building
This is my Dockerfile
FROM ubuntu:18.04
# RUN adduser --disabled-password --gecos "" ubuntu
# RUN usermod -aG sudo ubuntu
# RUN echo "ubuntu ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
RUN apt-get update && \
apt-get autoclean && \
apt-get autoremove
RUN echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | tee -a /etc/apt/sources.list.d/elastic-7.x.list
RUN apt-get install curl ca-certificates apt-utils wget apt-transport-https default-jre gnupg apt-transport-https software-properties-common -y
# RUN update-alternatives --config java
ENV LANG C.UTF-8
ENV JAVA_HOME /usr/lib/jvm/java-11-openjdk-amd64
ENV PATH $JAVA_HOME/bin:$PATH
RUN wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -
RUN apt update
RUN apt install ruby-full -y
RUN ruby --version
RUN apt install jruby -y
#install jruby (last version)
RUN jruby --version
RUN apt install logstash
#install plugins and bundles.
RUN cd /usr/share/logstash && gem install bundler
COPY logstash-pcap-test.conf /usr/share/logstash/
RUN mkdir /home/logstash/ && mkdir /home/logstash/traffic
COPY /traffic-example/* /home/logstash/traffic/
WORKDIR /usr/share/logstash
CMD ["/bin/bash","-c","bin/logstash -f logstash-pcap-test.conf --config.reload.automatic"]
And this is my docker-compose
version: "3"
services:
logstash_test:
build:
context: .
dockerfile: container/Dockerfile
image: logstash_test:latest
container_name: logstash_test
hostname: logstash_test
ports:
- 9600:9600
- 8089:8089
networks:
- elknetwork
networks:
elknetwork:
driver: bridge
Any thoughts?
Related
I'm new to Docker and trying to create a Dockerfile for this new Rails 7 app. I'm using vips instead of imagemagick for the memory benefits.
and my local machine is a mac so brew install vips takes care of my non docker development flow, but it hasn't gone so well using the ruby-vips gem, or installing from source.
Running $ docker compose up results in:
/usr/local/bundle/gems/ffi-1.15.5/lib/ffi/library.rb:145:in block in ffi_lib': Could not open library 'vips.so.42': vips.so.42: cannot open shared object file: No such file or directory. (LoadError)
With the following docker-compose.yml:
version: "3.9"
services:
db:
image: postgres
volumes:
- ./tmp/db:/var/lib/postgresql/data
environment:
POSTGRES_PASSWORD: password
web:
build: .
command: bash -c "rm -f tmp/pids/server.pid && bundle exec rails s -p 3000 -b '0.0.0.0'"
volumes:
- .:/myapp
ports:
- "3000:3000"
depends_on:
- db
and a Dockerfile:
FROM ruby:3.0.1
RUN apt-get update -qq && apt-get install -y nodejs postgresql-client
WORKDIR /myapp
COPY Gemfile /myapp/Gemfile
COPY Gemfile.lock /myapp/Gemfile.lock
RUN gem install ruby-vips
RUN bundle install
# Add a script to be executed every time the container starts.
COPY entrypoint.sh /usr/bin/
RUN chmod +x /usr/bin/entrypoint.sh
ENTRYPOINT ["entrypoint.sh"]
EXPOSE 3000
# Configure the main process to run when running the image
CMD ["rails", "server", "-b", "0.0.0.0"]
I've also tried installing from source (https://www.libvips.org/install.html) install of using ruby-vips with no luck.
TLDR; ruby-vips need libvips42 installed on your docker image.
Update your Dockerfile to use the following:
RUN apt-get update -qq && apt-get install -y --no-install-recommends nodejs postgresql-client libvips42
PS: run docker compose down and docker compose up --build to force a rebuild of your docker images.
I don't think you're actually installing libvips in your dockerfile. Try this:
FROM ruby:3.0.1
RUN apt-get update -qq \
&& apt-get install -y nodejs postgresql-client
RUN apt install -y --no-install-recommends libvips42
WORKDIR /myapp
...
However, this will install the libvips that comes with buster, and it's 8.7.x from five years ago (!!). Debian does not move quickly.
I would build current libvips from source. Something like this:
# based on buster
FROM ruby:3.0.1
RUN apt-get update && apt-get install -y \
build-essential \
unzip \
wget \
git \
pkg-config
# stuff we need to build our own libvips ... this is a pretty random selection
# of dependencies, you'll want to adjust these
RUN apt-get install -y \
glib-2.0-dev \
libexpat-dev \
librsvg2-dev \
libpng-dev \
libgif-dev \
libjpeg-dev \
libexif-dev \
liblcms2-dev \
liborc-dev
ARG VIPS_VERSION=8.12.2
ARG VIPS_URL=https://github.com/libvips/libvips/releases/download
RUN apt-get install -y \
wget
RUN cd /usr/local/src \
&& wget ${VIPS_URL}/v${VIPS_VERSION}/vips-${VIPS_VERSION}.tar.gz \
&& tar xzf vips-${VIPS_VERSION}.tar.gz \
&& cd vips-${VIPS_VERSION} \
&& ./configure --disable-deprecated \
&& make -j 4 V=0 \
&& make install
RUN gem install ruby-vips
That won't include support for GIF save, or for formats like HEIC or PDF. You'll probably want to adjust it a bit. And of course you should not build packages in your deployment docker image, you'd want to do that in a separate dockerfile.
Hopefully ruby-vips deployment will become more automated over the next few months now that's it's the rail7 default. It's rather manual right now.
I find a Big Blue Button Dockerfile on [ bigbluebutton /docker ][1] [1]: https://github.com/bigbluebutton/docker
And second week I tryed to write docker-compose.yml file.
How to write docker-compose.yml file. I tried but not success.
This is Dockerfile [ bigbluebutton /docker ][1] [1]: https://github.com/bigbluebutton/docker
FROM ubuntu:16.04
MAINTAINER ffdixon#bigbluebutton.org
ENV DEBIAN_FRONTEND noninteractive
# RUN echo 'Acquire::http::Proxy "http://192.168.0.130:3142";' > /etc/apt/apt.conf.d/01proxy
RUN apt-get update && apt-get install -y wget
RUN echo "deb http://ubuntu.bigbluebutton.org/xenial-200 bigbluebutton-xenial main " | tee /etc/apt/sources.list.d/bigbluebutton.list
RUN wget http://ubuntu.bigbluebutton.org/repo/bigbluebutton.asc -O- | apt-key add -
RUN apt-get update && apt-get -y dist-upgrade
# -- Setup tomcat7 to run under docker
RUN apt-get install -y \
haveged \
net-tools \
supervisor \
sudo \
tomcat7
RUN sed -i 's|securerandom.source=file:/dev/random|securerandom.source=file:/dev/urandom|g' /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/security/java.security
ADD mod/tomcat7 /etc/init.d/tomcat7
RUN chmod +x /etc/init.d/tomcat7
RUN apt-get install -y language-pack-en
RUN update-locale LANG=en_US.UTF-8
# -- Install BigBlueButton
RUN echo ttf-mscorefonts-installer msttcorefonts/accepted-mscorefonts-eula select true | debconf-set-selections
RUN apt-get install -y bigbluebutton
RUN apt-get install -y bbb-demo
# -- Install mongodb (for HTML5 client)
RUN sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 0C49F3730359A14518585931BC711F9BA15703C6
RUN echo "deb [ arch=amd64,arm64 ] http://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.4 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.4.list
RUN sudo apt-get update && sudo apt-get install -y mongodb-org curl
# -- Install nodejs (for HTML5 client)
RUN apt-get install -y apt-transport-https
RUN curl -s https://deb.nodesource.com/gpgkey/nodesource.gpg.key | apt-key add -
RUN echo 'deb http://deb.nodesource.com/node_8.x xenial main' > /etc/apt/sources.list.d/nodesource.list
RUN echo 'deb-src http://deb.nodesource.com/node_8.x xenial main' >> /etc/apt/sources.list.d/nodesource.list
RUN apt-get update && apt-get install -y nodejs
# -- Install HTML5 client
RUN apt-get install -y bbb-html5
# -- Install supervisor to run all the BigBlueButton processes (replaces systemd)
RUN apt-get install -y supervisor
RUN mkdir -p /var/log/supervisor
ADD supervisord.conf /etc/supervisor/conf.d/supervisord.conf
# -- Modify FreeSWITCH event_socket.conf.xml to listen to IPV4
ADD mod/event_socket.conf.xml /opt/freeswitch/etc/freeswitch/autoload_configs
# -- Finish startup
ADD setup.sh /root/setup.sh
ENTRYPOINT ["/root/setup.sh"]
CMD []
So this is my docker-compose file. But that file not worked. I lack knowledge and qualification.
version: '3'
services:
bigbluebutton:
build: .
image: bigbluebutton/bigbluebutton
ports:
- "80:80"
expose:
- "1935/tcp"
- "5066/tcp"
- "2202"
Thanks for your answers.
just you need lines below in Docckerfiles becuse bbb need configure and this code passing this step;
`-RUN apt-get install -y bigbluebutton(remove)
+RUN apt-get install -y bigbluebutton || :
+RUN gem install bundler -v 1.16.1
+RUN apt-get install -y bigbluebutton
RUN apt-get install -y bbb-demo`
this is not bug just compile error.
I think you did not indent correctly expose subsection. I would write:
version: '3'
services:
bigbluebutton:
build: .
image: bigbluebutton/bigbluebutton
ports:
- "80:80"
expose:
- "1935/tcp"
- "5066/tcp"
- "2202"
If you use bigbluebutton/bigbluebutton image from DockerHub, you don't need neither build section nor Dockerfile itself.
I want to install rclone on a docker image in heroku to able to use Rclone with python telegram bot
I made a heroku.yml file
build:
docker:
worker: Dockerfile
run:
worker: bash start.sh
And start.sh as
python3 -m bot
And Dockerfile as
FROM ubuntu:18.04
WORKDIR /usr/src/app
RUN docker pull rclone/rclone:latest
RUN docker run rclone/rclone:latest version
RUN chmod 777 /usr/src/app
RUN apt -qq update
RUN apt -qq install -y python3 python3-pip locales
COPY requirements.txt .
RUN pip3 install --no-cache-dir -r requirements.txt
COPY . .
RUN locale-gen en_US.UTF-8
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8
CMD ["bash","start.sh"]
I get error The command '/bin/sh -c docker pull rclone/rclone:latest' returned a non-zero code: 127 in the git bash CLI
What Am I doing wrong? Or what is the procedure?
Thanks in advance!
FROM ubuntu:16.04
WORKDIR /app
# line number 12 - 15 in your Dockerfile
RUN echo "LC_ALL=en_US.UTF-8" >> /etc/environment
RUN echo "LANG=en_US.UTF-8" >> /etc/environment
RUN more "/etc/environment"
RUN apt-get update
#RUN apt-get upgrade -y
#RUN apt-get dist-upgrade -y
RUN apt-get install curl htop git zip nano ncdu build-essential chrpath libssl-dev libxft-dev pkg-config glib2.0-dev libexpat1-dev gobject-introspection python-gi-dev apt-transport-https libgirepository1.0-dev libtiff5-dev libjpeg-turbo8-dev libgsf-1-dev fail2ban nginx -y
# Install Rclone
RUN curl -sL https://rclone.org/install.sh | bash
RUN rclone version
# Cleanup
RUN apt-get update && apt-get upgrade -y && apt-get autoremove -y
Based on this answer
You can try this, also don't try to use docker commands inside a Dockerfile.
When running laravel-dusk on in a docker container on gitlab I get an error saying it cannot connect to port 9515 on localhost.
Failed to connect to localhost port 9515: Connection refused
My docker file looks as following:
# Set the base image for subsequent instructions
FROM php:7.1
# Replace shell with bash so we can source files
RUN rm /bin/sh && ln -s /bin/bash /bin/sh
# Install packages
RUN apt-get update -yqq && apt-get install -y git wget curl libcurl4-gnutls-dev libicu-dev libmcrypt-dev libvpx-dev libjpeg-dev libpng-dev libxpm-dev zlib1g-dev libfreetype6-dev libxml2-dev libexpat1-dev libbz2-dev libgmp3-dev libldap2-dev unixodbc-dev libpq-dev libsqlite3-dev libaspell-dev libsnmp-dev libpcre3-dev libtidy-dev -yqq bzip2 libfontconfig xvfb chromium libmagickwand-dev
# Add chrome repo and install google-chrome-stable
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
RUN sh -c 'echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list'
RUN apt-get update
RUN apt-get install -y google-chrome-stable
# Install php environment
RUN docker-php-ext-install mbstring mcrypt pdo_mysql curl json intl gd xml zip bz2 opcache bcmath
# Install imagick
RUN pecl install imagick && docker-php-ext-enable imagick
# check installed modules
RUN php -m
# Install NVM
ENV NVM_DIR /usr/local/nvm
ENV NODE_VERSION 8.1.4
RUN curl --silent -o- https://raw.githubusercontent.com/creationix/nvm/v0.33.2/install.sh | bash
# install node and npm
RUN source $NVM_DIR/nvm.sh \
&& nvm install $NODE_VERSION \
&& nvm alias default $NODE_VERSION \
&& nvm use default
# add node and npm to path so the commands are available
ENV NODE_PATH $NVM_DIR/v$NODE_VERSION/lib/node_modules
ENV PATH $NVM_DIR/versions/node/v$NODE_VERSION/bin:$PATH
# Install Composer Package manager
ENV COMPOSER_ALLOW_SUPERUSER 1
RUN php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');" && php composer-setup.php \
php -r "unlink('composer-setup.php');" && mv composer.phar /usr/local/bin/composer
# Update packages
RUN apt-get update
# Xdebug
RUN pecl install -o -f xdebug \
&& rm -rf /tmp/* \ && docker-php-ext-enable xdebug
# Clear out the local repository of retrieved package files
RUN apt-get clean
# Install Laravel Envoy
RUN composer global require "laravel/envoy=~1.0"
# confirm installed versions
RUN node -v
RUN npm -v
RUN php -v
RUN google-chrome-stable --version
RUN composer --version
RUN php -m
EXPOSE 9515
Then my gitlab-ci file looks like the following:
image: registry.gitlab.com/path-to-repo/repo-name:latest
# Select what we should cache
cache:
paths:
- vendor/
services:
- mysql:5.7
variables:
# Configure mysql service (https://hub.docker.com/_/mysql/)
MYSQL_DATABASE: database-name
MYSQL_ROOT_PASSWORD: database-password
DB_HOST: mysql
DB_USERNAME: root
stages:
- test
unit_test:
stage: test
script:
- cp .env.runnable .env
- composer install
- php artisan config:clear
- php artisan key:generate
- php artisan migrate --seed
- php artisan db:seed --class=TestSeeder
- vendor/bin/phpunit --coverage-text --colors=never
- php artisan env
- php artisan dusk
At first I thought it was because I was not exposing port 9515, but I have exposed this in the Dockerfile. However, now I have no idea what the problem could be. Is there an issues that this port is used for another process? Could it be that the Chromedriver is running on a different port?
In your gitlab-ci file try to replace php artisan dusk with:
php artisan serve & php artisan dusk
I was getting the 9515 connection refused error and then I found this GitHub issue and added these dependencies to my dockerfile (granted Im using custom ubuntu image). I didn't need to add anything else that the GitHub comment mentions (no script etc).
FROM ubuntu:18.04
# ... my other previous dependencies (PHP, etc.)
RUN apt-get update && \
apt-get -y install libxpm4 libxrender1 libgtk2.0-0 libnss3 libgconf-2-4 && \
apt-get -y install chromium-browser && \
apt-get -y install xvfb gtk2-engines-pixbuf && \
apt-get -y install xfonts-cyrillic xfonts-100dpi xfonts-75dpi xfonts-base xfonts-scalable && \
apt-get -y install imagemagick x11-apps && \
apt-get upgrade -y && \
apt-get clean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* /usr/share/doc/*
Also, for local development I needed to update my .env (or use .env.dusk.local) APP_URL: to point to my docker host.
APP_URL=host.docker.internal:8000
For GitLab, you can try and update your .env APP_URL with the docker host by adding an additional setup in a before_script (after copying .env.example to .env). (host.docker.internal is not available in docker for linux). Also, Note your container will need net-tools for the netstat command.
before_script:
# ...
- docker_host=$(netstat -nr | grep '^0\.0\.0\.0' | awk '{print $2}') && sed -i "s|APP_URL=http://0.0.0.0|APP_URL=http://${docker_host}|" .env
Where APP_URL=http://0.0.0.0 is the value of what is already in your existing .env file.
docker-compose.yml:
version: '3'
services:
ezmove:
volumes:
- /host-dir:/home/container-dir
build:
context: .
args:
BRANCH: develop
Dockerfile:
FROM appcontainers/ubuntu:xenial
MAINTAINER user <user>
RUN apt-get update -y --no-install-recommends \
&& apt-get install -y --no-install-recommends python3.5-minimal python3.5-venv \
&& apt-get install -y --no-install-recommends git \
&& apt-get install -y --no-install-recommends python-pip \
&& pip install --upgrade pip \
&& rm -rf /var/lib/apt/lists/* \
&& mkdir -p /home/container-dir
WORKDIR /home/container-dir
RUN /bin/bash - c "sh ./script.sh"
At the time of build the docker container How to map local directory to container
When $ docker-compose up, it will starts to build container but after installation of the packag dependancies it will try to execute the script.sh file but got error "FILE NOT FOUND! "
Tried:
Not want todo git clone inside docker continer
Not want to store source code inside the continer
So, how to map the host OS file to the container at build time
you lack some COPY or ADD in your Dockerfile in order to copy your script.sh in your image.
Check the docs
https://docs.docker.com/engine/reference/builder/#add
https://docs.docker.com/engine/reference/builder/#copy
By the way, Docker is about isolation, so a running container should be isolated from the host, and certainly not access the host OS.