Create a complete Application Server Container using Multi Stage Build Docker - docker-multi-stage-build

Problem: To install multiple images to create a application server, but using multi-stage build and size small and keeping only one final image
Question1: Is that possible?
Question2: Does a container always expected to have a base OS, only 1 application dependency and then running compiled code based on 1 application dependency?
I have not found example online where we can create a complete application server (Web Server+Database Server+Application Server) using Docker multistage build.
I would like to have the following in an application server:
Install Alpine
Install nodejs
Install nginx
Install jenkins
Install mongodb
Install Python
This will be the base Image that can be replicated.
Don't want to use : Run apk or Run apt get to install the applications as the image size grows big
Want to use Multi-stage build to have final one image and small size of the image.
However, i want to keep the image size small using MultiStage Build.
FROM alpine:3.17.2 as base1
FROM python:alpine3.17 as base2
FROM nodejs:lts-alpine3.17 as base3
FROM nginx:stable-alpine as base4
FROM jenkins:2.375.3-lts-alpine as base5
FROM mongo:jammy as base6
COPY --from=base1 / /
COPY --from=base2 / /
COPY --from=base3 / /
COPY --from=base4 / /
COPY --from=base5 / /
COPY --from=base6 / /
[this will overwrite some directories]
Expectation
When i run the final image "base7", i can run any nodejs, mongo, python commands. Ingest data file into mongodb, then python analysis and using nodejs to display it.
Previously working without multi-stage build (Issue is size is big and many image layers created)
FROM ubuntu
RUN apt update
RUN apt upgrade
RUN apt-get -y install git
RUN apt-get -y install nodejs
RUN apt-get -y install wget
RUN apt-get -y install gnupg
RUN wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | apt-key add -
RUN sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
RUN apt update
RUN apt-get -y install jenkins
RUN apt-get -y install gnupg
RUN wget -qO - https://www.mongodb.org/static/pgp/server-6.0.asc | apt-key add -
RUN echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/6.0 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-6.0.list
RUN wget http://archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.16_amd64.deb
RUN apt update
RUN dpkg -i libssl1.1_1.1.1f-1ubuntu2.16_amd64.deb
RUN apt-get -y install libssl-dev
RUN apt-get -y install libssl1.1
RUN apt-get install -y mongodb-org
COPY . /learn
WORKDIR /learn

Related

issue in creating docker image from docker file

Created a Docker file in oreder to install Tomcat server from Unix as bashe os
My Dockerfile:
FROM ubuntu
RUN apt-get update && apt-get upgrade -y #to update os
RUN apt-get dist-upgrade
RUN apt-get install build-essential
RUN apt-get install openjdk-8-jdk # to install java 8
RUN apt-get wget -y #to install wget package
RUN apt-get wget https://mirrors.estointernet.in/apache/tomcat/tomcat-9/v9.0.37/bin/apache-tomcat-9.0.37.tar.gz #to download tomcat
RUN tar -xvzf apache-tomcat-9.0.37 # unzipping the tomcat
RUN mkdir tomcat # craeting tomacat directory
RUN cp apache-tomcat-9.0.37/* tomcat # copying tomact files to tomact directory
Command to create Docker Image from Docker file:
docker build -t [img name] -f [file name] .
On execution, while installing java package am getting like this:
'''After this operation, 242 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y'''
You are getting the prompt because the command is awaiting user input for whether or not to install a package. The -y flag you're using for a few of them (like wget) allows bash to assume a yes. Add this flag to all your installation commands.
By the way, there's quite a few potential issues with the Dockerfile you posted.
For example, you have RUN apt-get wget ...
Are you sure that is what you want to do, and not just RUN wget ...? Unless wget is a command that apt-get takes, which it isn't, it will cause unexpected behavior.
You also seem to be missing the command to start the Tomcat server, which can make it so that nothing happens when you attempt to run the image.
I think you should add DEBIAN_FRONTEND=noninteractive when running the apt-get commands, something like this:
RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get install build-essential -y
Also, it's considered bad practice to use multiple RUN steps which could be consolidated into one. More about Dockerfile best practices can be found here.

Dockerfile to create an image to be used as an container agent in jenkins

I am trying to use a node agent container in Jenkins to run npm instructions on it. So that, I am creating a Dockerfile to get a valid image with ssh and nodejs. The executor runs fine, but when I use npm it says that it doesn't know the command.
The same problem happens when (after building the dockerfile) I do docker exec -it af5451297d85 bash and after that, inside the container, I try to do npm --v (for example).
# This Dockerfile is used to build an image containing an node jenkins agent
FROM node:9.0
MAINTAINER Estefania Castro <estefania.castro#luceit.es>
# Upgrade and Install packages
RUN apt-get update && apt-get -y upgrade && apt-get install -y git openssh-server
# Install NGINX to test.
RUN apt-get install nginx -y
# Prepare container for ssh
RUN mkdir /var/run/sshd && adduser --quiet jenkins && echo "jenkins:jenkins" | chpasswd
RUN npm install
ENV CI=true
EXPOSE 22
CMD ["/usr/sbin/sshd", "-D"]
I would like to run npm instructions like npm install, npm publish, ... to manage my project in a jenkinsfile. Could anyone help?
Thanks
I have already solved the problem (after two weeks haha).
FROM jenkins/ssh-slave
# Install selected extensions and other stuff
RUN apt-get update && apt-get -y --no-install-recommends install && apt-get clean
RUN apt-get install -y curl
# Install nodejs
RUN curl -sL https://deb.nodesource.com/setup_8.x | bash -
RUN apt-get install -y nodejs && apt-get install -y nginx

Install RPM package in a pre-built node image

I am writing a Node app I want to containerize using a pre-built node image (https://hub.docker.com/_/node/). I need to deploy application that I only have a RPM package for and I cannot figure out where to start finding documentation or a small example to do this.
The examples I'm looking at use yum, which I don't have (from my understanding) in the pre-built node image.
COPY src/MyApp/lib/3rdPartyApp.x86_64.rpm ./3rdPartyApp.x86_64.rpm
RUN yum localinstall 3rdPartyApp.x86_64.rpm; yum clean all && \
rm ./3rdPartyApp.x86_64.rpm
My other option is to use a CentOS docker image which has yum. But I'm running in to problems getting Node installed there trying to use NVM. But I'm also reading I shouldn't try to use NVM when building a Docker container and there is a better way.
You can use alien to convert packages from one format to another.
FROM node
RUN apt-get update && apt-get install -y alien
COPY src/MyApp/lib/3rdPartyApp.x86_64.rpm ./3rdPartyApp.x86_64.rpm
RUN alien -d -i 3rdPartyApp.x86_64.rpm
This will leave a lot of extra files in your image. You can use two step build to clear it up.
FROM node AS builder
RUN apt-get update && apt-get install -y alien
COPY src/MyApp/lib/3rdPartyApp.x86_64.rpm ./3rdPartyApp.x86_64.rpm
RUN alien -d 3rdPartyApp.x86_64.rpm
FROM node
COPY --from=builder 3rdPartyApp.x86_64.deb .
RUN dpkg -i 3rdPartyApp.x86_64.deb && rm 3rdPartyApp.x86_64.deb
FROM centos:centos7.6.1810
# Enable EPEL to install Node.js and npm
RUN rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm && \
    yum -y update && \
    yum install -y npm git && \
    yum clean all

Building Go Application using confluent-kafka-go on Linux

I am trying to create a docker image with my go application. The application (which was developed on MacOS) depends on confluent-kafka-go which in turn depends on librdkafka-dev which I install in the Docker image like so:
FROM golang:1.1
RUN apt-get update
RUN apt-get -y install librdkafka-dev
VOLUME /workspace
WORKDIR /workspace/src/my/app/folder
ENTRYPOINT ["/bin/sh", "-c"]
I am getting the following error:
my/app/folder/vendor/github.com/confluentinc/confluent-kafka-go/kafka
../folder/vendor/github.com/confluentinc/confluent-kafka-go/kafka/00version.go:44:2: error: #error "confluent-kafka-go requires librdkafka v0.11.5 or later. Install the latest version of librdkafka from the Confluent repositories, see http://docs.confluent.io/current/installation.html"
As far as I understand the latest version is installed.
How can I fix it?
I had a similar issue a few weeks ago. IIRC confluent-kafka-go requires a recent version of librdkafka-dev, which simply was not yet released to alpine or others.
I was able to find it for ubuntu though, so my solution (which was more involved than I hoped for, but it worked), was to start from clean ubuntu, install librdkafka-dev, install Go version that I want and compile inside docker.
Here's how it looks:
FROM ubuntu
# Install the C lib for kafka
RUN apt-get update
RUN apt-get install -y --no-install-recommends apt-utils wget gnupg software-properties-common
RUN apt-get install -y apt-transport-https ca-certificates
RUN wget -qO - https://packages.confluent.io/deb/5.1/archive.key | apt-key add -
RUN add-apt-repository "deb [arch=amd64] https://packages.confluent.io/deb/5.1 stable main"
RUN apt-get update
RUN apt-get install -y librdkafka-dev
# Install Go
RUN add-apt-repository ppa:longsleep/golang-backports
RUN apt-get update
RUN apt-get install -y golang-1.11-go
# build the library
WORKDIR /go/src/gitlab.appsflyer.com/rantav/kafka-mirror-tester
COPY *.go ./
COPY // the rest of your go files. You may copy recursive if you want
COPY vendor vendor
RUN GOPATH=/go GOOS=linux /usr/lib/go-1.11/bin/go build -a -o main .
EXPOSE 8000
ENTRYPOINT ["./main"]
You can specify a version of package to be installed in apt-get command.
e.g
apt-get install librdkafka-dev=0.11.6~1confluent5.0.1-1
If that doesn't work then I think the apt sources doesn't have version 0.11.5 of librdkafka.
You can add a repository with the right version of librdkafka in /etc/apt/sources.list as described here:
https://docs.confluent.io/current/installation/installing_cp/deb-ubuntu.html#systemd-ubuntu-debian-install

Ruby docker image can't install nodejs on raspberry pi

I'm trying to get a rails server setup on a raspberry pi by building a Docker image.
Image:
FROM ruby:latest
RUN apt-get update -qq && apt-get install -y build-essential libpq-dev
RUN apt-get install -y nodejs
WORKDIR /webapp
COPY Gemfile* /webapp
RUN bundle install
COPY . /webapp/
CMD ["rails", "s", "-b", "0.0.0.0"]
But I'm getting
E: Package 'nodejs' has no installation candidate
EDIT:
I have tried adding the command from the nodejs site curl -sL https://deb.nodesource.com/setup_10.x | bash - && apt-get install -y nodejs
The reason the error Package 'nodejs' has no installation candidate was showing up regardless of how you tried to add the repo to apt-get is because Skipping acquire of configured file 'main/binary-armel/Packages' as repository 'https://deb.nodesource.com/node_10.x stretch InRelease' doesn't support architecture 'armel'.
My solution was to move to an arm32v7/ruby base image and running the same commands. This will install the nodejs_4.8.2~dfsg-1_armhf.deb package which is compatible with the arm architecture.

Resources