I am refactoring an angular application to switch from a VM architecture to Docker containers.
While building the container for Angular I came up with a Dockerfile to use as a builder in the multistage build. I worked like a charm on my Mac so i pushed it to our company github to be consumed by my colleagues.
The problem rise up when a colleague pulled from the repo and tried to build on his Mac (different models but more or less comparable, mine is a 2015, his is a 2016, both running Mojave).
This is the content of the dockerfile that is erroring:
# base image
FROM node:9.6.1 as builder
# install chrome for protractor tests
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
RUN sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list'
RUN apt-get update
RUN apt-get install -y software-properties-common
RUN add-apt-repository -y ppa:fontforge/fontforge
RUN apt-get install -y google-chrome-stable
RUN apt-get install -y fontforge ttfautohint gettext
I was assuming it to build fine as it was on my laptop but when he tries to launch the build he receives some errors in the step RUN apt-get update
The container gets a 404 while trying to update systemd and exits.
We tried also to spin up a new container FROM THE SAME image docker run --rm -it node:9.6.1 /bin/bash and copy-pasting all the instructions manually in the command line and it worked fine.
1. Why two different outcomes in two very similar but still different machines?
The entire point of docker containers should be abstracting the environment and create standalone environments for your applications, so why the same Dockerfile blueprints work perfectly on my machine and do not on his?
Also the docker demon runs fine and it starts the container, the issue we have is during the system upgrade inside the container build.
2. Why on the same machine, the Dockerfile build fails while the same steps succeed if run manually?
This completely blew up my mind, i may even understand the two different machines issue, but i cannot find any logical explanation to this one: same commands fed to command line work while they don't if executed via script.
apt-get update, in particular, generates results that vary over time. These include URLs to Debian packages, and standard Debian package management practice is to remove a package from the repository listings as soon as there’s a newer version.
If you previously ran the Dockerfile up through the RUN apt-get update, and then later changed what specific packages get installed, you could wind up in a state where Docker cached the results of the update operation, but they’re no longer valid.
The usual answer to this is to make sure to run apt-get update && apt-get install in a single RUN step:
RUN apt-get update \
&& apt-get install -y software-properties-common
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
RUN echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list
RUN add-apt-repository -y ppa:fontforge/fontforge
RUN apt-get update \
&& apt-get install -y google-chrome-stable fontforge ttfautohint gettext
Related
Problem: To install multiple images to create a application server, but using multi-stage build and size small and keeping only one final image
Question1: Is that possible?
Question2: Does a container always expected to have a base OS, only 1 application dependency and then running compiled code based on 1 application dependency?
I have not found example online where we can create a complete application server (Web Server+Database Server+Application Server) using Docker multistage build.
I would like to have the following in an application server:
Install Alpine
Install nodejs
Install nginx
Install jenkins
Install mongodb
Install Python
This will be the base Image that can be replicated.
Don't want to use : Run apk or Run apt get to install the applications as the image size grows big
Want to use Multi-stage build to have final one image and small size of the image.
However, i want to keep the image size small using MultiStage Build.
FROM alpine:3.17.2 as base1
FROM python:alpine3.17 as base2
FROM nodejs:lts-alpine3.17 as base3
FROM nginx:stable-alpine as base4
FROM jenkins:2.375.3-lts-alpine as base5
FROM mongo:jammy as base6
COPY --from=base1 / /
COPY --from=base2 / /
COPY --from=base3 / /
COPY --from=base4 / /
COPY --from=base5 / /
COPY --from=base6 / /
[this will overwrite some directories]
Expectation
When i run the final image "base7", i can run any nodejs, mongo, python commands. Ingest data file into mongodb, then python analysis and using nodejs to display it.
Previously working without multi-stage build (Issue is size is big and many image layers created)
FROM ubuntu
RUN apt update
RUN apt upgrade
RUN apt-get -y install git
RUN apt-get -y install nodejs
RUN apt-get -y install wget
RUN apt-get -y install gnupg
RUN wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | apt-key add -
RUN sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
RUN apt update
RUN apt-get -y install jenkins
RUN apt-get -y install gnupg
RUN wget -qO - https://www.mongodb.org/static/pgp/server-6.0.asc | apt-key add -
RUN echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/6.0 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-6.0.list
RUN wget http://archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.16_amd64.deb
RUN apt update
RUN dpkg -i libssl1.1_1.1.1f-1ubuntu2.16_amd64.deb
RUN apt-get -y install libssl-dev
RUN apt-get -y install libssl1.1
RUN apt-get install -y mongodb-org
COPY . /learn
WORKDIR /learn
Created a Docker file in oreder to install Tomcat server from Unix as bashe os
My Dockerfile:
FROM ubuntu
RUN apt-get update && apt-get upgrade -y #to update os
RUN apt-get dist-upgrade
RUN apt-get install build-essential
RUN apt-get install openjdk-8-jdk # to install java 8
RUN apt-get wget -y #to install wget package
RUN apt-get wget https://mirrors.estointernet.in/apache/tomcat/tomcat-9/v9.0.37/bin/apache-tomcat-9.0.37.tar.gz #to download tomcat
RUN tar -xvzf apache-tomcat-9.0.37 # unzipping the tomcat
RUN mkdir tomcat # craeting tomacat directory
RUN cp apache-tomcat-9.0.37/* tomcat # copying tomact files to tomact directory
Command to create Docker Image from Docker file:
docker build -t [img name] -f [file name] .
On execution, while installing java package am getting like this:
'''After this operation, 242 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y'''
You are getting the prompt because the command is awaiting user input for whether or not to install a package. The -y flag you're using for a few of them (like wget) allows bash to assume a yes. Add this flag to all your installation commands.
By the way, there's quite a few potential issues with the Dockerfile you posted.
For example, you have RUN apt-get wget ...
Are you sure that is what you want to do, and not just RUN wget ...? Unless wget is a command that apt-get takes, which it isn't, it will cause unexpected behavior.
You also seem to be missing the command to start the Tomcat server, which can make it so that nothing happens when you attempt to run the image.
I think you should add DEBIAN_FRONTEND=noninteractive when running the apt-get commands, something like this:
RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get install build-essential -y
Also, it's considered bad practice to use multiple RUN steps which could be consolidated into one. More about Dockerfile best practices can be found here.
Have a Dockerfile that installs from a python image and then I need it to install a specific (not latest) version of Google Chrome.
Here's what I have:
FROM python:3.6
# Tools
RUN apt-get update \
&& apt-get install -y vim less \
&& apt-get clean
# https://github.com/SeleniumHQ/docker-selenium/blob/master/NodeChrome/Dockerfile.txt
#============================================
# Google Chrome
#============================================
# can specify versions by CHROME_VERSION;
# e.g. google-chrome-stable=53.0.2785.101-1
# google-chrome-beta=53.0.2785.92-1
# google-chrome-unstable=54.0.2840.14-1
# latest (equivalent to google-chrome-stable)
# google-chrome-beta (pull latest beta)
#============================================
ARG CHROME_VERSION="google-chrome-stable"
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list \
&& apt-get update -qqy \
&& apt-get -qqy install \
${CHROME_VERSION:-google-chrome-stable} \
&& rm /etc/apt/sources.list.d/google-chrome.list \
&& rm -rf /var/lib/apt/lists/* /var/cache/apt/*
The Chrome installation steps were taken from here (as seen in the comments) and even using the version in the example I get the error
E: Version '53.0.2785.101-1' for 'google-chrome-stable' was not found
Tried other versions from https://chromereleases.googleblog.com/ and nothing works.
Do you know of a different way to install a specific version or if I'm doing something wrong with these steps?
This took me a while to find, but you're installing from Google's repository and they only keep the latest versions of Google Chrome in their repositories. You could probably search for 3rd party repositories that have older versions of Chrome, but I personally wouldn't recommend that.
The current version is 75.0.3770.100-1 for google-chrome-stable at the time of this post. Will that not work for you?
Lastly, I directly copied your dockerfile and it worked for me with the latest build of google-chrome-stable installed on the image. How were you running docker?
Here was my process:
Copied your docker file directly into ./Dockerfile
docker build ./
docker image ls
Copied my image id (90206843f24e in my case)
docker run --entrypoint "/bin/bash" -it 90206843f24e
You'll be dropped in a root shell on the docker image to "poke" around
run google-chrome -version to verify the above version is installed
I hope this works for you. Good luck and keep us posted!
I'm trying to create a dockerfile that will build an image with .net core 2.0 and Jenkins. I'm kind of new to Docker but want to include .net core 2.0 in my container with Jenkins so I don't have to worry about .net core being installed on the target machine and can build .net core apps with Jenkins in my container. Am I missing something here?
it builds fine up until it runs the apt-get update command and I get the following error:
E: Malformed entry 1 in list file /etc/apt/sources.list.d/dotnetdev.list (component)
E: The list of sources could not be read.
I'm using the steps to install on ubuntu at this link:
https://learn.microsoft.com/en-us/dotnet/core/linux-prerequisites?tabs=netcore2x
My Dockerfile looks like this:
FROM jenkins
# Install .NET Core SDK
USER root
RUN mkdir -p /jenkins
WORKDIR /jenkins
ENV DOTNET_CORE_SDK_VERSION 2.0
RUN curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor >/jenkins/microsoft.gpg
RUN mv microsoft.gpg /etc/apt/trusted.gpg.d/microsoft.gpg
RUN sh -c 'echo "deb [arch=amd64] https://packages.microsoft.com/repos/microsoft-ubuntu-xenial-prod xenial main" > /etc/apt/sources.list.d/dotnetdev.list'
RUN apt-get update
RUN apt-get install dotnet-sdk-2.0.0
As of this response you can use the following Dockerfile to get .NetCore 2 installed into the Jenkins container. You can obviously take this further and install the needed plugins and additional software as needed. I hope this helps you out!
FROM jenkins/jenkins:lts
# Switch to root to install .NET Core SDK
USER root
# Just for my sanity... Show me this distro information!
RUN uname -a && cat /etc/*release
# Based on instructiions at https://learn.microsoft.com/en-us/dotnet/core/linux-prerequisites?tabs=netcore2x
# Install depency for dotnet core 2.
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
curl libunwind8 gettext apt-transport-https && \
curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.gpg && \
mv microsoft.gpg /etc/apt/trusted.gpg.d/microsoft.gpg && \
sh -c 'echo "deb [arch=amd64] https://packages.microsoft.com/repos/microsoft-debian-stretch-prod stretch main" > /etc/apt/sources.list.d/dotnetdev.list' && \
apt-get update
# Install the .Net Core framework, set the path, and show the version of core installed.
RUN apt-get install -y dotnet-sdk-2.0.0 && \
export PATH=$PATH:$HOME/dotnet && \
dotnet --version
# Good idea to switch back to the jenkins user.
USER jenkins
You can run these commands inside the Docker container in order to install .NET Core. They can also be stored in a Dockerfile (as per #Zooly57)
Install the latest .NET Core 2.0:
sudo apt install libunwind8 gettext apt-transport-https
curl -sSL https://dot.net/v1/dotnet-install.sh | bash /dev/stdin --channel 2.0
Or LTS version of .NET Core
sudo apt install libunwind8 gettext apt-transport-https
curl -sSL https://dot.net/v1/dotnet-install.sh | bash /dev/stdin --channel LTS
Contents of the script here:
https://github.com/dotnet/cli/blob/master/scripts/obtain/dotnet-install.sh
Fantastic answer by Dennis, it's exactly what I ended up doing. It was a nice introduction to Docker as well :-)
Here's my Dockerfile for Jenkins 2.249.2 (LTS at the time of writing) on Debian 9 (stretch):
# Extend Jenkins 2.249.2 on Debian 9 (stretch)
FROM jenkins/jenkins:2.249.2-lts
# Switch to root user to install .NET SDK
USER root
# Print kernel and distro info
RUN echo "Distro info:" && uname -a && cat /etc/*release
# Install needed tools and upgrade installed packages
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
curl apt-transport-https software-properties-common \
&& apt-get upgrade -y
# Add Microsoft repository for .NET SDK
RUN curl -sSL https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
RUN apt-add-repository https://packages.microsoft.com/debian/9/prod/
# Install .NET SDK
RUN apt-get update \
&& apt-get install -y dotnet-sdk-3.1
# Switch back to jenkins user
USER jenkins
The dotnet command worked without setting any paths.
I guess when a newer version of Jenkins is released that uses Debian 10, I'll just update the FROM line then the Microsoft repository URL.
I believe you should follow below approach instead:
Develop your asp.net core app and check in to Git(Any source control)
Have a build server which has Jenkins, .Net Core, Docker installed
Configure Jenkins to communicate with Git (webhook/polling - to see if there is a check in)
And configure a Jenkins job which will do the following
Pull the latest from Git,
Restore,
Build,
Publish the asp.net core application,
Create a docker image which has a capability to run the asp.net core app in it
Upload the docker image just created to your Docker Hub
You may not want to do it exactly as mentioned above especially the source control part. But this approach works well.
I have followed this link while I made the above setup.
Hope it helps. Thanks!
For anyone who is struggling with this topic recently, this is what I added to the bottom of my Dockerfile to install the .NET SDK;
USER root
# Install dependencies
RUN apt-get install wget
RUN wget https://packages.microsoft.com/config/ubuntu/18.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb
RUN dpkg -i packages-microsoft-prod.deb
RUN rm packages-microsoft-prod.deb
# Install .NET SDK 6.0
RUN apt-get update;
RUN apt-get install -y apt-transport-https
RUN apt-get update
RUN apt-get install -y dotnet-sdk-6.0
RUN dotnet --version
This is based on installing the SDK on Ubuntu 18.04 as this is the version that AKS uses which was perfect for my scenario
https://learn.microsoft.com/en-us/dotnet/core/install/linux-ubuntu#dependencies
I'm trying to create a Docker image for ripping CDs (using abcde).
Here's the relevant portion of the Dockerfile:
FROM ubuntu:17.10
MAINTAINER Graham Nicholls <graham#rockcons.co.uk>
RUN apt update && apt -y install eject vim ruby abcde
...
Unfortunately, the package "abcde" pulls in a mail client (not sure which), and apt tries to configure that by asking what type of mail connection to configure (smarthost/relay etc).
When docker runs, it's not appearing to read from stdin, so I can't redirect into the docker process.
I've tried using --nodeps with apt (and replacing apt with apt-get); unfortunately --nodeps seems no-longer to be a supported option and returns:
E: Command line option --nodeps is not understood in combination with the other options
Someone has suggested using expect in response to a similar question, which I'd rather avoid. This seems to be a "difficult to google" problem - I can't find anything.
So, is there a way of passing in the answer to the config in apt, or of preventing apt from pulling in a mail client, which would be better - I'm not planning in sending updates to cddb.
The typical template to install apt packages in a docker container looks like:
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
eject \
vim \
ruby \
abcde \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
Running it with the "noninteractive" value removes any prompts. You don't want to set that as an ENV since that would also impact any interactive commands you run inside the container.
You also want to cleanup the package database when finished to reduce the layer size and avoid reusing a stale cached package database in a later step.
The no-install-recommends option will reduce the number of packages installed by only installing the required dependencies, not the additional recommended packages. This cuts the size of the root filesystem down by half for me.
If you need to pass a non-default configuration to a package, then use debconf. First run you install somewhere interactively and enter the options you want to save. Install debconf-utils. Then run:
debconf-get-selections | grep "${package_name}"
to view all the options you configured for that package. You can then pipe these options to debconf-set-selections in your container before running your install, e.g.:
RUN echo "postfix postfix/main_mailer_type select No configuration" \
| debconf-set-selections \
&& apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
....
or save your selections to a file that you copy in:
COPY debconf-selections /
RUN debconf-set-selections </debconf-selections \
&& apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
....