I am trying to create a docker image with my go application. The application (which was developed on MacOS) depends on confluent-kafka-go which in turn depends on librdkafka-dev which I install in the Docker image like so:
FROM golang:1.1
RUN apt-get update
RUN apt-get -y install librdkafka-dev
VOLUME /workspace
WORKDIR /workspace/src/my/app/folder
ENTRYPOINT ["/bin/sh", "-c"]
I am getting the following error:
my/app/folder/vendor/github.com/confluentinc/confluent-kafka-go/kafka
../folder/vendor/github.com/confluentinc/confluent-kafka-go/kafka/00version.go:44:2: error: #error "confluent-kafka-go requires librdkafka v0.11.5 or later. Install the latest version of librdkafka from the Confluent repositories, see http://docs.confluent.io/current/installation.html"
As far as I understand the latest version is installed.
How can I fix it?
I had a similar issue a few weeks ago. IIRC confluent-kafka-go requires a recent version of librdkafka-dev, which simply was not yet released to alpine or others.
I was able to find it for ubuntu though, so my solution (which was more involved than I hoped for, but it worked), was to start from clean ubuntu, install librdkafka-dev, install Go version that I want and compile inside docker.
Here's how it looks:
FROM ubuntu
# Install the C lib for kafka
RUN apt-get update
RUN apt-get install -y --no-install-recommends apt-utils wget gnupg software-properties-common
RUN apt-get install -y apt-transport-https ca-certificates
RUN wget -qO - https://packages.confluent.io/deb/5.1/archive.key | apt-key add -
RUN add-apt-repository "deb [arch=amd64] https://packages.confluent.io/deb/5.1 stable main"
RUN apt-get update
RUN apt-get install -y librdkafka-dev
# Install Go
RUN add-apt-repository ppa:longsleep/golang-backports
RUN apt-get update
RUN apt-get install -y golang-1.11-go
# build the library
WORKDIR /go/src/gitlab.appsflyer.com/rantav/kafka-mirror-tester
COPY *.go ./
COPY // the rest of your go files. You may copy recursive if you want
COPY vendor vendor
RUN GOPATH=/go GOOS=linux /usr/lib/go-1.11/bin/go build -a -o main .
EXPOSE 8000
ENTRYPOINT ["./main"]
You can specify a version of package to be installed in apt-get command.
e.g
apt-get install librdkafka-dev=0.11.6~1confluent5.0.1-1
If that doesn't work then I think the apt sources doesn't have version 0.11.5 of librdkafka.
You can add a repository with the right version of librdkafka in /etc/apt/sources.list as described here:
https://docs.confluent.io/current/installation/installing_cp/deb-ubuntu.html#systemd-ubuntu-debian-install
Related
Problem: To install multiple images to create a application server, but using multi-stage build and size small and keeping only one final image
Question1: Is that possible?
Question2: Does a container always expected to have a base OS, only 1 application dependency and then running compiled code based on 1 application dependency?
I have not found example online where we can create a complete application server (Web Server+Database Server+Application Server) using Docker multistage build.
I would like to have the following in an application server:
Install Alpine
Install nodejs
Install nginx
Install jenkins
Install mongodb
Install Python
This will be the base Image that can be replicated.
Don't want to use : Run apk or Run apt get to install the applications as the image size grows big
Want to use Multi-stage build to have final one image and small size of the image.
However, i want to keep the image size small using MultiStage Build.
FROM alpine:3.17.2 as base1
FROM python:alpine3.17 as base2
FROM nodejs:lts-alpine3.17 as base3
FROM nginx:stable-alpine as base4
FROM jenkins:2.375.3-lts-alpine as base5
FROM mongo:jammy as base6
COPY --from=base1 / /
COPY --from=base2 / /
COPY --from=base3 / /
COPY --from=base4 / /
COPY --from=base5 / /
COPY --from=base6 / /
[this will overwrite some directories]
Expectation
When i run the final image "base7", i can run any nodejs, mongo, python commands. Ingest data file into mongodb, then python analysis and using nodejs to display it.
Previously working without multi-stage build (Issue is size is big and many image layers created)
FROM ubuntu
RUN apt update
RUN apt upgrade
RUN apt-get -y install git
RUN apt-get -y install nodejs
RUN apt-get -y install wget
RUN apt-get -y install gnupg
RUN wget -q -O - https://pkg.jenkins.io/debian-stable/jenkins.io.key | apt-key add -
RUN sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
RUN apt update
RUN apt-get -y install jenkins
RUN apt-get -y install gnupg
RUN wget -qO - https://www.mongodb.org/static/pgp/server-6.0.asc | apt-key add -
RUN echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/6.0 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-6.0.list
RUN wget http://archive.ubuntu.com/ubuntu/pool/main/o/openssl/libssl1.1_1.1.1f-1ubuntu2.16_amd64.deb
RUN apt update
RUN dpkg -i libssl1.1_1.1.1f-1ubuntu2.16_amd64.deb
RUN apt-get -y install libssl-dev
RUN apt-get -y install libssl1.1
RUN apt-get install -y mongodb-org
COPY . /learn
WORKDIR /learn
I started studying docker recently and wanted to install php, apache and postgres in a ubuntu base image. But cannot run it after build.
Here is my Dockerfile
FROM ubuntu:16.04
RUN apt-get update
RUN apt-get -qq -y install curl
RUN apt-get -y install apache2
RUN apt-get -y install php7.0
RUN apt-get -y install libapache2-mod-php7.0
RUN apt-get -y install php7.0-mysql
RUN apt-get -y install php7.0-pgsql
RUN apt-get -y install php7.0-gd
RUN apt-get -y install php-pear
RUN apt-get -y install php7.0-curl
COPY . /var/www/html
EXPOSE 80
Can you please help me?
Thanks.
there are several issues here. First of all, I have the impression that you're mixing up docker and virtual machines. For this I suggest to go and read what are the differences.
Concerning your question, the problem is that the base image (ubuntu:16.04) has CMD bash (or something similar, I really didn't check) and therefore php is not considered.
In order to start apache, you should append the following:
CMD . /etc/apache2/envvars && /usr/sbin/apache2ctl -DFOREGROUND
Anyway, I'd suggest you give a check to the php official image.
I'm bulding nginx in a Debian-based docker image. Every time I run it, it shows me the current nginx version nginx/1.10.3. I need it to download the latest stable nginx.
This is my Dockerfile:
FROM debian:latest
RUN apt-get -y update
RUN apt-get install -yq gnupg2
RUN apt-get install -yq software-properties-common
RUN apt-get install -yq lsb-release
RUN apt-get install -yq curl
RUN add-apt-repository "deb http://archive.canonical.com/ $(lsb_release -sc) partner"
RUN add-apt-repository "deb http://nginx.org/packages/debian `lsb_release -cs` nginx"
RUN apt-get install -y nginx
RUN rm -rf /var/lib/apt/lists/
RUN echo "\ndaemon off;" >> /etc/nginx/nginx.conf
EXPOSE 80
CMD ["/usr/sbin/nginx"]
Docker image layers serve as a cache for subsequent builds. Without some sort of change in the Dockerfile, you're likely getting nginx 1.10.3 because it was cached from a previous build.
Instead of building your own nginx image, you should use the official nginx image, and choose the tag (e.g., 1.15.9) for the version you want.
First off, trivially, you need to apt-get update to fetch the index files from the repos you added before apt will find any packages there.
RUN add-apt-repository blah blah
RUN apt-get update -y # Add this
RUN apt-get install -y whatever
But also, you have invalid repos in the add-apt-repository section. The output of lsb_release -sc is a Debian code name like stretch which of course the Canonical partner repo doesn't have a section for; and the NGninx repo only supports Debian squeeze (though I would expect the packages to also work on newer versions of Debian).
Finally, you need to manage the keys of these repos, or otherwise mark them as safe. As a small bonus, I tried to condense your apt-get downloads slightly. Try this Dockerfile:
FROM debian:latest
RUN apt-get -y update
RUN apt-get install -yq gnupg2 \
software-properties-common curl # lsb-release
# XXX FIXME: the use of [trusted=yes] is really quick and dirty
RUN add-apt-repository "deb [trusted=yes] http://archive.canonical.com/ bionic partner"
RUN add-apt-repository "deb [trusted=yes] http://nginx.org/packages/debian squeeze nginx"
RUN apt-get update -y
RUN apt-get install -y nginx
RUN rm -rf /var/lib/apt/lists/
RUN echo "\ndaemon off;" >> /etc/nginx/nginx.conf
EXPOSE 80
CMD ["/usr/sbin/nginx"]
I'm trying to get a rails server setup on a raspberry pi by building a Docker image.
Image:
FROM ruby:latest
RUN apt-get update -qq && apt-get install -y build-essential libpq-dev
RUN apt-get install -y nodejs
WORKDIR /webapp
COPY Gemfile* /webapp
RUN bundle install
COPY . /webapp/
CMD ["rails", "s", "-b", "0.0.0.0"]
But I'm getting
E: Package 'nodejs' has no installation candidate
EDIT:
I have tried adding the command from the nodejs site curl -sL https://deb.nodesource.com/setup_10.x | bash - && apt-get install -y nodejs
The reason the error Package 'nodejs' has no installation candidate was showing up regardless of how you tried to add the repo to apt-get is because Skipping acquire of configured file 'main/binary-armel/Packages' as repository 'https://deb.nodesource.com/node_10.x stretch InRelease' doesn't support architecture 'armel'.
My solution was to move to an arm32v7/ruby base image and running the same commands. This will install the nodejs_4.8.2~dfsg-1_armhf.deb package which is compatible with the arm architecture.
I building docker image. But it does not success.
/Users/username/.vim exists on my host os, But it does not success.
How I can success docker- build?
Error Message:
Step 16 : ADD /Users/username/.vim/ /root/.vim
lstat Users/username/.vim/: no such file or directory
The following is my Dockerfile.
Dockerfile:
FROM ubuntu:latest
MAINTAINER MyName
RUN /bin/bash
RUN mkdir ~/cworks
RUN mkdir ~/pyworks
RUN apt-get -y update
RUN apt-get -y install curl
RUN apt-get -y install clang
RUN apt-get -y install man
RUN apt-get -y install vim
RUN apt-get -y install python3
RUN apt-get -y install git
RUN apt-get -y install make
RUN apt-get -y upgrade
RUN curl -kL https://bootstrap.pypa.io/get-pip.py | python3
ADD /Users/username/.vim/ /root/.vim
ADD /Users/username/.vimrc /root/.vimrc
Platform: OS X 10.11.4
You cannot execute ADD (or COPY) on a file that is not inside the directory that contains your Dockerfile.
The reason for that is that building docker images is meant to be a deterministic build. If I build the Dockerimage on my computer with your Dockerfile, I would have a different .vim
The docker team impose this limitation to encourage people using a self contained directory with a Dockerfile, and any file to add to it.
In your case, you will need to copy the file in the same directory of the Dockerfile first, and run:
ADD .vim /root/.vim
Or arguably better:
COPY .vim /root/.vim
Use this form instead. It works with hidden files(dot files) and filename contains whitespaces.
ADD ["/Users/username/.vim/", "/root/.vim"]
https://docs.docker.com/engine/reference/builder/#/add
Please remove tailing slash after .vim
So, it will go as
ADD /Users/username/.vim /root/.vim