if someone can help me with this, i would be very grateful, i have a docker image in which a kafka is displayed where i pretend to have 3 brokers and i would like that nothing more be created when the docker container is created, the script that i have to raise kafka will be executed, i have tried in many ways using CMD and ENTRYPOINT commands but i am not successful, the container is created for me but the script is not executed i have to enter the container to start it
Dockerfile
FROM ubuntu
RUN apt-get update
RUN apt-get install -y openjdk-8-jdk
RUN apt-get install -y wget \
&& wget http://apache.rediris.es/kafka/2.4.0/kafka_2.12-2.4.0.tgz \
&& tar -xzf kafka_2.12-2.4.0.tgz \
&& rm -R kafka_2.12-2.4.0.tgz
#WORKDIR /home
RUN chmod +x /kafka_2.12-2.4.0
### COPY ###
COPY server-1.properties /kafka_2.12-2.4.0/config/
COPY server-2.properties /kafka_2.12-2.4.0/config/
#ADD runzk-kf.sh .
COPY runzk-kf.sh /usr/local/bin/runzk-kf.sh
#COPY runzk-kf.sh .
RUN chmod +x /usr/local/bin/runzk-kf.sh
EXPOSE 2181
EXPOSE 9092
EXPOSE 9093
EXPOSE 9094
CMD ./bin/bash
script
#!/bin/sh
# turn on bash's job control
set -m
### RUN Zookeper
./kafka_2.12-2.4.0/bin/zookeeper-server-start.sh /kafka_2.12-2.4.0/config/zookeeper.properties &
### RUN Kafka brokers ###
./kafka_2.12-2.4.0/bin/kafka-server-start.sh /kafka_2.12-2.4.0/config/server.properties &
./kafka_2.12-2.4.0/bin/kafka-server-start.sh /kafka_2.12-2.4.0/config/server-1.properties &
./kafka_2.12-2.4.0/bin/kafka-server-start.sh /kafka_2.12-2.4.0/config/server-2.properties &
View all code
Sorry, but please don't do this.
Docker images should be one service, not 4. Use Compose or MiniKube + Helm Charts to orchestrate multiple.
It's not clear what property files you changed for that to work properly.
JDK 8 is end of life, use 11 or 13, which Kafka supports.
Just use existing Docker images. If you want something minimal, personally I use bitnami/kafka. If you want something more fully featured, take a look over at Confluent's repo on running 3 Brokers via Docker Compose.
Related
After reading Dbt documentation, I've had a hard time to figure out how to install dbt-core (or any other packages i.e. dbt-postgres, dbt-snowflake, etc) on Windows 10.
I have Docker Desktop installed, running a couple of containers already (mostly nodeJS containers, and Kafka). However, it was hard to understand how I would have those new Dbt containers available in my Docker Desktop Console.
I can see docker images were installed properly
$docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
**ghcr.io/dbt-labs/dbt-core 1.2.1 802a0d70aedc 4 weeks ago 538MB**
**ghcr.io/dbt-labs/dbt-bigquery 1.2.latest b7502bcd3b35 2 months ago 559MB**
...
postgres latest f8dd270e5152 7 weeks ago 376MB
dpage/pgadmin4 latest d13c9d7d0193 2 months ago 382MB
wurstmeister/kafka latest a692873757c0 4 months ago 468MB
wurstmeister/zookeeper latest 3f43f72cb283 3 years ago 510MB
Does anyone know how to I them to the Desktop Console?
I'm currently on Windows 10 and use a Docker image for my dbt project without needing WSL. Below is my Dockerfile and requirements.txt file with dbt-core and dbt-snowflake but feel free to swap the packages you need.
In my repo, my dbt project is in a folder at the root level named dbt.
requirements.txt
dbt-core==1.1.0
dbt-snowflake==1.1.0
Dockerfile
FROM public.ecr.aws/docker/library/python:3.8-slim-buster
COPY . /dbt
# Update and install system packages
RUN apt-get update -y && \
apt-get install --no-install-recommends -y -q \
git libpq-dev python-dev && \
apt-get clean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
# Install dbt
RUN pip install -U pip
RUN pip install -r dbt/requirements.txt
# TEMP FIX due to dependency updates. See https://github.com/dbt-labs/dbt-core/issues/4745
RUN pip install --force-reinstall MarkupSafe==2.0.1
# Install dbt dependencies
WORKDIR /dbt
RUN dbt deps
# Specify profiles directory
ENV DBT_PROFILES_DIR=.dbt
# Expose port for dbt docs
EXPOSE 8080
And then you can build and run it (I personally put both of these commands in a dbt_run.sh file and run with bash dbt_run.sh):
docker build -t dbt_image .
docker run \
-p 8080:8080 \
--env-file .env \
-it \
--mount type=bind,source="$(pwd)",target=/dbt \
dbt_image bash
If you make changes to your dbt project while the container is running they will be reflected in the container which makes it great for developing locally. Hope this helps!
I finally was able to pull the image. To add a container in the Docker desktop, I just needed to actually run it.
However, running a dbt-core container in docker, it returns an error:
right after I start the container it stops and returns exit(1), as per the screenshot.
I'm trying to run a Go app using Docker on Google Cloud Run but I'm getting this error:
Container failed to start. Failed to start and then listen on the port defined by the PORT environment variable. Logs for this revision might contain more information.
I fixed my port to be 8080 as stated in the docs but I think my Dockerfile is incorrect. Does anyone know what I'm missing?
FROM golang:1.12-alpine
RUN apk upgrade -U \
&& apk add \
ca-certificates \
git \
libva-intel-driver \
make \
&& rm -rf /var/cache/*
ENV GOOS linux
ENV GOARCH amd64
ENV CGO_ENABLED=0
ENV GOFLAGS "-ldflags=-w -ldflags=-s"
ENV PATH $GOPATH/bin:/usr/local/go/bin:$PATH
RUN echo $PATH
RUN mkdir -p "$GOPATH/src" "$GOPATH/bin" && chmod -R 777 "$GOPATH"
RUN go get -u github.com/cespare/reflex
# RUN reflex -h
# Setup modules after reflex install
ENV GO111MODULE=on \
GOFLAGS="$GOFLAGS -mod=vendor"
WORKDIR /go/src/bitbucket.org/team/app/
COPY . .
CMD [ "go", "run", "cmd/main.go" ]
Dockerfiles don't make your application listen on a specific port number.
The EXPOSE directive in Dockerfile is purely a documentation and doesn't do anything functional.
You have 2 options for a Go app:
Just refactor your code to read the PORT env variable: os.Getenv("PORT") and use it on the HTTP server address you’re starting:
port := os.Getenv("PORT")
http.ListenAndServe(":"+port)
Create a -port flag and read it during the entrypoint of your app in the Dockerfile:
e.g. if you can make go run main.go -port=8080 work, change your dockerfile to:
exec go run main.go -port=$PORT
These will get you what you want.
Ideally you should not use go run inside a container. Just do:
RUN go build -o /bin/my-app ./my/pkg
ENTRYPOINT /bin/my-app
to compile a Go program and use it directly. Otherwise, every time Cloud Run starts your container, you would be re-compiling it from scratch, which is not fast, this will increase your cold start times.
Aside from these you seem to have a lot of inconsistencies in your dockerfile. You set a lot of Go env vars like GOOS GOARCH but you don't actually go build your app (go run is an on-the-fly compilation and doesn't take the linker flags in GOFLAGS into account I believe). Look at sample Go dockerfiles to have a better idea on how to write idiomatic Go dockerfiles.
It seems that you are missing the EXPOSE in your Dockerfile. See https://docs.docker.com/engine/reference/builder/#expose
There is no cassandra alpine image available on Docker Hub.The cassandra image uses debian:stretch-slim as the base image which has vulnerabilities.
Is there any plan to release the alpine version of cassandra with lesser vulnerabilities?
Vulnerability scan of cassandra image through JFrog Xray:
The awesome thing about Docker is, you can make anything you want yourself! This is a very simplistic build, and I advice you to look at the official Dockerfile and how they set it up. But hopefully this gets you going.
Dockerfile:
FROM alpine:latest
# Cassandra 3.0 and later require Java 8u40 or later.
RUN apk --no-cache update \
&& apk --no-cache add openjdk8-jre
RUN wget http://apache.40b.nl/cassandra/3.11.3/apache-cassandra-3.11.3-bin.tar.gz \
&& tar -xzvf apache-cassandra-3.11.3-bin.tar.gz \
&& rm -rf apache-cassandra-3.11.3-bin.tar.gz
WORKDIR /apache-cassandra-3.11.3/bin/
# 7000: intra-node communication
# 7001: TLS intra-node communication
# 7199: JMX
# 9042: CQL
# 9160: thrift service
EXPOSE 7000 7001 7199 9042 9160
CMD ["./cassandra", "-R", "-f"]
Build it:
docker build -t alpine-cassandra .
Run it:
docker run alpine-cassandra
I'm trying to add redis to a php:7.0-apache image, using this Dockerfile:
FROM php:7.0-apache
RUN apt-get update && apt-get -y install build-essential tcl
RUN cd /tmp \
&& curl -O http://download.redis.io/redis-stable.tar.gz \
&& tar xzvf redis-stable.tar.gz \
&& cd redis-stable \
&& make \
&& make install
COPY php.ini /usr/local/etc/php/
COPY public /var/www/html/
RUN chown -R root:www-data /var/www/html
RUN chmod -R 1755 /var/www/html
RUN find /var/www/html -type d -exec chmod 1775 {} +
RUN mkdir -p /var/redis/6379
COPY 6379.conf /etc/redis/6379.conf
COPY redis_6379 /etc/init.d/redis_6379
RUN chmod 777 /etc/init.d/redis_6379
RUN update-rc.d redis_6379 defaults
RUN service apache2 restart
RUN service redis_6379 start
It build and run fines but redis is never started? When I run /bin/bash inside my container and manually input "service redis_6379 start" it works, so I'm assuming my .conf and init.d files are okay.
While I'm aware it'd much easier using docker-compose, I'm specifically trying to avoid having to use it for specific reasons.
There are multiple things wrong here:
Starting processes in dockerfile has no effect. A dockerfile builds an image. The processes need to be started at container construction time. This can be done using an entrypoint can be defined in the dockerfile by using ENTRYPOINT. That entrypoint is typically a script that is executed when an actual container is started.
There is no init process in docker by default. Issuing service calls will fail without further work. If you need to start multiple processes you can look for the docs of the supervisord program.
Running both redis and a webserver in one container is not best practice. For a php application using redis you'd typically have 2 containers - one running redis and one running apache and let them interact via network.
I suggest you read the docker documentation before continuing. All this is described in depth there.
I am agree with #Richard. Use two or more containers according to your needs then --link them, in order to get the things work!
I containerised a shiny app and attempted to deploy it on GCP using Kubernetes but each time I obtain the external IP address and load it in the browser, I get the "this site cannot be reached connection refused" error. So I have attempted to run the container on my localhost to troubleshoot and now I get the "127.0.0.1 didn’t send any data. ERR_EMPTY_RESPONSE" error. I searched tirelessly online for a solution but nothing seems to work for me. Plus none of the solutions on are for a shiny app docker container. Many of the fixes mention the port but I am still stuck. I have xampp installed on my mac by the way. Is it possible that xampp and my docker container are attempting to share the same port or is there a problem with my Docker file code? Pardon me but I am new to containers and have only been following the documentation procedure up till now. Below is my Docker file code:
Dockerfile
# Install R version 3.5.1
FROM r-base:3.5.1
# Install Ubuntu packages
RUN apt-get update && apt-get install -y \
sudo \
gdebi-core \
pandoc \
pandoc-citeproc \
libcurl4-gnutls-dev \
libcairo2-dev/unstable \
libxt-dev \
libssl-dev
# Download and install ShinyServer (latest version)
RUN wget --no-verbose https://s3.amazonaws.com/rstudio-shiny-server-os-
build/ubuntu-12.04/x86_64/VERSION -O "version.txt" && \
VERSION=$(cat version.txt) && \
wget --no-verbose "https://s3.amazonaws.com/rstudio-shiny-server-os-
build/ubuntu-12.04/x86_64/shiny-server-$VERSION-amd64.deb" -O ss-latest.deb &&
\
gdebi -n ss-latest.deb && \
rm -f version.txt ss-latest.deb
# Install R packages that are required
# TODO: add further package if you need!
RUN R -e
"install.packages(c('shiny','shinyjs','tools','foreign','XLConnect'),
repos='http://cran.rstudio.com/')"
# Copy configuration files into the Docker image
COPY shiny-server.conf /etc/shiny-server/shiny-server.conf
COPY /app /srv/shiny-server/
# Make the ShinyApp available at port 80
EXPOSE 80
# Copy further configuration files into the Docker image
COPY shiny-server.sh /usr/bin/shiny-server.sh
CMD ["/usr/bin/shiny-server.sh"]
I would appreciate it if someone could assist me.
You don't need shiny-server.
add
app <- shinyApp(ui = ui, server = server)
runApp(app, host ="0.0.0.0", port = 80, launch.browser = FALSE)
to your R script and
EXPOSE 80
CMD ["R", "-e", "library(shiny); source('/root/pathToYourScript/script.R')"]
to your Dockerfile.
I had to first create a Rprofile.site file and place it in the same directory as the dockerfile and shinyapp. Then I created my own base image with all the necessary libraries for the app and called it from my dockerfile. Here is the final code:
Rprofile.site
local({
options(shiny.port = 3838, shiny.host = "0.0.0.0")
})
Dockerfile
FROM bimage_rpackages
# Copy the app to the image
RUN mkdir /root/shinyapp
COPY app/shinyapp /root/shinyapp
COPY app/Rprofile.site /usr/lib/R/etc/
# Make the ShinyApp available at port 3838
EXPOSE 3838
CMD ["R", "-e", "shiny::runApp('/root/shinyapp')"]