I have the following heroku.yml:
build:
docker:
web: Dockerfile
run:
web: java $JVM_OPTS -cp target/myapp.jar clojure.main -m myapp.application
And I'm pushing to heroku, and the Docker container loads correctly and the app is deployed too without any warnings or errors, but the app crashes, with these logs:
2020-04-07T22:58:22.151943+00:00 heroku[web.1]: State changed from starting to crashed
2020-04-07T22:58:22.157295+00:00 heroku[web.1]: State changed from crashed to starting
2020-04-07T22:58:22.098498+00:00 app[web.1]: /bin/sh: 1: java: not found
2020-04-07T22:58:25.714638+00:00 heroku[web.1]: State changed from starting to crashed
2020-04-07T22:58:25.654321+00:00 app[web.1]: /bin/sh: 1: java: not found
Why is java not found and what am I doing wrong?
---EDIT---
This is the docker file
# We will use Ubuntu for our image
FROM ubuntu:latest
# Updating Ubuntu packages
ARG CLOJURE_TOOLS_VERSION=1.10.1.507
RUN apt-get -qq update && apt-get -qq -y install curl wget bzip2 openjdk-8-jdk-headless \
&& curl -sSL https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -o /tmp/miniconda.sh \
&& bash /tmp/miniconda.sh -bfp /usr/local \
&& rm -rf /tmp/miniconda.sh \
&& conda install -y python=3 \
&& conda update conda \
&& curl -o install-clojure https://download.clojure.org/install/linux-install-${CLOJURE_TOOLS_VERSION}.sh \
&& chmod +x install-clojure \
&& ./install-clojure && rm install-clojure \
&& wget https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein \
&& chmod a+x lein \
&& mv lein /usr/bin \
&& apt-get -qq -y autoremove \
&& apt-get autoclean \
&& rm -rf /var/lib/apt/lists/* /var/log/dpkg.log \
&& conda clean --all --yes
ENV PATH /opt/conda/bin:$PATH
RUN conda create -n pyclj python=3.6 && conda install -n pyclj numpy mxnet
## To install pip packages into the pyclj environment do
RUN conda run -n pyclj python3 -mpip install numpy
Related
My docker-compose.yml:
version: "3.3"
services:
build_and_run_service:
image: myapp:0
build: .
network_mode: host
volumes:
- './bin/cookie:/app/cookie'
- './bin/logs:/app/logs'
- './bin/warehouse:/app/warehouse'
Dockerfile doesn't contain CMD and ENTRYPOINT, so when I execute commands in that order:
docker build --tag myapp:0 .
docker run -d -t myapp:0
docker exec -it <container_id> /bin/bash
It works as expected.
For some reason the container is not working when using docker compose...
Commands order:
docker-compose up -d --build
docker-compose run -d build_and_run_service bash
What's wrong?
Both cases work fine on Windows but not on Ubuntu...
#edit
Dockerfile:
FROM ubuntu:20.04 as runtime
LABEL description="Build and run container - myapp"
RUN apt-get update
RUN apt-get install -y software-properties-common
RUN apt-get install -y nano
RUN apt-get install -y wget
RUN apt-get install -y curl
RUN apt-get install -y make
RUN apt-get install -y build-essential
RUN apt-get install -y tcl zlib1g-dev libssl-dev tk libcurl4-gnutls-dev libexpat1-dev gettext dos2unix
# Compilers
RUN apt-get install -y gcc-10
RUN apt-get install -y g++-10
RUN rm /usr/bin/gcc \
&& ln -s /usr/bin/gcc-10 /usr/bin/gcc
RUN rm /usr/bin/g++ \
&& ln -s /usr/bin/g++-10 /usr/bin/g++
# Postgres dev
RUN sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
RUN wget --no-check-certificate --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
RUN apt-get update
RUN apt-get install -y libpq-dev postgresql-server-dev-13
RUN cd /tmp \
&& wget --no-check-certificate https://www.openssl.org/source/openssl-1.1.1g.tar.gz \
&& tar -zxf openssl-1.1.1g.tar.gz \
&& cd openssl-1.1.1g \
&& ./config \
&& make \
&& make install \
&& rm /usr/bin/openssl \
&& ln -s /usr/local/bin/openssl /usr/bin/openssl \
&& ldconfig
RUN cd /tmp \
&& wget --no-check-certificate https://cmake.org/files/v3.19/cmake-3.19.6-Linux-x86_64.tar.gz \
&& tar -zxf cmake-3.19.6-Linux-x86_64.tar.gz \
&& mv cmake-3.19.6-Linux-x86_64 /usr/local/ \
&& ln -s /usr/local/cmake-3.19.6-Linux-x86_64/bin/cmake /usr/bin/cmake
RUN cd /tmp \
&& wget --no-check-certificate https://mirrors.edge.kernel.org/pub/software/scm/git/git-2.31.0.tar.gz \
&& tar -zxf git-2.31.0.tar.gz \
&& cd git-2.31.0 \
&& make prefix=/usr/local all \
&& make prefix=/usr/local install
RUN cd /tmp \
&& wget --no-check-certificate https://boostorg.jfrog.io/artifactory/main/release/1.75.0/source/boost_1_75_0.tar.gz \
&& tar -zxf boost_1_75_0.tar.gz \
&& cd boost_1_75_0 \
&& ./bootstrap.sh \
&& ./b2 \
&& ./b2 install
VOLUME ["/app/cookie", "/app/logs", "/app/warehouse"]
WORKDIR /app
COPY . /src
RUN cd /src \
&& mkdir build \
&& cd build
# Some building command
## PRIVATE ##
# Removes tmp
RUN cd /tmp \
&& rm -r *
I am trying to create a GPU microservice using Nvidia cuda Base image, but during the docker build, I am facing Driver not found issue, can someone point out what is missing here?
DockerFile:
FROM nvidia/cuda:10.1-devel
# Install some basic utilities
RUN apt-get update && apt-get install -y \
curl \
ca-certificates \
sudo \
git \
bzip2 \
libx11-6 \
&& rm -rf /var/lib/apt/lists/*
ENV CONDA_AUTO_UPDATE_CONDA=false
ENV PATH=/home/user/miniconda/bin:$PATH
RUN curl -sLo ~/miniconda.sh https://repo.continuum.io/miniconda/Miniconda3-py37_4.8.2-Linux-x86_64.sh \
&& chmod +x ~/miniconda.sh \
&& ~/miniconda.sh -b -p ~/miniconda \
&& rm ~/miniconda.sh \
&& conda install -y python==3.7 \
&& conda clean -ya
ENV PATH="/usr/local/cuda-10.1/bin:$PATH"
ENV LD_LIBRARY_PATH="/usr/local/cuda-10.1/lib64:$LD_LIBRARY_PATH"
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
ENV NVIDIA_VISIBLE_DEVICES=all
ENV FORCE_CUDA="1"
RUN conda install pytorch==1.4.0 torchvision==0.5.0 cudatoolkit=10.1 -c pytorch
RUN pip install -v -e .
Error:
"/home/user/miniconda/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 1013, in _get_cuda_arch_flags
capability = torch.cuda.get_device_capability()
File "/home/user/miniconda/lib/python3.7/site-packages/torch/cuda/__init__.py", line 320, in get_device_capability
prop = get_device_properties(device)
File "/home/user/miniconda/lib/python3.7/site-packages/torch/cuda/__init__.py", line 325, in get_device_properties
_lazy_init() # will define _get_device_properties and _CudaDeviceProperties
File "/home/user/miniconda/lib/python3.7/site-packages/torch/cuda/__init__.py", line 196, in _lazy_init
_check_driver()
File "/home/user/miniconda/lib/python3.7/site-packages/torch/cuda/__init__.py", line 101, in _check_driver
http://www.nvidia.com/Download/index.aspx""")
AssertionError:
Found no NVIDIA driver on your system. Please check that you
have an NVIDIA GPU and installed a driver from
http://www.nvidia.com/Download/index.aspx
The issues happens during execution of last step in docker file.
I tried using multiple Nvidia base docker images, but didn't help much. (cuda:10.1-base-ubuntu18.04, cuda:10.1-runtime-ubuntu18.04)
Any pointers appreciated.
After lot of trial and errors and going through a lot of documentation, this is what worked fine.
ARG PYTORCH=1.3
ARG CUDA=10.1
ARG CUDNN=7
FROM pytorch/pytorch:1.3-cuda10.1-cudnn7-devel
RUN mkdir /app
WORKDIR /app
ENV TORCH_CUDA_ARCH_LIST="5.2 6.0 6.1 7.0+PTX"
ENV TORCH_NVCC_FLAGS="-Xfatbin -compress-all"
ENV CMAKE_PREFIX_PATH="$(dirname $(which conda))/../"
RUN apt-get update && apt-get install -y libglib2.0-0 libsm6 libxrender-dev libxext6 \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# Install some basic utilities
RUN apt-get update && apt-get install -y \
curl \
ca-certificates \
sudo \
git \
bzip2 \
libx11-6 \
&& rm -rf /var/lib/apt/lists/*
RUN apt-get update && \
apt-get install -y --no-install-recommends \
build-essential g++ \
libglib2.0-0 libsm6 libxrender-dev libxext6 wget
# Create a non-root user and switch to it
RUN adduser --disabled-password --gecos '' --shell /bin/bash user \
&& chown -R user:user /app
RUN echo "user ALL=(ALL) NOPASSWD:ALL" > /etc/sudoers.d/90-user
USER user
# All users can use /home/user as their home directory
ENV HOME=/home/user
RUN chmod 777 /home/user
# Install Miniconda and Python 3.7
ENV CONDA_AUTO_UPDATE_CONDA=false
ENV PATH=/home/user/miniconda/bin:$PATH
RUN curl -sLo ~/miniconda.sh https://repo.continuum.io/miniconda/Miniconda3-py37_4.8.2-Linux-x86_64.sh \
&& chmod +x ~/miniconda.sh \
&& ~/miniconda.sh -b -p ~/miniconda \
&& rm ~/miniconda.sh \
&& conda install -y python==3.7 \
&& conda clean -ya
RUN conda install pytorch==1.4.0 torchvision==0.5.0 cudatoolkit=10.1 -c pytorch
RUN pip install -v -e .
Hope this helps!
Good luck!
I have the following Dockerfile:
FROM ubuntu:latest
RUN apt-get -qq update && apt-get -qq -y install wget\
&& wget https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein \
&& chmod a+x lein \
&& cp lein /usr/bin
RUN "lein -v"
to download lein from the url, and put it in /usr/bin. But still the
RUN lein -v
command doesn't work.
I get the error:
remote: Step 18/22 : RUN lein -v
remote: ---> Running in e5f404275fe2
remote: /bin/sh: 1: lein -v: not found
remote: The command '/bin/sh -c lein -v' returned a non-zero code: 127
remote:
remote: Verifying deploy...
remote:
remote: ! Push rejected to appname
On Heroku one-off dyno itself, the command works.
$ /bin/sh -c "lein -v"
the problem is that docker RUN expects the command unquoted, as is, or as an array of command and its args:
one of RUN lein -v or RUN ["lein", "-v"] should do the trick.
the other problem, that your image doesn't have java installed, so the command would fail anyway. So you need to install it somehow. Your final Dockerfile could look like this:
FROM ubuntu:latest
RUN apt-get -qq update && apt-get -qq -y install wget\
&& wget https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein \
&& chmod a+x lein \
&& cp lein /usr/bin
RUN DEBIAN_FRONTEND=noninteractive \
apt-get -y install default-jre-headless && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
RUN ["lein", "-v"]
-- EDIT --
In fact, adding the java dependency and changing to RUN ["lein", "-v"] doesn't work either. These are the first fifteen steps of the Dockerfile:
ARG CLOJURE_TOOLS_VERSION=1.10.1.507
RUN apt-get -qq update && apt-get -qq -y install curl wget bzip2 openjdk-8-jdk-headless\
&& curl -sSL https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -o /tmp/miniconda.sh \
# && curl -sSL https://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-x86_64.sh -o /tmp/miniconda.sh \
&& bash /tmp/miniconda.sh -bfp /usr/local \
&& rm -rf /tmp/miniconda.sh \
&& conda install -y python=3 \
&& conda update conda \
&& curl -o install-clojure https://download.clojure.org/install/linux-install-${CLOJURE_TOOLS_VERSION}.sh \
&& chmod +x install-clojure \
&& ./install-clojure && rm install-clojure \
# no need to install lein
&& wget https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein > /usr/bin/lein \
&& chmod 777 /usr/bin/lein \
&& apt-get -qq -y autoremove \
&& apt-get autoclean \
&& rm -rf /var/lib/apt/lists/* /var/log/dpkg.log \
&& conda clean --all --yes
ENV PATH /usr/bin:$PATH
ENV NODE_VERSION=12.18.1
RUN curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.34.0/install.sh | bash
ENV NVM_DIR=/root/.nvm
RUN . "$NVM_DIR/nvm.sh" && nvm install ${NODE_VERSION}
RUN . "$NVM_DIR/nvm.sh" && nvm use v${NODE_VERSION}
RUN . "$NVM_DIR/nvm.sh" && nvm alias default v${NODE_VERSION}
ENV PATH="/root/.nvm/versions/node/v${NODE_VERSION}/bin/:${PATH}"
RUN node --version
RUN npm --version
# ENV PATH /opt/conda/bin:$PATH
# RUN conda create -n pyclj python=3.7 && conda install -n pyclj numpy mxnet \
# && conda install -c conda-forge opencv
# ## To install pip packages into the pyclj environment do
# RUN conda run -n pyclj python3 -mpip install numpy opencv-python
FROM openjdk:8-alpine
RUN ["lein", "-v"]
and it gives the error
remote: Step 15/19 : RUN ["lein", "-v"]
remote: ---> Running in b817213d45b5
remote: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"lein\": executable file not found in $PATH": unknown
remote:
remote: Verifying deploy...
remote:
remote: ! Push rejected to
I have the following Dockerfile:
# We will use Ubuntu for our image
FROM ubuntu:latest
# Updating Ubuntu packages
ARG CLOJURE_TOOLS_VERSION=1.10.1.507
RUN apt-get -qq update && apt-get -qq -y install curl wget bzip2 openjdk-8-jdk-headless \
&& curl -sSL https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -o /tmp/miniconda.sh \
# && curl -sSL https://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-x86_64.sh -o /tmp/miniconda.sh \
&& bash /tmp/miniconda.sh -bfp /usr/local \
&& rm -rf /tmp/miniconda.sh \
&& conda install -y python=3 \
&& conda update conda \
&& curl -o install-clojure https://download.clojure.org/install/linux-install-${CLOJURE_TOOLS_VERSION}.sh \
&& chmod +x install-clojure \
&& ./install-clojure && rm install-clojure \
# no need to install lein
&& wget https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein \
&& chmod a+x lein \
&& mv lein /usr/bin \
&& apt-get -qq -y autoremove \
&& apt-get autoclean \
&& rm -rf /var/lib/apt/lists/* /var/log/dpkg.log \
&& conda clean --all --yes
ENV PATH /opt/conda/bin:$PATH
RUN conda create -n pyclj python=3.7 && conda install -n pyclj numpy mxnet \
&& conda install -c conda-forge opencv
## To install pip packages into the pyclj environment do
RUN conda run -n pyclj python3 -mpip install numpy opencv-python
FROM openjdk:8-alpine
RUN lein uberjar
COPY target/uberjar/vendo.jar /vendo/app.jar
EXPOSE 3000
CMD ["java", "-jar", "/vendo/app.jar", "--server.port=$PORT"]
And I'm pushing my project using git push heroku master, and I get the error:
remote: Step 8/11 : RUN lein uberjar
remote: ---> Running in 07533c6b0e9c
remote: /bin/sh: lein: not found
remote: The command '/bin/sh -c lein uberjar' returned a non-zero code: 127
Suggesting that lein wasn't installed. The wget in that first RUN command is supposed to install lein. How do I fix this issue?
You need to add a RUN lein to run the self-install.
I recently updated Chrome Browser to 78 version, and it has caused an issue.
I am running Selenium tests inside Linux Docker container in Headless Chrome mode, with latest chrome-78.0.3904.108, driver-78.0.3904.105 and selenium- 3.141.0, specflow packages -3.1.67.
I have tried almost all capabilities suggested on this forum to run Chrome headlessly inside Docker.
case "Headless_Chrome":
string driverPath = "/opt/selenium/";
string driverExecutableFileName = "chromedriver";
ChromeDriverService service_headless = ChromeDriverService.CreateDefaultService(driverPath, driverExecutableFileName);
chrome_options.BinaryLocation = "/opt/google/chrome/chrome";
chrome_options.AddArgument("--no-sandbox");
chrome_options.AddArgument("--headless");
chrome_options.AddArgument("--window-size=1420,1080");
chrome_options.AddArgument("--disable-extensions");
chrome_options.AddArgument("--proxy-server='direct://'");
chrome_options.AddArgument("--proxy-bypass-list=*");
chrome_options.AddArgument("--disable-gpu"); //even will come redundant in case of linux
chrome_options.AddArgument("--disable-dev-shm-usage"); // to fix - error: unknown error: session deleted because of page crash
chrome_options.AddArgument("--remote-debugging-port=9222");
chrome_options.AddArgument("--remote-debugging-address=0.0.0.0");
chrome_options.AddArgument("--disable-infobars");
chrome_options.AddArgument("--user-data-dir=/data");
chrome_options.AddArgument("--disable-features=VizDisplayCompositor"); //to save from zombie chrome process running
//chrome_options.AddArgument("--disable-setuid-sandbox");
//chrome_options.AddArgument("--privileged"); // can be a security risk
//chrome_options.AddArgument("--lang=en_US");
//chrome_options.AddArgument("--start-maximized");
//chrome_options.AddAdditionalCapability("useAutomationExtension", false);
driver = new ChromeDriver(service_headless, chrome_options, TimeSpan.FromSeconds(120));
break;
My Docker file(License key is replaced with xxxxx ) :
FROM microsoft/dotnet:2.2-sdk
ENV PATH="${PATH}:/root/.dotnet/tools"
RUN dotnet tool install --global SpecFlow.Plus.License
RUN specflow-plus-license register --licenseKey KBD0xxxxxxxxxxxxxxxxxxxxiqQGIUTnUBAU/wn/EAAA== --issuedTo "xxxxxxxxxxxxxx"
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8
ENV LC_ALL en_US.UTF-8
ENV C en_US.UTF-8
ENV TERM xterm
ENV TZ Europe/Copenhagen
USER root
# Install Chrome
RUN apt-get update && apt-get install -y \
apt-transport-https ca-certificates curl gnupg hicolor-icon-theme \
libcanberra-gtk* libgl1-mesa-dri libgl1-mesa-glx libpango1.0-0 libpulse0 \
libv4l-0 fonts-symbola \
--no-install-recommends \
&& curl -sSL https://dl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& echo "deb [arch=amd64] https://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google.list \
&& apt-get update && apt-get install -y google-chrome-stable --no-install-recommends \
&& apt-get purge --auto-remove -y curl \
&& rm -rf /var/lib/apt/lists/*
#RUN dpkg -s google-chrome-stable
#RUN apt-get update && apt-get search google-chrome-stable && apt-get show google-chrome-stable
# Download the google-talkplugin And ChromeDrive
ARG CHROME_DRIVER_VERSION="latest"
RUN set -x \
&& apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates curl unzip \
&& rm -rf /var/lib/apt/lists/* \
&& curl -sSL "https://dl.google.com/linux/direct/google-talkplugin_current_amd64.deb" -o /tmp/google-talkplugin-amd64.deb \
&& dpkg -i /tmp/google-talkplugin-amd64.deb \
&& rm -rf /tmp/*.deb \
&& CD_VERSION=$(if [ ${CHROME_DRIVER_VERSION:-latest} = "latest" ]; then echo $(wget -qO- https://chromedriver.storage.googleapis.com/LATEST_RELEASE); else echo $CHROME_DRIVER_VERSION; fi) \
&& echo "Using chromedriver version: "$CD_VERSION \
&& mkdir /opt/selenium \
&& curl -sSL "https://chromedriver.storage.googleapis.com/$CD_VERSION/chromedriver_linux64.zip" -o /tmp/chromedriver.zip \
&& unzip -o /tmp/chromedriver -d /opt/selenium/ \
&& rm -rf /tmp/*.zip \
&& apt-get purge -y --auto-remove curl unzip
# Add chrome user
RUN groupadd -r chrome && useradd -r -g chrome -G audio,video chrome \
&& mkdir -p /home/chrome/Downloads && chown -R chrome:chrome /home/chrome
#ENV DISPLAY=:99
WORKDIR /data/WebShopTestAutomation
# copy code
RUN mkdir -p /data && mkdir /reports
COPY ./source /data
RUN ls -ls /data
RUN cd /data/WebShopTestAutomation && dotnet build
CMD ["dotnet", "vstest", "--logger:trx;LogFileName=/reports/TestResults/report.trx", "/data/WebShopTestAutomation/bin/Debug/netcoreapp2.2/WebShopTestAutomation.dll"]
Closing this issue,
as Root cause was this : default.srprofile which was used in project was not working in Docker, it is just not read.
Have raised a issue at Git hub Specflow
For details:
https://github.com/techtalk/SpecFlow/issues/1841