I use a script to build thee dockerfile. below is my script...
echo "FROM ubuntu:14.04" >> Dockerfile
echo "RUN rm /bin/sh && ln -s /bin/bash /bin/sh" >> Dockerfile
echo "RUN apt-get -y update && apt-get upgrade -y" >> Dockerfile
echo "RUN apt-get install -y software-properties-common" >> Dockerfile
echo "RUN apt-get -y update && add-apt-repository -y ppa:webupd8team/java" >> Dockerfile
echo "RUN echo debconf shared/accepted-oracle-license-v1-1 select true | debconf-set-selections" >> Dockerfile
echo "RUN echo debconf shared/accepted-oracle-license-v1-1 seen true | debconf-set-selections" >> Dockerfile
echo "RUN apt-get -y update && apt-get install -y oracle-java8-installer" >> Dockerfile
echo "RUN apt-get install -y curl " >> Dockerfile
echo "RUN apt-get install -y unzip " >> Dockerfile
echo "RUN apt-get -y update && curl -s get.sdkman.io | bash" >> Dockerfile
echo 'RUN source "$HOME/.sdkman/bin/sdkman-init.sh"' >> Dockerfile
echo 'RUN source ~/.profile' >> Dockerfile
echo "RUN yes | sdk install groovy" >> Dockerfile
...
docker build -t imagename:version ./
...
but I get the below error
RUN yes | sdk install groovy
---> Running in 09056add5ab7
/bin/sh: sdk: command not found
The command '/bin/sh -c yes | sdk install groovy' returned a non-zero code: 127
if I dont use this command "sdk install groovy" the build is sucessfull. and then i an run the image and issue the same command and it works.
Any help, any idea why this is hapenning?
RUN yes | /bin/bash -l -c 'sdk install groovy'
worked.
Related
I run a given Dockerfile in order to build image for my TeamCity Agent
FROM jetbrains/teamcity-agent:2022.10.1-linux-sudo
RUN curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
RUN sudo sh -c 'echo deb https://apt.kubernetes.io/ kubernetes-xenial main > /etc/apt/sources.list.d/kubernetes.list'
RUN curl -sL https://deb.nodesource.com/setup_16.x | sudo -E bash -
# https://github.com/AdoptOpenJDK/openjdk-docker/blob/master/12/jdk/ubuntu/Dockerfile.hotspot.releases.full
RUN sudo apt-get update && \
sudo apt-get install -y ffmpeg gnupg2 git sudo kubectl \
binfmt-support qemu-user-static mc jq
#RUN wget -O - https://apt.kitware.com/keys/kitware-archive-la3est.asc 2>/dev/null | gpg --dearmor - | sudo tee /etc/apt/trusted.gpg.d/kitware.gpg >/dev/null
#RUN sudo apt-add-repository 'deb https://apt.kitware.com/ubuntu/ focal main' && \
# sudo apt-get update && \
RUN sudo apt install -y cmake build-essential wget
RUN sudo curl -L https://nodejs.org/dist/v14.17.3/node-v14.17.3-linux-x64.tar.gz --output node-v14.17.3-linux-x64.tar.gz
RUN sudo tar -xvf node-v14.17.3-linux-x64.tar.gz
RUN echo 'export PATH="$HOME/node-v14.17.3-linux-x64/bin:$PATH"' >> ~/.bashrc
RUN echo "The version of Node.js is $(node -v)"
All the code was right, but then I decided to add node.js installation to the Dockerfile. that begins from this line:
RUN sudo curl -L https://nodejs.org/dist/v14.17.3/node-v14.17.3-linux-x64.tar.gz --output node-v14.17.3-linux-x64.tar.gz
However, the problem right now is that I have the following error, during execution of the last line of the Dockerfile:
RUN echo "The version of Node.js is $(node -v)"
Output for this line is:
Step 10/22 : RUN echo "The version of Node.js is $(node -v)"
21:07:41 ---> Running in 863b0e75e45a
21:07:42 /bin/sh: 1: node: not found
You need to make the 2 following changed in your Dockerfile for your node installation to be included in your $PATH env var -
Remove the $HOME variable from the path you're concating, as you are currently downloading node to your root folder and not the $HOME folder -
RUN echo 'export PATH="/node-v14.17.3-linux-x64/bin:$PATH"' >> ~/.bashrc
Either source ~/.bashrc explicitly for the $PATH changes to take place or run the export command as part of the Dockerfile
Once you apply these 2 changes, the error should go away.
I'm trying to use asdf-direnv in docker. Following the README of asdf-direnv, I make this Dockerfile
FROM nvidia/cuda:10.2-devel-ubuntu18.04
RUN chsh -s /bin/bash
SHELL ["/bin/bash", "-ic", "-l"]
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update
# Python
RUN apt-get install -y make build-essential libssl-dev zlib1g-dev \
libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm \
libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev
# Utils
RUN apt-get install -y git
RUN apt-get clean
WORKDIR /venv
RUN git clone https://github.com/asdf-vm/asdf.git ~/.asdf --branch v0.8.1
RUN echo ". $HOME/.asdf/asdf.sh" >> ~/.bashrc
RUN echo ". $HOME/.asdf/completions/asdf.bash" >> ~/.bashrc
RUN asdf plugin add direnv
RUN asdf install direnv 2.28.0
RUN asdf local direnv 2.28.0
RUN echo "eval \"\$(asdf exec direnv hook bash)\"" >> ~/.bashrc
RUN echo "direnv() { asdf exec direnv \"\$#\"; }" >> ~/.bashrc
RUN mkdir -p ~/.config/direnv/
RUN echo "source \"\$(asdf direnv hook asdf)\"" >> ~/.config/direnv/direnvrc
RUN echo "export DIRENV_LOG_FORMAT=" >> ~/.config/direnv/direnvrc
RUN asdf plugin add python
RUN asdf install python 3.8.7
RUN asdf local python 3.8.7
RUN echo "use asdf" >> .envrc
RUN echo "layout python" >> .envrc
RUN direnv allow
RUN echo $(which python)
CMD ["/bin/bash"]
The issue is, the line RUN echo $(which python) works properly when I run but not when I build the image. I got
/root/.asdf/shims/python when building docker build . -t venv-gpu -f docker-gpu/Dockerfile
/venv/.direnv/python-3.8.7/bin/python when docker run --gpus all -it venv-gpu
How can I fixed this?
I am trying to convert a bash script to a Dockerfile since we are going the containerization route with AWS Batch
Basically I install CPLEX (an optimization library) and Anaconda, install some related packages, check if my environment it good to go, and then kick off a shell script to run the batch job.
Here is a snippet of my Dockerfile:
FROM amazonlinux:latest
# Download packages for container
RUN yum update -y
RUN yum -y install which unzip aws-cli \
RUN yum install -y tar.x86_64
RUN yum install gzip -y
RUN yum install ncompress -y
RUN yum -y install wget
RUN yum install -y nano
# Set working directory
WORKDIR /setup
#: Copy CPLEX installer binary and installation script.
COPY cplex_odee1210.linux-x86-64.bin /setup/
COPY cplex_installer_input.sh /setup/
#: Install CPLEX and update .bashrc
RUN chmod +x /setup/cplex_odee1210.linux-x86-64.bin
RUN chmod +x cplex_installer_input.sh
RUN ./cplex_installer_input.sh | bash cplex_odee1210.linux-x86-64.bin
RUN echo 'export PATH=$PATH:/opt/ibm/ILOG/CPLEX_Optimizer1210/cplex/bin/x86-64_linux' >>/root/.bashrc \
&& /bin/bash -c "source ~/.bashrc"
ENV PATH $PATH:/opt/ibm/ILOG/CPLEX_Optimizer1210/cplex/bin/x86-64_linux
#: Download Anaconda
COPY Anaconda3-2019.10-Linux-x86_64.sh /setup/
RUN bash Anaconda3-2019.10-Linux-x86_64.sh -b -p /home/ec2-user/anaconda3
RUN echo 'export PATH=$PATH:/home/ec2-user/anaconda3/bin' >>/root/.bashrc \
&& /bin/bash -c "source ~/.bashrc"
ENV PATH $PATH:/home/ec2-user/anaconda3/bin
RUN conda install pandas -y \
&& conda install numpy -y \
&& conda install ujson -y \
&& pip install docplex \
&& pip install boto3 \
&& pip install grpcio \
&& pip install grpcio-tools
RUN python3 -m docplex.mp.environment
ADD fetch_and_run.sh /usr/local/bin/fetch_and_run.sh
ENTRYPOINT ["/usr/local/bin/fetch_and_run.sh"]
From there, I kick off a bash script
#!/bin/bash
date
echo "Args: $#"
env
echo "script_path: $1"
echo "script_name: $2"
echo "path_prefix: $3"
echo "jobID: $AWS_BATCH_JOB_ID"
echo "jobQueue: $AWS_BATCH_JQ_NAME"
echo "computeEnvironment: $AWS_BATCH_CE_NAME"
echo "current directory: $(pwd)"
mkdir /tmp/scripts/
aws s3 cp $1 /tmp/scripts/$2
python3 /tmp/scripts/${#:2}
But for some reason, I keep getting
/tmp/tmp.hQlWYBEFs/batch-file-temp: line 20: python3: command not found
Do I need to change some PATH variables? Why isn't Docker picking up my Python 3 version?
The image needs to have python3 installed. Building images works off of files and programs that exist in the container. The python3 you have installed on your own system is not available.
I am trying to switch from Cmake to gradle. I want to configure gradle to work as follow
$ cd myapp && ls myapp
$ Dockerfile build.gradle src
$ gradle build
Build the docker image from docker file
start container
build the application
The docker image contains complete environment for my app.
FROM debian:stretch
RUN apt-get update -y && apt install -y git \
python3-dev libncurses5-dev libxml2-dev \
libedit-dev swig doxygen graphviz xz-utils ninja-build
RUN echo "deb http://ftp.de.debian.org/debian stretch main" >> /etc/apt/source.list
RUN apt-get update && apt-get install -y openjdk-8-jre openjdk-8-jdk
# Clang 8 as a compiler
RUN apt-get update && apt-get install -y \
xz-utils \
build-essential \
curl \
&& rm -rf /var/lib/apt/lists/* \
&& curl -SL http://releases.llvm.org/8.0.0/clang+llvm-8.0.0-x86_64-linux-gnu-ubuntu-18.04.tar.xz \
| tar -xJC . && \
mv clang+llvm-8.0.0-x86_64-linux-gnu-ubuntu-18.04 clang_8.0.0 && \
echo 'export PATH=/clang_8.0.0/bin:$PATH' >> ~/.bashrc && \
echo 'export LD_LIBRARY_PATH=/clang_8.0.0/lib:LD_LIBRARY_PATH' >> ~/.bashrc
#
RUN apt-get update
#install sdkman
RUN rm /bin/sh && ln -s /bin/bash /bin/sh
RUN apt-get -qq -y install curl wget unzip zip
RUN curl -s "https://get.sdkman.io" | bash
RUN source "$HOME/.sdkman/bin/sdkman-init.sh"
#install gradle
RUN yes | /bin/bash -l -c 'sdk install gradle 6.1'
PS: This is cpp project
You can build a docker image from Gradle tasks by using **com.bmuschko:gradle-docker-plugin:3.1.0 plugin**
buildscript {
repositories {
jcenter()
mavenCentral()
}
dependencies {
classpath 'com.bmuschko:gradle-docker-plugin:3.1.0'
}
}
apply plugin: 'com.bmuschko.docker-remote-api'
import com.bmuschko.gradle.docker.tasks.image.Dockerfile
import com.bmuschko.gradle.docker.tasks.image.DockerBuildImage
import com.bmuschko.gradle.docker.tasks.image.*
task buildImage(type: DockerBuildImage) {
group = ''
inputDir = file('.')
tag = 'image name:'+tag
}
read the documentation for more details https://bmuschko.github.io/gradle-docker-plugin/
Build an image from Gradle task - ./gradlew taskname
To start container and run the command inside it you can use CMD or ENTRYPOINT and specify the command in Dockerfile
CMD ["start.sh"]
in start.sh you can specify your command to be executed after running the container
Let me slightly clean up that Dockerfile first:
FROM debian:stretch
RUN echo "deb http://ftp.de.debian.org/debian stretch main" >> /etc/apt/source.list
RUN apt-get update -y && apt install -qq -y \
python3-dev libncurses5-dev libxml2-dev \
libedit-dev swig doxygen graphviz xz-utils ninja-build \
openjdk-8-jre openjdk-8-jdk \
xz-utils curl git build-essential wget unzip zip
# Clang 8 as a compiler
RUN curl -SL http://releases.llvm.org/8.0.0/clang+llvm-8.0.0-x86_64-linux-gnu-ubuntu-18.04.tar.xz \
| tar -xJC . && \
mv clang+llvm-8.0.0-x86_64-linux-gnu-ubuntu-18.04 clang_8.0.0 && \
echo 'export PATH=/clang_8.0.0/bin:$PATH' >> ~/.bashrc && \
echo 'export LD_LIBRARY_PATH=/clang_8.0.0/lib:LD_LIBRARY_PATH' >> ~/.bashrc
#install sdkman
RUN ln -fs /bin/bash /bin/sh
RUN curl -s "https://get.sdkman.io" | bash
RUN source "$HOME/.sdkman/bin/sdkman-init.sh"
RUN yes | /bin/bash -l -c 'sdk install gradle 6.1'
RUN mkdir /src /work
WORKDIR /src
ENTRYPOINT gradle build -p /src
The important bits are at the bottom: it creates a /src directory and executes gradle build there. All that remains for you is to make that directory available when you build.
Assuming you built the container once with docker build -t my-build-container ., you can run it as follows:
docker run -v $(pwd):/src my-build-container
Depending on your build system, this might pollute your source tree with various build artifacts owned by root. If so, consider switching to out-of-tree builds by changing the default working directory to /work instead. All build results will go to /work, and you can extract them from the container afterwards.
Add docker plugin first
buildscript {
dependencies {
classpath("se.transmode.gradle:gradle-docker:1.2")
}
}
create simple task like this in build.gradle file
task buildDocker(type: Docker, dependsOn: build) {
push = false
project.group = 'testProject'
project.archivesBaseName = jar.baseName
applicationName = jar.baseName
dockerfile = file('src/main/docker/Dockerfile')
doFirst {
copy {
from jar
into stageDir
}
}
}
So I have a Dockerfile, using which I create an image. The instruction in Dockerfile are:
#This is a docker file
FROM ubuntu:14.04
MAINTAINER amit
# Install python-pip
RUN apt-get update && apt-get install -y python-pip
# Install virtual-env
RUN mkdir ~/.virtualenvs
RUN pip install virtualenv
RUN pip install virtualenvwrapper
RUN touch ~/.bashrc
RUN echo "export WORKON_HOME=$HOME/.virtualenvs" >> ~/.bashrc
RUN echo "source /usr/local/bin/virtualenvwrapper.sh" >> ~/.bashrc
RUN /bin/bash -c "source /usr/local/bin/virtualenvwrapper.sh && mkvirtualenv be"
# INSTALL REQUIRED PACKAGES
RUN apt-get update && apt-get install -y \
xclip \
python-dev \
libffi-dev \
libpam0g-dev \
sqlite3 \
libsqlite3-dev \
subversion \
g++ \
libxslt1-dev \
libxml2-dev \
zlib1g-dev \
swig \
node \
git \
libssl-dev
# Expose port
EXPOSE 5000
# Get the source ideally one should do a get on source release
COPY src /src
WORKDIR /src
RUN touch installer.sh
RUN echo "#!/bin/bash" >> installer.sh
RUN echo "source `which virtualenvwrapper.sh`" >> installer.sh
RUN echo "workon be" >> installer.sh
RUN echo "./tools/install_dependencies" >> installer.sh
RUN echo "deactivate be" >> installer.sh
RUN chmod +x installer.sh
Now I build an image from this. I execute the installer.sh by running the docker container and everything works great.
But when some one pulls this very image from the repository , and runs the file installer.sh, there is an error stating that virtualenv be is not present and then one has to do "mkvirtualenv be" once again.
What is wrong here? Shouldn't the "virtualenv be" automatically be present in the container?