I have installed docker-compose using the command
sudo apt install docker-compose
It installed docker-compose version 1.8.0 and build unknown
I need the latest version of docker-compose or at least a version of 1.9.0
Can anyone please let me know what approach I should take to upgrade it or uninstall and re-install the latest version.
I have checked the docker website and can see that they are recommending this to install the latest version'
sudo curl -L https://github.com/docker/compose/releases/download/1.21.0/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose
But before that, I have to uninstall the present version, which can be done using the command
sudo rm /usr/local/bin/docker-compose
but this can be used only when the installation was done using curl. I am not sure if the installation was done by curl as I have used
sudo apt install docker-compose
Please let me know what should I do now to uninstall and re-install the docker-compose.
First, remove the old version:
If installed via apt-get
sudo apt-get remove docker-compose
If installed via curl
sudo rm /usr/local/bin/docker-compose
If installed via pip
pip uninstall docker-compose
Then find the newest version on the release page at GitHub or by curling the API and extracting the version from the response using grep or jq (thanks to dragon788, frbl, and Saber Hayati for these improvements):
# curl + grep
VERSION=$(curl --silent https://api.github.com/repos/docker/compose/releases/latest | grep -Po '"tag_name": "\K.*\d')
# curl + jq
VERSION=$(curl --silent https://api.github.com/repos/docker/compose/releases/latest | jq .name -r)
Finally, download to your favorite $PATH-accessible location and set permissions:
DESTINATION=/usr/local/bin/docker-compose
sudo curl -L https://github.com/docker/compose/releases/download/${VERSION}/docker-compose-$(uname -s)-$(uname -m) -o $DESTINATION
sudo chmod 755 $DESTINATION
The easiest way to have a permanent and sustainable solution for the Docker Compose installation and the way to upgrade it, is to just use the package manager pip with:
pip install docker-compose
I was searching for a good solution for the ugly "how to upgrade to the latest version number"-problem, which appeared after you´ve read the official docs - and just found it occasionally - just have a look at the docker-compose pip package - it should reflect (mostly) the current number of the latest released Docker Compose version.
A package manager is always the best solution if it comes to managing software installations! So you just abstract from handling the versions on your own.
If you tried sudo apt-get remove docker-compose and get E: Unable to locate package docker-compose, try this method :
This command must return a result, in order to check it is installed here :
ls -l /usr/local/bin/docker-compose
Remove the old version :
sudo rm -rf docker-compose
Download the last version (check official repo : docker/compose/releases) :
sudo curl -L "https://github.com/docker/compose/releases/download/1.24.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
(replace 1.24.0 if needed)
Finally, apply executable permissions to the binary:
sudo chmod +x /usr/local/bin/docker-compose
Check version :
docker-compose -v
If the above methods aren't working for you, then refer to this answer: https://stackoverflow.com/a/40554985
curl -L "https://github.com/docker/compose/releases/download/1.22.0/docker-compose-$(uname -s)-$(uname -m)" > ./docker-compose
sudo mv ./docker-compose /usr/bin/docker-compose
sudo chmod +x /usr/bin/docker-compose
Based on #eric-johnson's answer, I'm currently using this in a script:
#!/bin/bash
compose_version=$(curl https://api.github.com/repos/docker/compose/releases/latest | jq .name -r)
output='/usr/local/bin/docker-compose'
curl -L https://github.com/docker/compose/releases/download/$compose_version/docker-compose-$(uname -s)-$(uname -m) -o $output
chmod +x $output
echo $(docker-compose --version)
it grabs the latest version from the GitHub api.
Here is another oneliner to install the latest version of docker-compose using curl and sed.
curl -L "https://github.com/docker/compose/releases/download/`curl -fsSLI -o /dev/null -w %{url_effective} https://github.com/docker/compose/releases/latest | sed 's#.*tag/##g' && echo`/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose && chmod +x /usr/local/bin/docker-compose
Do it in three steps. (showing for apt-get installs)
Uninstall the last one. e.g. for apt-get installs
sudo apt-get remove docker-compose
Install the new one (https://docs.docker.com/compose/install/)
sudo curl -L "https://github.com/docker/compose/releases/download/1.29.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
and then
sudo chmod +x /usr/local/bin/docker-compose
Check your version
docker-compose --version
Simple Solution to update docker-compose
This will remove the existing binary of docker-compose and install a new version.
sudo cd /usr/local/bin && sudo rm -rf docker-compose
sudo sudo curl -SL https://github.com/docker/compose/releases/download/v2.2.3/docker-compose-linux-x86_64 -o /usr/local/bin/docker-compose
sudo chmod +x docker-compose
for the latest version visit https://github.com/docker/compose/releases and replace the latest one with v2.1.1
I was trying to install docker-compose on "Ubuntu 16.04.5 LTS" but after installing it like this:
sudo curl -L "https://github.com/docker/compose/releases/download/1.26.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
I was getting:
-bash: /usr/local/bin/docker-compose: Permission denied
and while I was using it with sudo I was getting:
sudo: docker-compose: command not found
So here's the steps that I took and solved my problem:
sudo curl -L "https://github.com/docker/compose/releases/download/1.26.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo ln -sf /usr/local/bin/docker-compose /usr/bin/docker-compose
sudo chmod +x /usr/bin/docker-compose
use this from command line: sudo curl -L "https://github.com/docker/compose/releases/download/1.22.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
Write down the latest release version
Apply executable permissions to the binary:
sudo chmod +x /usr/local/bin/docker-compose
Then test version:
$ docker-compose --version
If you installed with pip, to upgrade you can just use:
pip install --upgrade docker-compose
or as Mariyo states with pip3 explicitly:
pip3 install --upgrade docker-compose
Using latest flag in url will redirect you to the latest release of the repo
As OS name is lower case in github's filename, you should convert uname -s to lower case using sed -e 's/\(.*\)/\L\1/'.
sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s|sed -e 's/\(.*\)/\L\1/')-$(uname -m)" -o /usr/local/bin/docker-compose && sudo chmod +x /usr/local/bin/docker-compose
On mac (also working on ubuntu):
sudo curl -L "https://github.com/docker/compose/releases/download/<release-version>/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
NOTE: write the as here:
https://github.com/docker/compose/releases
Use,
$ sudo curl -L "https://github.com/docker/compose/releases/download/1.24.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose && sudo chmod +x /usr/local/bin/docker-compose
$ docker-compose -v
Docker Engine and Docker Compose Plugin
Since Microsoft took over Docker they worked on porting docker-compose to their Docker Engine CLI plugins. For future support and updates I would recommend using docker compose plugin (Notice the missing dash) which can be install via the docker-compose-plugin package. The following instructions assume that you are using Ubuntu as Distro or any Distro thats using apt as package manager.
Installation Preparations
Update your mirrors:
sudo apt-get update
Make sure the following packages are installed:
sudo apt-get install \
ca-certificates \
curl \
gnupg \
lsb-release
After that add the official Docker GPG Key:
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
And finally add the the stable repository:
echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
Also make sure Docker Engine and other needed dependencies are installed:
sudo apt-get install docker-ce docker-ce-cli containerd.io
Installation of docker compose plugin
sudo apt-get install docker-compose-plugin
Any future updates of the plugin are easily applied via apt.
For further reference take a look at the official installation instructions of Docker Engine and Docker Compose.
After a lot of looking at ways to perform this I ended up using jq, and hopefully I can expand it to handle other repos beyond Docker-Compose without too much work.
# If you have jq installed this will automatically find the latest release binary for your architecture and download it
curl --silent "https://api.github.com/repos/docker/compose/releases/latest" | jq --arg PLATFORM_ARCH "$(echo `uname -s`-`uname -m`)" -r '.assets[] | select(.name | endswith($PLATFORM_ARCH)).browser_download_url' | xargs sudo curl -L -o /usr/local/bin/docker-compose --url
On ubuntu desktop 18.04.2, I have the 'local' removed from the path when using the curl command to install the package and it works for me. See above answer by Kshitij.
In my case, using Windows + WSL2 with Ubuntu 20.04, was necessary only this:
sudo apt update
and then:
sudo apt upgrade
Centos/RHEL
Follow my answer below if you're using Centos7 with an x86-64 architecture. This answer is also available in my github.
Stop Your Docker Containers
I noticed other answers did not talk about stopping your docker containers/images instances before attempting to upgrade gracefully. Assumptions are inevitable but can be costly. Onward we go!
Options to update Docker-Compose
There are 2 options to upgrade docker-compose if you first downloaded and installed docker-compose using the Curl command.
Using Curl, jq package, and Github's direct URL to the docker-compose repository.
Using Curl, Sed, and Github's direct URL to the docker-compose repository.
Note: some of the commands below require "sudo" privileges.
Demonstration
The script below was saved to a file called "update_docker_compose.sh". You need to give this file executable permissions.
Like so:
chmod +x update_docker_compose.sh
"docker_docker_compose.sh" file content:
#!/bin/bash
# author: fullarray (stackoverflow user)
# Contribution shared on: stackoverflow.com
# Contribution also available on: github.com
# date: 06112022
# Stop current docker container running
docker stop containerID
# Remove current docker network running
docker rm containerID
# Remove image of target application(s)
docker image rm imageID
# Delete either dangling (unatagged images) docker containers or images or network
docker system prune -f
# This step depends on the jq package.
# Uncomment jq package installation command below if using Centos7 x86-64.
# sudo yum install jq
# Declare variable to get latest version of docker-compose from github repository
compose_version=$(curl https://api.github.com/repos/docker/compose/releases/latest | jq .name -r)
# Declare variable to target installation directory
target_install_dir='/usr/local/bin/docker-compose'
# Get OS and build (assumes Linux Centos7 and x86_64)
get_local_os_build=$(uname -s)-$(uname -m)
# Execute curl command to carry download and installation operation
curl -L https://github.com/docker/compose/releases/download/$compose_version/docker-compose-$get_local_os_build -o $target_install_dir
# Use chmod to modify permissions to target installation directory (to make it executable)
chmod +x $target_install_dir
# Print docker-compose version to terminal to verify upgrade
$(docker-compose --version)
Edit the script with variables specific to your environment
The script above has a few variables you need to edit with values specific to your docker environment. For instance, you need to replace container ID and image ID with the values that the following commands output.
docker ps
and
docker images output
Once you finalize creating the file (including the edits). Switch to the directory that contains the file. For example, if you created the file in /home/username/script/update_docker_compose.sh
cd /home/username/script
Last, run the script by executing the following
./update_docker_compose.sh
Option 2
Create a script file name "update_docker_compose.sh"
Edit the file and add the following content:
#!/bin/bash
# author: fullarray (stackoverflow user)
# Contribution shared on: stackoverflow.com
# Contribution also available on: github.com
# date: 06112022
# Stop current docker container running
docker stop containerID
# Remove current docker network running
docker rm containerID
# Remove image of target application(s)
docker image rm imageID
# Delete either dangling (unatagged images) docker containers or images or network
docker system prune -f
# Declare variable to target installation directory
target_install_dir='/usr/local/bin/docker-compose'
# Get OS and build (assumes Linux Centos7 and x86_64)
get_local_os_build=$(uname -s)-$(uname -m)
# Execute curl and sed command to carry out download and installation operation
# compose_latest_version=$(curl -L "https://github.com/docker/compose/releases/download/`curl -fsSLI -o /dev/null -w %{url_effective} https://github.com/docker/compose/releases/latest | sed 's#.*tag/##g' && echo`/docker-compose-$get_local_os_build") -o $target_install_dir
# Use chmod to modify permissions to target installation directory (to make it executable)
chmod +x $target_install_dir
# Print docker-compose version to terminal to verify upgrade
$(docker-compose --version)
Edit the script with variables specific to your environment
The script above also has a few variables you need to edit with values specific to your docker environment. For instance, you need to replace container ID and image ID with the values that the following commands output.
docker ps
and
docker images output
Once you finalize creating the file (including the edits). Switch to the directory that contains the file. For example, if you created the file in /home/username/script/update_docker_compose.sh
cd /home/username/script
Last, run the script by executing the following
./update_docker_compose.sh
This is the method of installing docker compose version 2.12.x
Update debian package manager
# apt-get update
# apt-get install docker-compose-plugin
Then install the plugin manualy
DOCKER_CONFIG=${DOCKER_CONFIG:-$HOME/.docker}
mkdir -p $DOCKER_CONFIG/cli-plugins
curl -SL https://github.com/docker/compose/releases/download/v2.12.2/docker-compose-linux-x86_64 -o $DOCKER_CONFIG/cli-plugins/docker-compose
Give permisson of execution of file
chmod +x $DOCKER_CONFIG/cli-plugins/docker-compose
Last test the installation
docker compose version
// Docker Composer Version v2.12.2
If you have homebrew you can also install via brew
$ brew install docker-compose
This is a good way to install on a Mac OS system
Most of these solutions are outdated or make you install old version.
To install the latest
sudo apt install jq
DOCKER_COMPOSE_VERSION=$(curl --silent https://api.github.com/repos/docker/compose/releases/latest | jq .name -r)
sudo curl -L "https://github.com/docker/compose/releases/download/$DOCKER_COMPOSE_VERSION/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
Well, my case was pretty weird. I am using wsl2, and Docker Desktop (Windows 11). I stop getting this error after rename the folder "docker" to "config-dev-server" and update de Dockerfile like this this:
COPY ./docker/apache/apache2.conf /etc/apache2/apache2.conf
to
COPY ./config-dev-server/apache/apache2.conf /etc/apache2/apache2.conf
With a newer Docker Desktop for Mac 3.3.0, you don't need to install Docker Compose as a seperate package. Docker Compose comes as a first class citizen installed with Docker by default. Check out the below CLI:
docker compose version
Docker Compose version 2.0.0-beta.1%
Related
I am trying to download java using yum on centOs which I specified in Dockerfile.
After pulling centOs image the run crushed and throw this error!?
also to mention that my server instance is AWS EC2!
Step 2/9 : RUN yum install java -y
---> Running in 39fc233aa965
CentOS Linux 8 - AppStream 184 B/s | 38 B 00:00
Error: Failed to download metadata for repo 'appstream': Cannot prepare internal mirrorlist: No URLs in mirrorlist
The command '/bin/sh -c yum install java -y' returned a non-zero code: 1
Try editing your dockerfile
FROM centos
RUN cd /etc/yum.repos.d/
RUN sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-*
RUN sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-*
RUN yum -y install java
CMD /bin/bash
Refer to this code
failed-metadata-repo-appstream-centos-8
If you don't already have it, you'll need the gpg keys:
wget 'http://mirror.centos.org/centos/8-stream/BaseOS/x86_64/os/Packages/centos-gpg-keys-8-3.el8.noarch.rpm'
sudo rpm -i 'centos-gpg-keys-8-3.el8.noarch.rpm'
Then it's as simple as transitioning like so:
dnf --disablerepo '*' --enablerepo=extras swap centos-linux-repos centos-stream-repos
Don't worry -- it doesn't remove any repos, it simply temporarily ignores all of yours, and downloads information regarding the new mirrors.
You may at this point want to actually upgrade your packages:
sudo dnf distro-sync
You'll now be able to use "yum" as usual.
Go to /etc/yum.repos.d/
cd /etc/yum.repos.d/
Run
sudo sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-*
sudo sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-*
sudo yum update -y
Then do what you want
I tried to use CentOS 8 with wsl and got the same error. Steps to fix the problem (as root):
# sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-*
# sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-*
# dnf distro-sync
# dnf -y install java
The top voted answer did not work for me (by #Hashbrown). The answer with Dockerfile was not for my case either.
Use these commands to update centOS8.0 on AWS EC2:
sudo sed -i -e "s|mirrorlist=|#mirrorlist=|g" \
-e "s|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g" \
/etc/yum.repos.d/CentOS-*
CentOS 8 reached EOL on 2021-12-31 (announcement).
Therefore, the URLs to the mirrors don't work anymore. Instead of using sed to modify the URLs to point to the archived mirrors, CentOS officially recommends to convert from CentOS Linux 8 to CentOS Stream 8 via:
dnf --disablerepo '*' --enablerepo extras swap centos-linux-repos centos-stream-repos
dnf distro-sync
After that, dnf/yum will work again.
Try this
FROM centos
RUN cd /etc/yum.repos.d/
RUN sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-*
RUN sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-*
RUN yum -y install java
CMD /bin/bash
Please follow the below-mentioned steps:
Go to the /etc/yum.repos.d/ directory.
cd /etc/yum.repos.d/
Run the below commands to hash the mirror-list in all yum.repos.d files then replace the existed Baseurl with the vault.centos.org
sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-*
sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-*
Then run yum update or install any package you want
yum update -y
Update your docker file with below. It should work.
RUN sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-*
RUN sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-*
RUN yum update -y
Go to /etc/yum.repos.d/ directory. Open .repo file and manually edit mirrorlist from $releasever to 8-stream.
For example : /etc/yum.repos.d/CentOS-Linux-BaseOS.repo
open file in vi
sudo vi /etc/yum.repos.d/CentOS-Linux-BaseOS.repo
comment mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=BaseOS&infra=$infra
#mirrorlist=http://......
within vi, copy paste mirrorlist=http://...... line
yy and p
uncomment and edit the copied line by replacing $releasever to 8-stream
mirrorlist=http://mirrorlist.centos.org/?release=8-stream&arch=$basearch&repo=BaseOS&infra=$infra
save and exit vi
:wq
Repeat above 5-steps for other .repo files.
I have a Rancher Deskop(dockerd) on M1 MacOS and when I am trying to build below dockerfile I am getting an error such as below. Here is the command how I am trying to build the image docker build -t te-grafana-dashboards-toolchain --no-cache .
I tried to change the platforms but nonae of them worked for me. I am a bit lost about this platform issue for M1 but any help will be appreciated, What I am doing wrong? What might the root cause of this?
Removing intermediate container 70af516d5d6b
---> a69229847153
Step 5/6 : RUN GO111MODULE="on" go get github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb; ln -s $(go env GOPATH)/bin/jb /usr/bin/jb
---> Running in 13545862fffe
qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory
qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory
Removing intermediate container 13545862fffe
Dockerfile
FROM --platform=linux/amd64 ubuntu:focal
RUN apt update; apt install -y curl jq build-essential python3.8 python3-pip docker-compose jsonnet bison mercurial
RUN ln -s /usr/bin/python3.8 /usr/bin/python
RUN curl -OL https://golang.org/dl/go1.17.linux-amd64.tar.gz; mkdir /etc/golang; tar -xvzf go1.17.linux-amd64.tar.gz -C /etc/golang; ln -s /etc/golang/go/bin/go /usr/bin/go; rm -f go1.17.linux-amd64.tar.gz
RUN GO111MODULE="on" go get github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb; ln -s $(go env GOPATH)/bin/jb /usr/bin/jb
WORKDIR /workspace
Incidentally, in case it's helpful to another who lands here, I have the same issue on an M1 Max MacBook Pro laptop attempting to do a docker build from a company repo that should be a pretty well traveled path, but I might be the only one (it's a small company) that has an ARM64 M1 "Apple Silicon" Mac. However I found the solution (well, a solution) to my situation was exactly the opposite of the solution to the OP's, and that was to add --platform=linux/amd64 to the FROM line of the docker file.
Otherwise it was using an ARM64 image to start from without me being the wiser but then later in the Dockerfile the build attempts to install and execute code compiled for x86_64. Starting the build process by requesting the base image be linux/amd64 ends up with then the base image having /lib64/ld-linux-x86-64.so.2. This probably means everything is being emulated as x86_64 on the ARM64 CPU via qemu-x86_64 and so if you have the option to start from an ARM64 image and can compile within the container during build time any software you can't install as ARM64 binaries, it'll probably go faster when you later run the container on the M1 based Mac. I'm not able to try that myself just yet for this case.
Modifying Dockerfile seems to be the most popular answer but you can also set the DOCKER_DEFAULT_PLATFORM environment variable to linux/amd64.
export DOCKER_DEFAULT_PLATFORM=linux/amd64
The cause seems to reside in the AArch64 image.
this resolved my issue.
FROM ubuntu:focal
RUN apt update; apt install -y curl jq build-essential python3.8 python3-pip docker-compose jsonnet bison mercurial
RUN ln -s /usr/bin/python3.8 /usr/bin/python
RUN curl -OL https://golang.org/dl/go1.17.linux-arm64.tar.gz; mkdir /etc/golang; tar -xvzf go1.17.linux-arm64.tar.gz -C /etc/golang; ln -s /etc/golang/go/bin/go /usr/bin/go; rm -f go1.17.linux-arm64.tar.gz
RUN GO111MODULE="on" go get github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb#latest; ln -s /root/go/bin/jb /usr/bin/jb
WORKDIR /workspace
Instead of editing the Dockerfile, as suggested in this answer, or setting an environment variable, as suggested in this answer, I prefer to pass the platform as an argument to the docker build command, with the --platform flag. The command used by the OP would then be:
docker build --platform linux/amd64 -t te-grafana-dashboards-toolchain --no-cache .
Passing following flag to C preprocessor as CPPFLAGS solved similar issue in my M1
-DPNG_ARM_NEON_OPT=0
Pass the value as env var with key CPPFLAGS to relevant service.
Provided the base image includes the target architecture, another option that might work in your case is using Docker's built-in TARGETARCH build arg. This works for me on macOS M1.
FROM ubuntu:focal
ARG TARGETARCH
RUN apt update; apt install -y curl jq build-essential python3.8 python3-pip docker-compose jsonnet bison mercurial
RUN ln -s /usr/bin/python3.8 /usr/bin/python
RUN curl -OL https://golang.org/dl/go1.17.linux-${TARGETARCH}.tar.gz; mkdir /etc/golang; tar -xvzf go1.17.linux-${TARGETARCH}.tar.gz -C /etc/golang; ln -s /etc/golang/go/bin/go /usr/bin/go; rm -f go1.17.linux-${TARGETARCH}.tar.gz
RUN GO111MODULE="on" go get github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb; ln -s $(go env GOPATH)/bin/jb /usr/bin/jb
WORKDIR /workspace
I want to use Kubeflow to check it out and see if it fits my projects. I want to deploy it locally as a development server so I can check it out, but I have Windows on my computer and Kubeflow only works on Linux. I'm not allowed to dual boot this computer, I could install a virtual machine, but I thought it would be easier to use docker, and oh boy was I wrong. So, the problem is, I want to install Kubernetes in a docker container, right now this is the Dockerfile I've written:
# Docker file with local deployment of Kubeflow
FROM ubuntu:18.04
ENV USER=Joao
ENV PASSWORD=Password
ENV WK_DIR=/home/${USER}
# Setup Ubuntu
RUN apt-get update -y
RUN apt-get install -y conntrack sudo wget
RUN useradd -rm -d /home/${USER} -s /bin/bash -g root -G sudo -u 1001 -p ${PASSWORD} ${USER}
WORKDIR ${WK_DIR}
# Installing Docker CE
RUN apt-get install -y apt-transport-https ca-certificates curl software-properties-common
RUN curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
RUN add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"
RUN apt-get update -y
RUN apt-get install -y docker-ce docker-ce-cli containerd.io
# Installing Kubectl
RUN curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.15.0/bin/linux/amd64/kubectl
RUN chmod +x ./kubectl
RUN mv ./kubectl /usr/local/bin/kubectl
# Installing Minikube
RUN curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
RUN install minikube-linux-amd64 /usr/local/bin/minikube
ENV PATH="${PATH}:${WK_DIR}"
COPY start.sh start.sh
CMD sh start.sh
With this, just to make the deployment easier, I also have a docker-compose.yaml that looks like this:
services:
kf-local:
build: .
volumes:
- path/to/folder:/usr/kubeflow
privileged: true
And start.sh looks like this:
service docker start
minikube start \
--extra-config=apiserver.service-account-issuer=api \
--extra-config=apiserver.service-account-signing-key-file=/var/lib/minikube/certs/apiserver.key \
--extra-config=apiserver.service-account-api-audiences=api \
--driver=docker
The problem is, whenever I try running this I get the error:
X Exiting due to DRV_AS_ROOT: The "docker" driver should not be used with root privileges.
I've tried creating a user and running it from there also but then I'm not being able to run sudo, any idea how I could install Kubernetes on a Docker container?
As you thought you are right in case of using VM and that be easy to test it out.
Instead of setting up Kubernetes on docker you can use Linux base container for development testing.
There is linux container available name as LXC container. Docker is kind of application container while in simple words LXC is like VM for local development testing. you can install the stuff into rather than docker setting up application inside image.
read some details about lxc : https://medium.com/#harsh.manvar111/lxc-vs-docker-lxc-101-bd49db95933a
you can also run it on windows and try it out at : https://linuxcontainers.org/
If you have read the documentation of Kubeflow there is also one option multipass
Multipass creates a Linux virtual machine on Windows, Mac or Linux
systems. The VM contains a complete Ubuntu operating system which can
then be used to deploy Kubernetes and Kubeflow.
Learn more about Multipass : https://multipass.run/#install
Insufficient user permissions on the docker groups and minikube directory cause this error ("X Exiting due to DRV_AS_ROOT: The "docker" driver should not be used with root privileges.").
You can fix that error by adding your user to the docker group and setting permissions to the minikube profile directory (change the $USER with your username in the two commands below):
sudo usermod -aG docker $USER && newgrp docker
sudo chown -R $USER $HOME/.minikube; chmod -R u+wrx $HOME/.minikube
I am trying to set up whole docker eco system in ubuntu linux running in virtualbox. I succeed in installing docker engine. But I cannot install docker compose and docker machine. Below are the steps I followed to install docker machine.
$ base=https://github.com/docker/machine/releases/download/v0.14.0 &&
curl -L $base/docker-machine-$(uname -s)-$(uname -m) >/tmp/docker- machine &&
sudo install /tmp/docker-machine /usr/local/bin/docker-machine
I am getting below error
/usr/local/bin/docker-machine: line 1: Not: command not found
While running the command docker-machine --version
first uninstall older versions first
sudo apt-get remove docker docker-engine docker.io
Update the apt package index:
$ sudo apt-get update
Add Docker’s official GPG key:
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
this 4 lines are one single command. copy paste it.
sudo apt-key fingerprint 0EBFCD88
pub 4096R/0EBFCD88 2017-02-22
Key fingerprint = 9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88
uid Docker Release (CE deb) <docker#docker.com>
sub 4096R/F273FCD8 2017-02-22
set up the stable repository. You always need the stable repository,
sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"
Update the apt package index one more time.
sudo apt-get update
Install the latest version of Docker CE
sudo apt-get install docker-ce
test if it is installed
docker --version
INSTALL DOCKER COMPOSE
Run this command to download the current stable release of Docker Compose:
sudo curl -L "https://github.com/docker/compose/releases/download/1.24.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
Apply executable permissions to the binary:
sudo chmod +x /usr/local/bin/docker-compose
test it
docker-compose --version
I'm looking to use Docker compose with v2 docker-compose.yml syntax.
When I'm following the documentation (https://store.docker.com/editions/community/docker-ce-server-ubuntu?tab=description) old docker and docker-compose version has installed :
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
$ sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"
$ sudo apt-get update
$ sudo apt install docker docker-compose
.../...
$ dpkg -l |grep docker
ii docker 1.5-1 amd64 System tray for KDE3/GNOME2 docklet applications
rc docker-ce 17.03.1~ce-0~ubuntu-yakkety amd64 Docker: the open-source application container engine
ii docker-compose 1.5.2-1 all Punctual, lightweight development environments using Docker
ii docker.io 1.12.6-0ubuntu1~16.10.1 amd64 Linux container runtime
ii python-docker 1.8.0-0ubuntu1 all Python wrapper to access docker.io's control socket
ii python-dockerpty 0.4.1-1 all Pseudo-tty handler for docker Python client (Python 2.x)
In order to use version 2 docker compose file format I need more recent version.
Documentation page of Docker Compose : https://github.com/docker/docker.github.io/blob/master/compose/compose-file/compose-versioning.md
Version 2 files are supported by Compose 1.6.0+ and require a Docker
Engine of version 1.10.0+.
How do I get recent version of docker engine & docker compose ?
I googled this many times today without fine any "simple" solution.
My system : Ubuntu 16.10
Thanks for your help,
David
The easiest way to install Docker is to run:
curl -sSL https://get.docker.com/ | sh
You will probably need to run apt-get remove docker docker-compose first.
To install docker-compose, I normally grab the latest release from the docker-compose Github project e.g:
curl -L https://github.com/docker/compose/releases/download/1.12.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose