Different docker image being created from command line and JetBrains WebStorm - docker

I am creating a docker image. When I create it from WebStorm it creates a different image ( that doesn't run most importantly ) than when I create it with docker build.
Is this something that WebStorm does that I am unaware of given both methods are using the exact same files and Dockerfile yet are creating different images?
FROM cm2network/steamcmd:root
RUN apt update \
&& apt install -y curl \
&& apt install -y nodejs -y npm
WORKDIR /home/steam/steamcmd
VOLUME /home/steam/steamcmd
COPY . .
RUN npm install
USER steam
EXPOSE 3000
CMD ["node", "./src/index.js"]
The error says that the vdf package ( first one required ) isn't installed.
Log from terminal when running docker build -t <tag> .
[+] Building 12.1s (10/10) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 310B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 34B 0.0s
=> [internal] load metadata for docker.io/cm2network/steamcmd:root 0.0s
=> [1/5] FROM docker.io/cm2network/steamcmd:root 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 7.20kB 0.0s
=> CACHED [2/5] RUN apt update && apt install -y curl && apt install -y nodejs -y npm 0.0s
=> CACHED [3/5] WORKDIR /home/steam/steamcmd 0.0s
=> [4/5] COPY . . 0.1s
=> [5/5] RUN npm install 9.8s
=> exporting to image 2.1s
=> => exporting layers 2.1s
=> => writing image sha256:b1c36641fc4d49cff4d38092879d556c6257a96cc55b4401827cac7c5f72c5c9 0.0s
=> => naming to docker.io/library/steamcmd-terminal 0.0s
Log from WebStorm when using the native docker config:
Deploying 'steamcmd-webstorm Dockerfile: Dockerfile'…
Building image…
Preparing build context archive…
[==================================================>]50/50 files
Done
Sending build context to Docker daemon…
[==================================================>] 18.14kB
Done
Step 1/9 : FROM cm2network/steamcmd:root
---> d490902b909c
Step 2/9 : RUN apt update && apt install -y curl && apt install -y nodejs -y npm
---> Running in 56188b34650b
<Taken out the entire logs here as it's massive but nothing exciting seems to be going on here. Just installing Node and npm>
Step 3/9 : WORKDIR /home/steam/steamcmd
---> Running in 5c9c276586ce
Removing intermediate container 5c9c276586ce
---> c93a98ea80fe
Step 4/9 : VOLUME /home/steam/steamcmd
---> Running in e608a8c467f4
Removing intermediate container e608a8c467f4
---> 03f450c267b2
Step 5/9 : COPY . .
---> bcd9ce25e8a5
Step 6/9 : RUN npm install
---> Running in 16a46251730b
added 93 packages, and audited 94 packages in 8s
38 packages are looking for funding
run `npm fund` for details
found 0 vulnerabilities
Removing intermediate container 16a46251730b
---> 49bac07990ea
Step 7/9 : USER steam
---> Running in a50475541075
Removing intermediate container a50475541075
---> 9f8b28506b7f
Step 8/9 : EXPOSE 3000
---> Running in 17e53097f7b4
Removing intermediate container 17e53097f7b4
---> 11788721f0c2
Step 9/9 : CMD ["node", "./src/index.js"]
---> Running in 99d37cb33066
Removing intermediate container 99d37cb33066
---> 95ce7c9c21b7
Successfully built 95ce7c9c21b7
Successfully tagged steamcmd-webstorm:latest
'steamcmd-webstorm Dockerfile: Dockerfile' has been deployed successfully.

Related

dockerfile cannot build: CONDA env create

Hi there I'm new to Docker and Dockerfiles in general,
However, I will need to create one in order to load an application on a server using WDL. With that said, there are few important aspects of this Dockerfile:
requires to create a Conda environment
in there I have to install Snakemake (through Mamba)
finally, I will need to git clone a repository and follow the steps to generate an executable for the application, later invoked by Snakemake
Luckily, it seems most of the pieces are already on dockerhub; correct if I'm wrong based on the script (see below)
# getting ubuntu base image & anaconda3 loaded
2 FROM ubuntu:latest
3 FROM continuumio/anaconda3:2021.05
4 FROM condaforge/mambaforge:latest
5 FROM snakemake/snakemake:stable
6
7 FROM node:alpine
8 RUN apk add --no-cache git
9 RUN apk add --no-cache openssh
10
11 MAINTAINER Name <email>
12
13 WORKDIR /home/xxx/Desktop/Pangenie
14
15 ## ACTUAL PanGenIe INSTALLATION
16 RUN git clone https://github.com/eblerjana/pangenie.git /home/xxx/Desktop/Pangenie
17 # create the environment
18 RUN conda env create -f environment.yml
19 # build the executable
20 RUN conda activate pangenie
21 RUN mkdir build; cd build; cmake .. ; make
First, I think that loading also Mamba and Snakemake would allow me to simply launch the application, as the tools are already set-up by the Dockerfile. Then, I ideally would like to build from the repository the executable, still I get an error at line 18 when I try to create a Conda environment, this is what I get:
[+] Building 1.7s (10/10) FINISHED
[internal] load build definition from Dockerfile
0.1s => => transferring dockerfile: 708B 0.1s => [internal] load .dockerignore 0.1s => => transferring context: 2B 0.1s => [internal] load metadata for docker.io/library/node:alpine 1.4s => [auth] library/node:pull token for registry-1.docker.io 0.0s => [stage-4 1/6] FROM docker.io/library/node:alpine#sha256:1a04e2ec39cc0c3a9657c1d6f8291ea2f5ccadf6ef4521dec946e522833e87ea
0.0s => CACHED [stage-4 2/6] RUN apk add --no-cache git 0.0s => CACHED [stage-4 3/6] RUN apk add --no-cache openssh 0.0s => CACHED [stage-4 4/6] WORKDIR /home/mat/Desktop/Pangenie 0.0s => CACHED [stage-4 5/6] RUN git clone https://github.com/eblerjana/pangenie.git /home/mat/Desktop/Pangenie
0.0s => ERROR [stage-4 6/6] RUN conda env create -f environment.yml 0.1s
[stage-4 6/6] RUN conda env create -f environment.yml:
#10 0.125 /bin/sh: conda: not found executor failed running [/bin/sh -c conda env create -f environment.yml]: exit code: 127
Now, I'm not really experienced as I said, and I spent some time looking for a solution and tried different things, but nothing worked out... if anyone has an idea or even suggesions on how to fix this Dockerfile, please let me know.
Thanks in advance!

Error when trying to build my golang application in docker while using the mysql driver

I have a simple application that uses github.com/go-sql-driver/mysql to connect to a MySQL database and execute simple queries. This all works fine on my local machine, however when I try to build it using docker build I get the following output:
[+] Building 4.1s (9/10)
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 104B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/golang:onbuild 1.3s
=> [auth] library/golang:pull token for registry-1.docker.io 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 5.63kB 0.0s
=> CACHED [1/2] FROM docker.io/library/golang:onbuild#sha256:c0ec19d49014d604e4f62266afd490016b11ceec103f0b7ef44 0.0s
=> [2/2] COPY . /go/src/app 0.1s
=> [3/2] RUN go-wrapper download 2.0s
=> ERROR [4/2] RUN go-wrapper install 0.6s
------
> [4/2] RUN go-wrapper install:
#8 0.465 + exec go install -v
#8 0.535 github.com/joho/godotenv
#8 0.536 github.com/go-sql-driver/mysql
#8 0.581 # github.com/go-sql-driver/mysql
#8 0.581 ../github.com/go-sql-driver/mysql/driver.go:88: undefined: driver.Connector
#8 0.581 ../github.com/go-sql-driver/mysql/driver.go:99: undefined: driver.Connector
#8 0.581 ../github.com/go-sql-driver/mysql/nulltime.go:36: undefined: sql.NullTime
------
executor failed running [/bin/sh -c go-wrapper install]: exit code: 2
My go version is up to date and I am using the following dockerfile:
FROM golang:onbuild
To my knowledge this should go get all the packages it requires. I've also tried it this way:
FROM golang:onbuild
RUN go get "github.com/go-sql-driver/mysql"
This had the same output.
Note that in my code I import the package like this:
import _ "github.com/go-sql-driver/mysql"
I also use other packages from github, these seem to work fine.
The Docker community has generally been steering away from the Dockerfile ONBUILD directive, since it makes it very confusing what will actually happen in derived images (see the various comments around "is that really the entire Dockerfile?"). If you search Docker Hub for the golang:onbuild image you'll discover that this is Go 1.7 or 1.8; Go modules were introduced in Go 1.11.
You'll need to update to a newer base image, and that means writing out the Dockerfile steps by hand. For a typical Go application this would look like
FROM golang:1.18 AS build
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY ./ ./
RUN go build -o myapp .
FROM ubuntu:20.04
COPY --from=build /app/myapp /usr/local/bin
CMD ["myapp"]
(In the final stage you may need to RUN apt-get update && apt-get install ... a MySQL client library or other tools.)

WORKDIR $HOME cannot normalize nothing

I'm new to docker. I've a very simple Dockerfile but when I try to build an image from this Dockerfile, i get "cannot normalize nothing" error.
FROM alpine:latest
RUN apk add vim
CMD ["bash"]
WORKDIR $HOME
For building the image, I use the following command:
$ docker build -t aamirglb/test:1.0 .
I used Play With Docker (PWD) to test this script and here is the output.
pwd output
If I comment out WORKDIR $HOME line, the build is successful.
Any help in understanding this error will be highly appreciated.
Thanks in advance.
For those that can't reproduce, this error does not happen with buildkit:
$ docker build -t test-workdir -f df.workdir-empty .
[+] Building 0.2s (6/6) FINISHED
=> [internal] load build definition from df.workdir-empty 0.0s
=> => transferring dockerfile: 43B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 34B 0.0s
=> [internal] load metadata for docker.io/library/alpine:latest 0.0s
=> [1/3] FROM docker.io/library/alpine:latest 0.0s
=> CACHED [2/3] RUN apk add vim 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:cadafef9a77c95b1b32499bcc1beba016ff0a7071710a1c37eb4e5f32e5d1c94 0.0s
=> => naming to docker.io/library/test-workdir 0.0s
But if you build with the classic builder, the error does appear:
$ DOCKER_BUILDKIT=0 docker build -t test-workdir -f df.workdir-empty .
Sending build context to Docker daemon 23.04kB
Step 1/4 : FROM alpine:latest
---> 14119a10abf4
Step 2/4 : RUN apk add vim
---> Running in a286e7f3107a
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/x86_64/APKINDEX.tar.gz
(1/5) Installing xxd (8.2.4708-r0)
(2/5) Installing lua5.3-libs (5.3.6-r0)
(3/5) Installing ncurses-terminfo-base (6.2_p20210612-r0)
(4/5) Installing ncurses-libs (6.2_p20210612-r0)
(5/5) Installing vim (8.2.4708-r0)
Executing busybox-1.33.1-r3.trigger
OK: 26 MiB in 19 packages
Removing intermediate container a286e7f3107a
---> acdd6e1963db
Step 3/4 : CMD ["bash"]
---> Running in 6deb306db96a
Removing intermediate container 6deb306db96a
---> 960f0de2f376
Step 4/4 : WORKDIR $HOME
cannot normalize nothing
And before you get your hopes up, workdir in the buildkit example is not set to the user's home directory:
$ docker inspect test-workdir:latest --format '{{.Config.WorkingDir}}'
/
So first, what does cannot normalize nothing mean? Normalizing is the process of taking a path like /.//some/../path and turning that into /path, cleaning the string of unnecessary content and converting it into a uniform path. That process doesn't work on nothing, it needs a string or value, or at least the classic builder does (buildkit seems to have a sensible default).
Why doesn't $HOME default to /root? Because that variable was never set in docker. The only environment variable defined in there is a path:
$ docker inspect test-workdir:latest --format '{{.Config.Env}}'
[PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin]
When you launch a shell like /bin/sh in the container, that shell is what is defining $HOME and various other variables:
$ docker run -it --rm test-workdir /bin/sh
/ # env
HOSTNAME=c68d4370e413
SHLVL=1
HOME=/root
TERM=xterm
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
Since there was no ENV or ARG to set $HOME, you cannot use it in the WORKDIR step.

Yarn build using rollup hangs in docker build process

I am trying to create a docker image for a svelte application. The svelte application I am using is just the svelte template with TypeScript initiated. I have made no changes to the application itself, my only goal is to serve the svelte application using Docker. I am using yarn berry (v2) as my package manager and rollupjs for the build process. This is my first question so please let me know if something is missing.
I am expecting the image to be built, but when the build process reaches the "yarn build" command, it never completes. It doesn't crash either, but it never completes the build step either. It seems to hang at the bundling step:
=> => # src/main.ts → public/build/bundle.js...
The build command from package.json:
"build": "rollup -c"
My Dockerfile was initially taken from here: https://sveltesociety.dev/recipes/publishing-and-deploying/dockerize-a-svelte-app/. However, I had to make some changes to it for yarn v2 and I found the following stack thread: Using Yarn 2 (Berry) for packaging application in a Docker image. Adding some modifications form that question left me with the following Dockerfile:
WORKDIR /app
COPY package.json ./
RUN yarn set version berry
COPY .yarn ./.yarn
COPY yarn.lock .yarnrc.yml ./
RUN yarn plugin import workspace-tools
RUN yarn workspaces focus -A --production
COPY . ./
RUN yarn build # This is where it gets stuck
FROM nginx:1.19-alpine
COPY --from=build /app/public /usr/share/nginx/html
I run the command docker build -t svelte-image . to build and this is the output I get:
[+] Building 572.6s (16/18)
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 37B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 34B 0.0s
=> [internal] load metadata for docker.io/library/nginx:1.19-alpine 1.2s
=> [internal] load metadata for docker.io/library/node:12 1.1s
=> CACHED [stage-1 1/2] FROM docker.io/library/nginx:1.19-alpine#sha256:07ab71a2c8e4ecb19a5a5abcfb3a4f175946c001c8af288b1aa766d67b0 0.0s
=> CACHED [build 1/6] FROM docker.io/library/node:12#sha256:cc4adb82efc04b74b9f96326e682ad04be2df84d23e40609802eb6d6c207abde 0.0s
=> [internal] load build context 0.1s
=> => transferring context: 69.58kB 0.0s
=> CACHED [stage-1 2/3] RUN rm /etc/nginx/nginx.conf /etc/nginx/conf.d/default.conf 0.0s
=> CACHED [build 2/10] WORKDIR /app 0.0s
=> CACHED [build 3/10] COPY package.json ./ 0.0s
=> CACHED [build 4/10] RUN yarn set version berry 0.0s
=> [build 5/10] COPY .yarn ./.yarn 0.2s
=> [build 6/10] COPY yarn.lock .yarnrc.yml ./ 0.0s
=> [build 7/10] RUN yarn plugin import workspace-tools 1.5s
=> [build 8/10] RUN yarn workspaces focus -A --production 0.5s
=> [build 9/10] COPY . ./ 0.1s
=> [build 10/10] RUN yarn build 568.9s
=> => # src/main.ts → public/build/bundle.js... # -> Never completes
I have been trying to figure out why the build hangs, but I am unable to find the source of error. It works just fine when building locally with yarn build. Does anyone have any clue as to why it hangs in the bundling process?
I did not figure out what was wrong, but I managed to build the docker image and run it successfully. Here is the working Dockerfile:
FROM node:16 AS build
WORKDIR /app
COPY package.json ./
RUN yarn set version berry
COPY .yarn ./.yarn #
COPY yarn.lock .yarnrc.yml ./
RUN yarn install
# These two lines are from the original example. It did not work to use this,
# but I leave it here to highlight the difference
## RUN yarn plugin import workspace-tools
## RUN yarn workspaces focus -A --production
COPY . ./
RUN yarn build
FROM nginx:1.19-alpine
COPY --from=build /app/public /usr/share/nginx/html

Docker cli use older logging?

I just installed docker desktop on my windows box, but I uses the new output style, i'd like to switch back to the old style, having trouble finding the exact command or profile part to change.
What I have
docker build .
[+] Building 0.8s (10/10) FINISHED
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 32B 0.0s
=> [internal] load .dockerignore 0.1s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/php:7.4.12-fpm-buster 0.5s
=> [1/6] FROM docker.io/library/php:7.4.12-fpm-buster#sha256:07db4f537d7ea591cd9cecda712aed03ac1aaba8f243961c396 0.0s
=> CACHED [2/6] RUN apt-get update && apt-get upgrade -y && apt-get install git zip -y 0.0s
=> CACHED [3/6] WORKDIR /var/www 0.0s
=> CACHED [4/6] RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename= 0.0s
=> CACHED [5/6] RUN composer --version 0.0s
=> CACHED [6/6] RUN composer require google/cloud google/auth phpseclib/phpseclib 0.0s
=> exporting to image 0.1s
=> => exporting layers 0.0s
=> => writing image sha256:ee8e9007493a15d9ba26d4cf46cdbc7c618a9ab949c7ff9c5e5e2ce717f039d5 0.0s
What I want
docker build .
Sending build context to Docker daemon 2.048kB
Step 1/7 : FROM php:7.4.12-fpm-buster
---> 15d55c4fd75d
Step 2/7 : RUN apt-get update && apt-get upgrade -y && apt-get install git zip -y
---> Running in 6d719912d1e1
...
The new logging style comes from the BuildKit features.
You can disable this in the Docker Desktop GUI:
select the Docker Engine tab
set "features"{ "buildkit" : false }
Then if you want to use the logging features again, you can run with DOCKER_BUILDKIT=1. I believe you can run with DOCKER_BUILDKIT=0 if you want to selectively disable but haven't tested that yet.
Of course, be aware that you'll lose out on the following features that BuildKit adds to Docker:
Docker images created with BuildKit can be pushed to Docker Hub just like Docker images created with legacy build
the Dockerfile format that works on legacy build will also work with BuildKit builds
The new --secret command line option allows the user to pass secret information for building new images with a specified Dockerfile

Resources