Debug docker script more efficiently - docker

I'm getting started on my first docker script and i'm trying to debug the last step and it's very slow debugging b/c the steps before it take a few minutes to run so if i have a typo, i have to re run the entire script.
Is there a more efficient way to debug a docker script? or do i have to rebuild the whole thing every time?
#FROM ubuntu:14.04
FROM node:0.10.40
#FROM mongo:2.6.11
# The port we're running the server on
EXPOSE 10645
# Set this as the working directory
WORKDIR /myproject/hapi
# Move the myproject files to /myproject in the docker container
ADD . /myproject/hapi
# Install the server dependencies
RUN pwd && ls -al && npm install
# Start everything up
CMD npm start
Log output:
^CR5033505:myproject m089269$ docker build -t myproject-hapi .
Sending build context to Docker daemon 932.2 MB
Step 0 : FROM node:0.10.40
---> a7d8016a6fdb
Step 1 : EXPOSE 10645
---> Running in ebc4f8ebbf7b
---> 701320586e6a
Removing intermediate container ebc4f8ebbf7b
Step 2 : WORKDIR /myproject/hapi
---> Running in 1998f97b252a
---> 1414baf38920
Removing intermediate container 1998f97b252a
Step 3 : ADD . /myproject/hapi
---> c80e665da20b
Removing intermediate container f6904fab79ce
Step 4 : RUN pwd && ls -al && npm install
---> Running in a3ef28ed70ae
/myproject/hapi
total 68
drwxr-xr-x 9 root root 4096 Oct 30 18:35 .
drwxr-xr-x 3 root root 4096 Oct 30 18:35 ..
-rw-r--r-- 1 root root 509 Apr 10 2015 .editorconfig
drwxr-xr-x 8 root root 4096 Oct 30 18:33 .git
-rw-r--r-- 1 root root 491 Oct 20 15:09 .gitignore
drwxr-xr-x 8 root root 4096 Aug 19 14:51 .idea
-rw-r--r-- 1 root root 1781 Apr 10 2015 .jscsrc
-rw-r--r-- 1 root root 6164 Apr 10 2015 .tfignore
-rw-r--r-- 1 root root 430 Oct 30 18:33 Dockerfile
-rw-r--r-- 1 root root 371 Oct 30 18:16 Dockerfile-client
-rwxr-xr-x 1 root root 1374 Oct 30 15:15 README.md
drwxr-xr-x 5 root root 4096 Oct 21 21:18 ab-testing-deploy
drwxr-xr-x 3 root root 4096 Oct 30 15:15 build
drwxr-xr-x 14 root root 4096 Oct 30 15:15 client
drwxr-xr-x 2 root root 4096 Apr 10 2015 githooks
drwxr-xr-x 10 root root 4096 Oct 30 15:15 hapi
npm ERR! install Couldn't read dependencies
npm ERR! Linux 4.1.10-boot2docker
npm ERR! argv "node" "/usr/local/bin/npm" "install"
npm ERR! node v0.10.40
npm ERR! npm v2.14.1
npm ERR! path /myproject/hapi/package.json
npm ERR! code ENOPACKAGEJSON
npm ERR! errno 34
npm ERR! package.json ENOENT, open '/myproject/hapi/package.json'
npm ERR! package.json This is most likely not a problem with npm itself.
npm ERR! package.json npm can't find a package.json file in your current directory.
npm ERR! Please include the following file with any support request:
npm ERR! /myproject/hapi/npm-debug.log
The command '/bin/sh -c pwd && ls -al && npm install' returned a non-zero code: 34

Unfortunately when you're doing your ADD command you're invalidating the docker build cache, so any commands after that will be run from scratch. From the docs:
Note: The first encountered ADD instruction will invalidate the cache for all following instructions from the Dockerfile if the contents of have changed. This includes invalidating the cache for RUN instructions.
https://docs.docker.com/reference/builder/#add
I think in this case it would be easiest to run an interactive container and run each command from your Dockerfile one by one so you can see which one fails and be in a position to run it again immediately after you attempt to fix the error.
For example:
docker run -ti -v /path/to/your/code:/myproject/hapi node:0.10.40 bash
$ cd /myproject/hapi
$ npm install
Then when you do an ls -la you'll be able to see that package.json either isn't there or something else is wrong, and once you've figured it out you can add that command into your Dockerfile.

Related

Bug or misuse : docker layer content not available in final image

I am having a hard time trying to build a docker image for some python project, but at some point a RUN command resulting content is not available in the next RUN.
I mean I have a RUN command that generate a directory with some content and that directory is not available for the next RUN command.
Here is the Dockerfile snippet :
RUN poetry config virtualenvs.create true && \
poetry config virtualenvs.in-project true && \
poetry install -n --no-root --no-dev && \
ls -la
RUN ls -la
The first ls -la shows a .venv directory (see below) while it is not present anymore in the second one.
And here is the build output:
Step 9/14 : RUN poetry config virtualenvs.create true && poetry config virtualenvs.in-project true && poetry install -n --no-root --no-dev && ls -la
---> Running in 3a93a4994133
...
drwxrwxr-x 4 root odoo 4096 Sep 29 20:08 .
drwxr-xr-x 1 root root 4096 Sep 16 02:35 ..
drwxr-xr-x 4 odoo odoo 4096 Sep 29 20:08 .venv
-rw-rw-r-- 1 odoo odoo 15824 Sep 2 23:18 poetry.lock
-rw-rw-r-- 1 odoo odoo 2057 Sep 2 23:50 pyproject.toml
drwxr-xr-x 3 odoo odoo 4096 Sep 29 20:08 src
Removing intermediate container 3a93a4994133
---> 694c4e54863c
Step 10/14 : RUN ls -la
---> Running in 5d0ebcf8f1e9
total 32
drwxrwxr-x 3 root odoo 4096 Sep 29 20:08 .
drwxr-xr-x 1 root root 4096 Sep 16 02:35 ..
-rw-rw-r-- 1 odoo odoo 15824 Sep 2 23:18 poetry.lock
-rw-rw-r-- 1 odoo odoo 2057 Sep 2 23:50 pyproject.toml
drwxr-xr-x 3 odoo odoo 4096 Sep 29 20:08 src
Removing intermediate container 5d0ebcf8f1e9
---> 750255eac6bb
If someone can provide any lead, it would be really appreciated.
Thanks

How to install MariaDB ODBC drivers in Docker/Alpine

I see that Alpine supports in its Docker container a MariaDB ODBC Driver as listed here. I need to install it to be used with pyodbc.
What is the Dockerfile command that installs the driver in the image?
Something along the lines of RUN apk add mariadb-connector-odbc
?
My clean install of Alpine had an /etc/apk/repositories file that looked like
#/media/cdrom/apks
http://mirror.reenigne.net/alpine/v3.13/main
#http://mirror.reenigne.net/alpine/v3.13/community
#http://mirror.reenigne.net/alpine/edge/main
#http://mirror.reenigne.net/alpine/edge/community
#http://mirror.reenigne.net/alpine/edge/testing
With that setup
apk add mariadb-connector-odbc
failed. However, after uncommenting the last line …
#/media/cdrom/apks
http://mirror.reenigne.net/alpine/v3.13/main
#http://mirror.reenigne.net/alpine/v3.13/community
#http://mirror.reenigne.net/alpine/edge/main
#http://mirror.reenigne.net/alpine/edge/community
http://mirror.reenigne.net/alpine/edge/testing
… the same command succeeded
localhost:~# apk add mariadb-connector-odbc
fetch http://mirror.reenigne.net/alpine/edge/testing/x86_64/APKINDEX.tar.gz
(1/3) Installing readline (8.1.0-r0)
(2/3) Installing unixodbc (2.3.9-r1)
(3/3) Installing mariadb-connector-odbc (3.1.11-r0)
Executing busybox-1/32/1-r6.trigger
OK: 903 MiB in 146 packages
localhost:~# ls -la /usr/lib/mariadb
total 536
drwxr-xr-x 2 root root 4096 May 14 22:43 .
drwxr-xr-x 6 root root 4096 May 14 22:43 ..
-rwxr-xr-x 1 root root 537856 Jan 6 02:39 libmaodbc.so

Docker: created files disappear between layers

Running Docker version 17.06.0-ce, build 02c1d87, I have a dockerfile that looks like this:
FROM maven:3.5.2-jdk-8-alpine as builder
RUN chmod -R 777 /root/.m2 &&\
mkdir -p /root/.m2/repository/com/foo/bar &&\
echo "Text" > /root/.m2/repository/com/foo/bar/baz.txt &&\
ls -R -a -l /root/.m2/repository/com/foo
RUN ls -R -a -l /root/.m2/repository/com/foo
The first RUN command successfully creates a file, but the second command can't find it:
Step 1/46 : FROM maven:3.5.2-jdk-8-alpine as builder
---> 293423a981a7
Step 2/46 : RUN chmod -R 777 /root/.m2 && mkdir -p /root/.m2/repository/com/foo/bar && echo "Text" > /root/.m2/repository/com/foo/bar/baz.txt && ls -R -a -l /root/.m2/repository/com/foo
---> Running in a1c0fd142856
/root/.m2/repository/com/foo:
total 12
drwxr-xr-x 3 root root 4096 Nov 30 13:32 .
drwxr-xr-x 3 root root 4096 Nov 30 13:32 ..
drwxr-xr-x 2 root root 4096 Nov 30 13:32 bar
/root/.m2/repository/com/foo/bar:
total 12
drwxr-xr-x 2 root root 4096 Nov 30 13:32 .
drwxr-xr-x 3 root root 4096 Nov 30 13:32 ..
-rw-r--r-- 1 root root 5 Nov 30 13:32 baz.txt
---> b997ccbfd5b0
Step 3/46 : RUN ls -R -a -l /root/.m2/repository/com/foo
---> Running in 603671c87ecc
ls: /root/.m2/repository/com/foo: No such file or directory
The command '/bin/sh -c ls -R -a -l /root/.m2/repository/com/foo' returned a non-zero code: 1
What's going on? (NB. this is a toy example, but there is a real issue in that JARs installed into the Maven repository seem to disappear between layers.)
The upstream maven image defines this directory as a volume. Once an image does this, you cannot reliably make changes to that directory in the image.
From their Dockerfile:
ARG USER_HOME_DIR="/root"
...
VOLUME "$USER_HOME_DIR/.m2"
The Dockerfile documentation describes this behavior:
Changing the volume from within the Dockerfile: If any build steps change the data within the volume after it has been declared, those changes will be discarded.
Your options are to:
Use another directory for your build
Request that the upstream image removes this VOLUME definition
Build your own image without this definition (it's fairly easy to fork their repo and do your own build)
For more details, you can see an old blog post by me about this behavior and the problems it creates.

Error on docker run

i'm having a problem when i do docker run on a image that i created using this dockerfile:
FROM node
WORKDIR /Saiph
EXPOSE 3000
ENTRYPOINT ["npm", "start"]
COPY . /Saiph
RUN npm install
The error is this:
PS D:\saiph> docker run 1ba8ca0d9b3b
npm info it worked if it ends with ok
npm info using npm#5.3.0
npm info using node#v8.4.0
npm info lifecycle saiph#1.0.0~prestart: saiph#1.
npm info lifecycle saiph#1.0.0~start: saiph#1.0.0
> saiph#1.0.0 start /Saiph
> cd server && node server
sh: 1: cd: can't cd to server
npm info lifecycle saiph#1.0.0~start: Failed to e
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! saiph#1.0.0 start: `cd server && node server`
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the saiph#1.0.0 start script.
I've tried to understand the error but i didn't understand it,
Sorry about my bad english,
Thanks
You issue is the difference between Windows and Linux file system. In windows if a folder named Server exists then you can use cd server or cd Server and it would work.
Linux is case sensitive. So either change your script to use the specific case or rename the folder from Server to server. You will need to do the same thing in code also to rename requires and other file related things to exact case
#TarunLalwani
PS D:\seith> docker run 15027498053c ls -al
total 72
drwxr-xr-x 1 root root 4096 Sep 9 19:15 .
drwxr-xr-x 1 root root 4096 Sep 9 19:16 ..
drwxr-xr-x 7 root root 4096 Sep 9 00:41 .git
-rwxr-xr-x 1 root root 6222 Sep 9 00:40 .gitignore
drwxr-xr-x 4 root root 4096 Sep 9 16:29 .vs
drwxr-xr-x 2 root root 4096 Sep 8 22:55 Client
drwxr-xr-x 3 root root 4096 Sep 9 00:38 Database
-rwxr-xr-x 1 root root 92 Sep 9 19:15 Dockerfile
-rwxr-xr-x 1 root root 1089 Sep 9 00:37 LICENSE
drwxr-xr-x 3 root root 4096 Sep 9 00:38 Server
drwxr-xr-x 76 root root 4096 Sep 9 16:29 node_modules
-rwxr-xr-x 1 root root 18807 Sep 9 19:15 package-lock.json
-rwxr-xr-x 1 root root 368 Sep 9 16:29 package.json

Can't access mounted volume in docker

UPDATE: This wasn't an issue when I setup my project on a Mac. I was using Fedora 24 in this problem.
I am trying to access my app in my docker instance. When I try and ls the mounted directory, I get a permission error:
root#591d02d0d6d2:/app#
ls: cannot open directory .: Permission denied
This is what my docker file looks like:
FROM pvlltvk/ubuntu-trusty-php-fpm-5.6
RUN apt-get install -y\
php5-curl \
php5-sybase \
freetds-dev \
libxml2-dev
ADD freetds.conf /etc/freetds/freetds.conf
USER root
RUN echo 'alias sf="php /app/app/console"' >> ~/.bashrc
WORKDIR /app
I know it's a permissions issue but I couldn't get any solutions to work. If I run ls -lh I can see that the owner id is 1000 and not root.
root#591d02d0d6d2:/# ls -lh
total 24K
drwxrwxr-x. 11 1000 1000 4.0K Dec 12 17:42 app //my project
drwxr-xr-x. 2 root root 4.0K Dec 13 08:25 bin
drwxr-xr-x. 2 root root 6 Apr 10 2014 boot
drwxr-xr-x. 5 root root 360 Dec 15 02:58 dev

Resources