I'm trying to install a specific version of npm package within the node:alpine image (latest tag), so this is what I specified in my Dockerfile:
Edit (this is the full Dockerfile):
FROM node:alpine as build-stage
ARG SONAR_VERSION=4.6.0.2311
RUN apk add --no-cache zip unzip
RUN wget -c https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-$SONAR_VERSION-linux.zip
RUN unzip -q sonar-scanner-cli-$SONAR_VERSION-linux.zip
RUN rm sonar-scanner-cli-$SONAR_VERSION-linux.zip
# Adding the $install_directory/bin directory to path
RUN cd sonar-scanner-$SONAR_VERSION-linux
RUN echo $PWD
RUN export PATH="$PATH:/$PWD/bin"
RUN npm i -g eslint#6.7.2
RUN npm i -g eslint-plugin-vue --save-dev
However when I run & execute the built image, the installed version of eslint is the latest instead of what I'd specified in the Dockerfile.
Result from npm list -g:
/ # npm list -g
/usr/local/lib
+-- eslint-plugin-vue#7.9.0
+-- eslint#7.25.0
`-- npm#7.11.2
If I run npm i -g eslint#6.7.2 within the container itself, the version from npm list -g will be exactly the one I specified.
What is the reason of this scenario?
Related
docker can't find file develop.sh even though it's in the root directory
my Dockerfile:
FROM node:16.13.0
WORKDIR /app/medusa
COPY package.json .
COPY develop.sh .
COPY yarn.* .
RUN apt-get update
RUN apt-get install -y python
RUN npm install -g npm#latest
RUN npm install -g #medusajs/medusa-cli#latest
RUN npm install
COPY . .
ENTRYPOINT ["./develop.sh"]
Edit: I am trying to run an open source project called medusa, you can find the code here, I haven't changed any thing except node version in Dockerfile
as per #Charles Duffy suggestion: changing the entrypoint to ENTRYPOINT ["/bin/sh", "./develop.sh"] solved the issue
I'm using pnpm in Dockerfile I have one dependency which is installed from GitHub.
PNPM by default use yarn to install dependency from Git.
Problem with PNPM is it is not able to access the yarn, I think some kind of permission problem.
ERROR:
ERR_PNPM_PREPARE_PKG_FAILURE Command failed with exit code 1: /usr/local/bin/yarn install
The command '/bin/sh -c pnpm install' returned a non-zero code: 1
Here is my Dockerfile
FROM node:alpine
RUN npm install -g pnpm
WORKDIR /app
COPY ["package.json", "pnpm-lock.yaml", "./"]
RUN pnpm install
COPY . .
RUN pnpm build
ENV PORT=8080
EXPOSE 80
CMD [ "node", "./build/index.js" ]
Update
This is repo that is used from GitHub. Baileys
Everything works perfect when I try to install packages without Dockerfile If I run pnpm install everything just works. But When I run the build command for Dockerfile it will create problem.
docker build -t name .
As you stated, pnpm uses yarn to install dependencies from Git. From your output, you can see that yarn failed. If you run inside Docker container yarn add https://github.com/adiwajshing/Baileys.git, it would output:
info No lockfile found.
[1/4] Resolving packages...
error Couldn't find the binary git
node:alpine image is missing git.
To resolve your problem, simply install git before pnpm install in Dockerfile:
FROM node:alpine
RUN apk add --no-cache git
RUN npm install -g pnpm
...
This is my Dockerfile:
RUN mkdir /mcvitty
COPY . /mcvitty
RUN cd /mcvitty &&\
npm install -g gulp#3.9.1 &&\
npm link gulp --force &&\
npm install jshint#2.9.5 &&\
npm install gulp-jshint#2.0.4 &&\
npm install gulp-sass#3.1.0 &&\
npm install gulp-concat#2.6.1 &&\
npm install gulp-uglify#3.0.0 &&\
npm install gulp-rename#1.2.2 &&\
npm install gulp-minify-css#1.2.4 &&\
npm install gulp-image-resize#0.13.0
RUN gulp
RUN gulp resize-images
Whenever I run my container I get the following error:
[08:35:34] Local gulp not found in /
[08:35:34] Try running: npm install gulp
The command '/bin/sh -c gulp' returned a non-zero code: 1
Adding
npm install gulp#3.9.1
to have a local gulp, as suggested by someone does not work either.
Any suggestion?
I have Gulp running in a container. The main differences are that I set a working directory:
RUN mkdir /mcvitty
WORKDIR /mcvitty
COPY . /mcvitty
I don't run npm link gulp, either.
If that doesn't work, remove the RUN gulp commands and add them to your docker run command, e.g.
docker run -it --rm --entrypoint '/bin/sh' myimage -c 'gulp && gulp images'
I fixed the problem by CD back into my dir before running gulp:
RUN cd /mcvitty && gulp
I am running my monolith application in a docker container and k8s on GKE.
The application contains python & node dependencies also webpack for front end bundle.
We have implemented CI/CD which is taking around 5-6 min to build & deploy new version to k8s cluster.
Main goal is to reduce the build time as much possible. Written Dockerfile is multi stage.
Webpack is taking more time to generate the bundle.To buid docker image i am using already high config worker.
To reduce time i tried using the Kaniko builder.
Issue :
As docker cache layers for python code it's working perfectly. But when there is any changes in JS or CSS file we have to generate bundle.
When there is any changes in JS & CSS file instead if generate new bundle its use caching layer.
Is there any way to separate out build new bundle or use cache by passing some value to docker file.
Here is my docker file :
FROM python:3.5 AS python-build
WORKDIR /app
COPY requirements.txt ./
RUN pip install -r requirements.txt &&\
pip3 install Flask-JWT-Extended==3.20.0
ADD . /app
FROM node:10-alpine AS node-build
WORKDIR /app
COPY --from=python-build ./app/app/static/package.json app/static/
COPY --from=python-build ./app ./
WORKDIR /app/app/static
RUN npm cache verify && npm install && npm install -g --unsafe-perm node-sass && npm run sass && npm run build
FROM python:3.5-slim
COPY --from=python-build /root/.cache /root/.cache
WORKDIR /app
COPY --from=node-build ./app ./
RUN apt-get update -yq \
&& apt-get install curl -yq \
&& pip install -r requirements.txt
EXPOSE 9595
CMD python3 run.py
I would suggest to create separate build pipelines for your docker images, where you know that the requirements for npm and pip aren't so frequent.
This will incredibly improve the speed, reducing the time of access to npm and pip registries.
Use a private docker registry (the official one or something like VMWare harbor or SonaType Nexus OSS).
You store those build images on your registry and use them whenever something on the project changes.
Something like this:
First Docker Builder // python-builder:YOUR_TAG [gitrev, date, etc.)
docker build --no-cache -t python-builder:YOUR_TAG -f Dockerfile.python.build .
FROM python:3.5
WORKDIR /app
COPY requirements.txt ./
RUN pip install -r requirements.txt &&\
pip3 install Flask-JWT-Extended==3.20.0
Second Docker Builder // js-builder:YOUR_TAG [gitrev, date, etc.)
docker build --no-cache -t js-builder:YOUR_TAG -f Dockerfile.js.build .
FROM node:10-alpine
WORKDIR /app
COPY app/static/package.json /app/app/static
WORKDIR /app/app/static
RUN npm cache verify && npm install && npm install -g --unsafe-perm node-sass
Your Application Multi-stage build:
docker build --no-cache -t app_delivery:YOUR_TAG -f Dockerfile.app .
FROM python-builder:YOUR_TAG as python-build
# Nothing, already "stoned" in another build process
FROM js-builder:YOUR_TAG AS node-build
ADD ##### YOUR JS/CSS files only here, required from npm! ###
RUN npm run sass && npm run build
FROM python:3.5-slim
COPY . /app # your original clean app
COPY --from=python-build #### only the files installed with the pip command
WORKDIR /app
COPY --from=node-build ##### Only the generated files from npm here! ###
RUN apt-get update -yq \
&& apt-get install curl -yq \
&& pip install -r requirements.txt
EXPOSE 9595
CMD python3 run.py
A question is: why do you install curl and execute again the pip install -r requirements.txt command in the final docker image?
Triggering every time an apt-get update and install without cleaning the apt cache /var/cache/apt folder produces a bigger image.
As suggestion, use the docker build command with the option --no-cache to avoid caching result:
docker build --no-cache -t your_image:your_tag -f your_dockerfile .
Remarks:
You'll have 3 separate Dockerfiles, as I listed above.
Build the Docker images 1 and 2 only if you change your python-pip and node-npm requirements, otherwise keep them fixed for your project.
If any dependency requirement changes, then update the docker image involved and then the multistage one to point to the latest built image.
You should always build only the source code of your project (CSS, JS, python). In this way, you have also guaranteed reproducible builds.
To optimize your environment and copy files across the multi-stage builders, try to use virtualenv for python build.
I'm trying to dockerise my pelican site project. I've created a docker-compose.yml file and a Dockerfile.
However, every time I try to build my project (docker-compose up) I get the following errors for both pip install and npm install:
npm WARN saveError ENOENT: no such file or directory, open '/src/package.json'
...
Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt'
The directory structure of the project is as follows:
- **Dockerfile**
- **docker-compose.yml**
- content/
- pelican-plugins/
- src/
- Themes/
- Pelican config files
- requirements.txt
- gulpfile.js
- package.js
All the pelican makefiles etc. are in the src directory.
I'm trying to load the content, src, and pelican-plugins directories as volumes so I can modify them on my local machine for the docker container to use.
Here is my Dockerfile:
FROM python:3
WORKDIR /src
RUN apt-get update -y
RUN apt-get install -y python-pip python-dev build-essential
# Install Node.js 8 and npm 5
RUN apt-get update
RUN apt-get -qq update
RUN apt-get install -y build-essential
RUN apt-get install -y curl
RUN curl -sL https://deb.nodesource.com/setup_8.x | bash
RUN apt-get install -y nodejs
# Set the locale
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8
RUN npm install
RUN python -m pip install --upgrade pip
RUN pip install -r requirements.txt
ENV SRV_DIR=/src
RUN chmod +x $SRV_DIR
RUN make clean
VOLUME /src/output
RUN make devserver
RUN gulp
And here is my docker-compose.yml file:
version: '3'
services:
web:
build: .
ports:
- "80:80"
volumes:
- ./content:/content
- ./src:/src
- ./pelican-plugins:/pelican-plugins
volumes:
logvolume01: {}
It definitely looks like I have set up my volumes directories properly in dockerfiles...
Thanks in advance!
Your Dockerfile doesn't COPY (or ADD) any files at all, so the /src directory is empty.
You can verify this yourself. When you run docker build it will print out output like:
Step 13/22 : ENV LC_ALL en_US.UTF-8
---> Running in 3ab80c3741f8
Removing intermediate container 3ab80c3741f8
---> d240226b6600
Step 14/22 : RUN npm install
---> Running in 1d31955d5b28
npm WARN saveError ENOENT: no such file or directory, open '/src/package.json'
The last line in each step with just a hex number is actually a valid image ID that's the final result of running each step, and you can then:
% docker run --rm -it d240226b6600 sh
# pwd
/src
# ls
To fix this you need a line in the Dockerfile like
COPY . .
You probably also need to change into the src subdirectory to run npm install and the like as you've shown your directory layout. This can look like:
WORKDIR /src
COPY . .
# Either put "cd" into the command itself
# (Each RUN command starts a fresh container at the current WORKDIR)
RUN cd src && npm install
# Or change WORKDIRs
WORKDIR /src/src
RUN pip install -r requirements.txt
WORKDIR /src
Remember that everything in the Dockerfile happens before any setting in docker-compose.yml outside the build: block is even considered. Environment variables, volume mounts, and networking options for a container have no effect on the image build sequence.
In terms of Dockerfile style, your VOLUME declaration will have some tricky unexpected side effects and probably is unnecessary; I'd remove it. Your Dockerfile is also missing the CMD that the container should run. You should also combine RUN apt-get update && apt-get install into single commands; the way Docker layer caching works and the way the Debian repositories work, it's very easy to wind up with a cached package index that names files from a week ago that don't exist any more.
While the setup you're describing is fairly popular, it also essentially hides everything the Dockerfile does with your local source tree. The npm install you're describing here, for example, will be a no-op because the volume mount will hide /src/src/node_modules. I generally find it easier to just run python, npm, etc. locally while I'm developing, rather than write and debug this 50-line YAML file and run sudo docker-compose up.