I have all this npm stuff in my Dockerfile and it is taking a long time to build my docker image. How can I speed this up, and ideally cache the results? Nothing is changing, so I wouldn't expect this to take such a long time (about 20 seconds now).
FROM python:3.6-alpine
# python stuff
COPY requirements.txt /app/requirements.txt
RUN pip3 install --upgrade pip
RUN pip3 install -r /app/requirements.txt
# npm stuff
RUN apk add --update nodejs-npm
RUN npm init -y
RUN npm i webpack webpack-cli --save-dev
RUN npm i #babel/core babel-loader #babel/preset-env #babel/preset-react babel-plugin-transform-class-properties --save-dev
RUN npm i react react-dom prop-types --save
RUN npm i react-bootstrap bootstrap
RUN npm i weak-key --save
I did try this solution using the COPY package.json but babel and webpack did not seem to like that (and it did not work).
NOTE: I need to use python:3.6-alpine as this is an existing Django application that is integrating React.js
Related
Here is my Dockerfile
# syntax=docker/dockerfile:1
FROM ruby:3.0.3
RUN apt-get update -qq && apt-get install -y nodejs npm yarn postgresql-client
RUN curl --compressed -o- -L https://yarnpkg.com/install.sh | bash
RUN mkdir /myapp
WORKDIR /myapp
COPY Gemfile /myapp/Gemfile
COPY Gemfile.lock /myapp/Gemfile.lock
RUN bundle install
RUN rails assets:precompile
COPY . /myapp
RUN npm install -g yarn -y
RUN yarn install
RUN yarn build:css
RUN rails webpacker:install
RUN rails webpacker:clobber
RUN rails webpacker:compile
# Add a script to be executed every time the container starts.
COPY entrypoint.sh /usr/bin/
RUN chmod +x /usr/bin/entrypoint.sh
ENTRYPOINT ["entrypoint.sh"]
EXPOSE 3000
# Start the main process.
CMD ["rails", "server", "-b", "0.0.0.0"]
And on my package.json I have the following
"scripts": {
"build:css": "sass ./app/assets/stylesheets/application.bootstrap.scss ./app/assets/builds/application.css --no-source-map --load-path=node_modules"
}
When I run docker-compose build works fine but the command RUN yarn build:css has no efect, the builds folder is empty.
I recently ran into an issue similar to this and the source of the problem was in fact npm and yarn.
I was able to get my application up and running in a docker container.
A properly configured application using cssbundling-rails should not need to run the command
"scripts": { "build:css": "sass ./app/assets/stylesheets/application.bootstrap.scss ./app/assets/builds/application.css --no-source-map --load-path=node_modules" },
to get the application to generate the build/application.css
npm known issue
Check your npm version, there is an issue if you are not using npm 7.1+ that basically prevents yarn from setting things up correctly.
Run npm -v and if necessary, upgrade npm with npm install -g npm#latest
If you had to upgrade npm, you will then need to close out your terminal and launch a new one, verify that the npm version is greater than 7.1.
After upgrading, re run rails css:install:bootstrap
And everything should be configured correctly.
Bundle Watcher
From the documentation for cssbundling-rails
"You develop using this approach by running the bundler in watch mode in a terminal with yarn build:css --watch (and your Rails server in another, if you're not using something like puma-dev). You can also use ./bin/dev, which will start both the Rails server and the CSS build watcher (along with a JS build watcher, if you're also using jsbundling-rails)."
So, you can either run yarn build:css --watch from a separate terminal in your rails application or just use the ./bin/dev but either way should resolve the issue you are experience.
I'm trying to install a specific version of npm package within the node:alpine image (latest tag), so this is what I specified in my Dockerfile:
Edit (this is the full Dockerfile):
FROM node:alpine as build-stage
ARG SONAR_VERSION=4.6.0.2311
RUN apk add --no-cache zip unzip
RUN wget -c https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-$SONAR_VERSION-linux.zip
RUN unzip -q sonar-scanner-cli-$SONAR_VERSION-linux.zip
RUN rm sonar-scanner-cli-$SONAR_VERSION-linux.zip
# Adding the $install_directory/bin directory to path
RUN cd sonar-scanner-$SONAR_VERSION-linux
RUN echo $PWD
RUN export PATH="$PATH:/$PWD/bin"
RUN npm i -g eslint#6.7.2
RUN npm i -g eslint-plugin-vue --save-dev
However when I run & execute the built image, the installed version of eslint is the latest instead of what I'd specified in the Dockerfile.
Result from npm list -g:
/ # npm list -g
/usr/local/lib
+-- eslint-plugin-vue#7.9.0
+-- eslint#7.25.0
`-- npm#7.11.2
If I run npm i -g eslint#6.7.2 within the container itself, the version from npm list -g will be exactly the one I specified.
What is the reason of this scenario?
I am working on an existing project that was built by Droopler distribution and docker4drupal. As per Droopler distribution documentation, it requires npm and gulp to compiled sass easily and docker4drupal don't have it by default.
I tried to install in the container by sudo apt-get install npm and it says:
sudo: apt-get: command not found
Is there any way to install npm and gulp in docker4drupal for my existing project?
I am able to install npm & gulp, access php container and install those.
access php container docker-compose exec php bash
install npm sudo apk update && sudo apk add npm
install gulp globally sudo npm install -g gulp
This is how I do it, maybe there is a better way.
Reference: Install npm and gulp
This is my Dockerfile:
RUN mkdir /mcvitty
COPY . /mcvitty
RUN cd /mcvitty &&\
npm install -g gulp#3.9.1 &&\
npm link gulp --force &&\
npm install jshint#2.9.5 &&\
npm install gulp-jshint#2.0.4 &&\
npm install gulp-sass#3.1.0 &&\
npm install gulp-concat#2.6.1 &&\
npm install gulp-uglify#3.0.0 &&\
npm install gulp-rename#1.2.2 &&\
npm install gulp-minify-css#1.2.4 &&\
npm install gulp-image-resize#0.13.0
RUN gulp
RUN gulp resize-images
Whenever I run my container I get the following error:
[08:35:34] Local gulp not found in /
[08:35:34] Try running: npm install gulp
The command '/bin/sh -c gulp' returned a non-zero code: 1
Adding
npm install gulp#3.9.1
to have a local gulp, as suggested by someone does not work either.
Any suggestion?
I have Gulp running in a container. The main differences are that I set a working directory:
RUN mkdir /mcvitty
WORKDIR /mcvitty
COPY . /mcvitty
I don't run npm link gulp, either.
If that doesn't work, remove the RUN gulp commands and add them to your docker run command, e.g.
docker run -it --rm --entrypoint '/bin/sh' myimage -c 'gulp && gulp images'
I fixed the problem by CD back into my dir before running gulp:
RUN cd /mcvitty && gulp
I am running my monolith application in a docker container and k8s on GKE.
The application contains python & node dependencies also webpack for front end bundle.
We have implemented CI/CD which is taking around 5-6 min to build & deploy new version to k8s cluster.
Main goal is to reduce the build time as much possible. Written Dockerfile is multi stage.
Webpack is taking more time to generate the bundle.To buid docker image i am using already high config worker.
To reduce time i tried using the Kaniko builder.
Issue :
As docker cache layers for python code it's working perfectly. But when there is any changes in JS or CSS file we have to generate bundle.
When there is any changes in JS & CSS file instead if generate new bundle its use caching layer.
Is there any way to separate out build new bundle or use cache by passing some value to docker file.
Here is my docker file :
FROM python:3.5 AS python-build
WORKDIR /app
COPY requirements.txt ./
RUN pip install -r requirements.txt &&\
pip3 install Flask-JWT-Extended==3.20.0
ADD . /app
FROM node:10-alpine AS node-build
WORKDIR /app
COPY --from=python-build ./app/app/static/package.json app/static/
COPY --from=python-build ./app ./
WORKDIR /app/app/static
RUN npm cache verify && npm install && npm install -g --unsafe-perm node-sass && npm run sass && npm run build
FROM python:3.5-slim
COPY --from=python-build /root/.cache /root/.cache
WORKDIR /app
COPY --from=node-build ./app ./
RUN apt-get update -yq \
&& apt-get install curl -yq \
&& pip install -r requirements.txt
EXPOSE 9595
CMD python3 run.py
I would suggest to create separate build pipelines for your docker images, where you know that the requirements for npm and pip aren't so frequent.
This will incredibly improve the speed, reducing the time of access to npm and pip registries.
Use a private docker registry (the official one or something like VMWare harbor or SonaType Nexus OSS).
You store those build images on your registry and use them whenever something on the project changes.
Something like this:
First Docker Builder // python-builder:YOUR_TAG [gitrev, date, etc.)
docker build --no-cache -t python-builder:YOUR_TAG -f Dockerfile.python.build .
FROM python:3.5
WORKDIR /app
COPY requirements.txt ./
RUN pip install -r requirements.txt &&\
pip3 install Flask-JWT-Extended==3.20.0
Second Docker Builder // js-builder:YOUR_TAG [gitrev, date, etc.)
docker build --no-cache -t js-builder:YOUR_TAG -f Dockerfile.js.build .
FROM node:10-alpine
WORKDIR /app
COPY app/static/package.json /app/app/static
WORKDIR /app/app/static
RUN npm cache verify && npm install && npm install -g --unsafe-perm node-sass
Your Application Multi-stage build:
docker build --no-cache -t app_delivery:YOUR_TAG -f Dockerfile.app .
FROM python-builder:YOUR_TAG as python-build
# Nothing, already "stoned" in another build process
FROM js-builder:YOUR_TAG AS node-build
ADD ##### YOUR JS/CSS files only here, required from npm! ###
RUN npm run sass && npm run build
FROM python:3.5-slim
COPY . /app # your original clean app
COPY --from=python-build #### only the files installed with the pip command
WORKDIR /app
COPY --from=node-build ##### Only the generated files from npm here! ###
RUN apt-get update -yq \
&& apt-get install curl -yq \
&& pip install -r requirements.txt
EXPOSE 9595
CMD python3 run.py
A question is: why do you install curl and execute again the pip install -r requirements.txt command in the final docker image?
Triggering every time an apt-get update and install without cleaning the apt cache /var/cache/apt folder produces a bigger image.
As suggestion, use the docker build command with the option --no-cache to avoid caching result:
docker build --no-cache -t your_image:your_tag -f your_dockerfile .
Remarks:
You'll have 3 separate Dockerfiles, as I listed above.
Build the Docker images 1 and 2 only if you change your python-pip and node-npm requirements, otherwise keep them fixed for your project.
If any dependency requirement changes, then update the docker image involved and then the multistage one to point to the latest built image.
You should always build only the source code of your project (CSS, JS, python). In this way, you have also guaranteed reproducible builds.
To optimize your environment and copy files across the multi-stage builders, try to use virtualenv for python build.