I'm running
docker build --build-arg npm_token=//NPM TOKEN HERE// -t test .
and the build is failing with an error message of
error Command "dotenv" not found even though the dotenv nom package is included in the package.json and the yarn.lock files. Here is my docker file:
# image has Cypress npm module installed globally in /root/.npm/node_modules
# and Cypress binary cached in /root/.cache/Cypress folder
FROM cypress/included:9.7.0
WORKDIR /usr/src/app
# Set up NPM token to access private GitHub packages
ARG npm_token
ENV NPM_TOKEN=$npm_token
COPY .npmrc ./
RUN npm config set //npm.pkg.github.com/:_authToken $NPM_TOKEN
COPY config cypress .env package.json cypress.json yarn.lock tsconfig.json ./
RUN yarn \
dotenv -- node e2e-tests.js
Does anyone know why this is happening?
you should install dotenv-cli to run .env commands :
yarn add dotenv-cli
Related
My git repo structure
app
- app.server
- server files
- app.client
- node_modules
- public
- src
- .dockerignore
- Dockerfile
- package.json
- package-log.json
I've set up CI/CD with GitHub actions but something is wrong in my Dockerimage for my client application (React)?
Error message: COPY failed: file not found in build context or excluded by .dockerignore: stat package.json: file does not exist
My .dockeringore file:
node_modules
build
.dockerignore
Dockerfile
Dockerfile.prod
My Dockerfile:
# pull official base image
FROM node:13.12.0-alpine
# set working directory
WORKDIR /app.client
# add `/app/node_modules/.bin` to $PATH
ENV PATH /app.client/node_modules/.bin:$PATH
# install app dependencies
COPY package.json ./
COPY package-lock.json ./
RUN npm install --silent
RUN npm install react-scripts#3.4.1 -g --silent
# add app
COPY ./ ./
# start app
CMD ["npm", "start"]
My GitHub action command for invoking the Dockerfile:
docker build app.client/ -t mycontainerregistry.azurecr.io/appdb:${{ github.sha }}
This is part of a publish to Azure Container registry that I'm trying to learn. I guess the Dockerfile works, because before failing at step 4/9 it goes thru the Dockerfile:
Step 1/9 : FROM node:13.12.0-alpine
13.12.0-alpine: Pulling from library/node
aad63a933944: Pulling fs layer
... (and so on)
Step 3/6 : COPY package.json ./
COPY failed: file not found in build context or excluded by .dockerignore: stat package.json: file does not exist
Error: Process completed with exit code 1.
WORKDIR refers to the working directory in the container. The working directory on the host is still the root of your repo.
I'm trying to create a Docker container to act as a test environment for my application. I am using the following Dockerfile:
FROM node:14.4.0-alpine
WORKDIR /test
COPY package*.json ./
RUN npm install .
CMD [ "npm", "test" ]
As you can see, it's pretty simple. I only want to install all dependencies but NOT copy the code, because I will run that container with the following command:
docker run -v `pwd`:/test -t <image-name>
But the problem is that node_modules directory is deleted when I mount the volume with -v. Any workaround to fix this?
When you bind mount test directory with $PWD, you container test directory will be overridden/mounted with $PWD. So you will not get your node_modules in test directory anymore.
To fix this issue you can use two options.
You can run npm install in separate directory like /node and mount your code in test directory and export node_path env like export NODE_PATH=/node/node_modules
then Dockerfile will be like:
FROM node:14.4.0-alpine
WORKDIR /node
COPY package*.json ./
RUN npm install .
WORKDIR /test
CMD [ "npm", "test" ]
Or you can write a entrypoint.sh script that will copy the node_modules folder to the test directory at the container runtime.
FROM node:14.4.0-alpine
WORKDIR /node
COPY package*.json ./
RUN npm install .
WORKDIR /test
COPY Entrypoint.sh ./
ENTRYPOINT ["Entrypoint.sh"]
and Entrypoint.sh is something like
#!/bin/bash
cp -r /node/node_modules /test/.
npm test
Approach 1
A workaround is you can do
CMD npm install && npm run dev
Approach 2
Have docker install node_modules on docker-compose build and run the app on docker-compose up.
Folder Structure
docker-compose.yml
version: '3.5'
services:
api:
container_name: /$CONTAINER_FOLDER
build: ./$LOCAL_FOLDER
hostname: api
volumes:
# map local to remote folder, exclude node_modules
- ./$LOCAL_FOLDER:/$CONTAINER_FOLDER
- /$CONTAINER_FOLDER/node_modules
expose:
- 88
Dockerfile
FROM node:14.4.0-alpine
WORKDIR /test
COPY ./package.json .
RUN npm install
# run command
CMD npm run dev
I have a Dockerfile and when I run it locally, everything works fine, however my build through GitHub actions seems to fail, the error I am getting is:
error An unexpected error occurred: "ENOENT: no such file or directory, stat '/home/runner/work/akira/akira/README.md'".
I tried to remove the yarn.lock but without success, a full log of the build that fails can be found here, my Dockerfile is below:
Dockerfile:
FROM node:14.0.0 AS base
WORKDIR /usr/src/app
FROM base as builder
COPY ./lerna.json .
COPY ./package.json .
COPY ./tsconfig.json .
COPY ./yarn.lock .
COPY ./packages/akira/prisma ./packages/akira/prisma
COPY ./packages/akira/src ./packages/akira/src
COPY ./packages/akira/types ./packages/akira/types
COPY ./packages/akira/package*.json ./packages/akira/
COPY ./packages/akira/tsconfig.json ./packages/akira
RUN yarn install --frozen-lockfile
RUN yarn build
FROM builder as migrate
RUN yarn workspace akira prisma migrate up --experimental
FROM base AS app
COPY --from=builder /usr/src/app/yarn.lock .
COPY --from=builder /usr/src/app/packages/akira/dist ./dist
COPY --from=builder /usr/src/app/packages/akira/prisma ./prisma
COPY --from=builder /usr/src/app/packages/akira/package.json .
RUN yarn install --production
USER node
ENV NODE_ENV=production
EXPOSE 4000
CMD ["node", "dist/index.js"]
If you look at your GitHub Actions workflow,
or the log from the failing build that you linked, it seems to be running yarn commands outside of docker.
It looks like yarn is struggling with the README symlink, not sure why, but as it seems you want to build with docker, I would try the following:
Replace this part of the yaml
- name: Use Node.js
uses: actions/setup-node#master
with:
node-version: 14.4.0
- name: Install dependencies
run: yarn --frozen-lockfile
- name: Build packages
run: yarn build
with something like
- name: Build docker image
run: docker build .
Edit:
As pointed out in below comment, the Dockerfile includes a side-effect of deploying database migrations.
If you don't want to run everything from the Dockerfile in the Build pipeline,
you can leverage multi-stage builds and stop at a specific stage.
I.e., move the migrations into its own stage:
FROM node:14.0.0 AS base
WORKDIR /usr/src/app
FROM base as builder
COPY ./lerna.json .
<< lines omitted >>
RUN yarn install --frozen-lockfile
RUN yarn build
FROM builder AS migr
RUN yarn workspace akira prisma migrate up --experimental
FROM base AS app
COPY --from=builder /usr/src/app/yarn.lock .
<< lines omitted >>
Then you can stop after the builder stage with
docker build --target builder .
Edit 2:
Or you could keep the build pipeline and Dockerfile as it is, and instead fix the broken symlink, i.e. revert commit 0c87fa3
The npm version is located at package.json.
I have a Dockerfile, simplified as follows:
FROM NODE:carbon
COPY ./package.json ${DIR}/
RUN npm install
COPY . ${DIR}
RUN npm build
Correct my understanding,
If ./package.json changes, is it true that the writable docker image layers changes are from 2 to 5?
Assuming that I do not have any changes on npm package dependencies,
How could I change the project version but I do not want docker rebuild image layer for RUN npm install ?
To sum up, the behavior you describe using Docker is fairly standard (as soon as package.json has changed and has a different hash, COPY package.json ./ will be trigerred again as well as each subsequent Dockerfile command).
Thus, the docker setup outlined in the official doc of Node.js does not alleviate this, and proposes the following Dockerfile:
FROM node:carbon
# Create app directory
WORKDIR /usr/src/app
# Install app dependencies
# A wildcard is used to ensure both package.json AND package-lock.json are copied
# where available (npm#5+)
COPY package*.json ./
RUN npm install
# If you are building your code for production
# RUN npm install --only=production
# Bundle app source
COPY . .
EXPOSE 8080
CMD [ "npm", "start" ]
But if you really want to find ways to avoid rerunning npm install from scratch most of the time, you could just maintain two different files package.json and, say, package-deps.json, import (and rename) package-deps.json and run npm install, then use the proper package.json afterwards.
And if you want to have more checks to be sure that the dependencies of both files are not out-of-sync, it happens that the role of file package-lock.json may be of some help, if you use the new npm ci feature that comes with npm 5.8 (cf. the corresponding changelog) instead of using npm install.
In this case, as the latest version of npm available in Docker Hub is npm 5.6, you'll need to upgrade it beforehand.
All things put together, here is a possible Dockerfile for this use case:
FROM node:carbon
# Create app directory
WORKDIR /usr/src/app
# Upgrade npm to have "npm ci" available
RUN npm install -g npm#5.8.0
# Import conf files for dependencies
COPY package-lock.json package-deps.json ./
# Note that this REQUIRES to run the command "npm install --package-lock-only"
# before running "docker build …" and also REQUIRES a "package-deps.json" file
# that is in sync w.r.t. package.json's dependencies
# Install app dependencies
RUN mv package-deps.json package.json && npm ci
# Note that "npm ci" checks that package.json and package-lock.json are in sync
# COPY package.json ./ # subsumed by the following command
COPY . .
EXPOSE 8080
CMD [ "npm", "start" ]
Disclaimer: I did not try the solution above on a realistic example as I'm not a regular node user, but you may view this as a useful workaround for the dev phase…
I am using yarn workspaces and I have this packages in my package.json:
"workspaces": ["packages/*"]
I am trying to create a docker image to deploy and I have the following Dockerfile:
# production dockerfile
FROM node:9.2
# add code
COPY ./packages/website/dist /cutting
WORKDIR /cutting
COPY package.json /cutting/
RUN yarn install --pure-lockfile && yarn cache clean --production
CMD npm run serve
But I get the following error:
error An unexpected error occurred:
"https://registry.yarnpkg.com/#cutting%2futil: Not found"
#cutting/util is the name of one of my workspace packages.
So the problem is that there is no source code in the docker image so it is trying to install it from yarnpkg.
what is the best way to handle workspaces when deploying to a docker image.
This code won't work outside of the docker vm, so it will refuse in the docker, too.
The problem is you have built a code, and copy the bundled code. The yarn workspaces is looking for a package.json that you don't have in the dist folder. The workspaces is just creating a link in a common node_modules folder to the other workspace that you are using. The source code is needed there. (BTW why don't you build code inside the docker vm? That way source code and dist would also be available.)
Here is my dockerfile. I use yarn workspaces and lerna, but without lerna should be similar. You want to build your shared libraries and then test the build works locally by running your code in your dist folder.
###############################################################################
# Step 1 : Builder image
FROM node:11 AS builder
WORKDIR /usr/src/app
ENV NODE_ENV production
RUN npm i -g yarn
RUN npm i -g lerna
COPY ./lerna.json .
COPY ./package* ./
COPY ./yarn* ./
COPY ./.env .
COPY ./packages/shared/ ./packages/shared
COPY ./packages/api/ ./packages/api
# Install dependencies and build whatever you have to build
RUN yarn install --production
RUN lerna bootstrap
RUN cd /usr/src/app/packages/shared && yarn build
RUN cd /usr/src/app/packages/api && yarn build
###############################################################################
# Step 2 : Run image
FROM node:11
LABEL maintainer="Richard T"
LABEL version="1.0"
LABEL description="This is our dist docker image"
RUN npm i -g yarn
RUN npm i -g lerna
ENV NODE_ENV production
ENV NPM_CONFIG_LOGLEVEL error
ARG PORT=3001
ENV PORT $PORT
WORKDIR /usr/src/app
COPY ./package* ./
COPY ./lerna.json ./
COPY ./.env ./
COPY ./yarn* ./
COPY --from=builder /usr/src/app/packages/shared ./packages/shared
COPY ./packages/api/package* ./packages/api/
COPY ./packages/api/.env* ./packages/api/
COPY --from=builder /usr/src/app/packages/api ./packages/api
RUN yarn install
CMD cd ./packages/api && yarn start-production
EXPOSE $PORT
###############################################################################