gcloud, docker and nextjs: Couldn't find a `pages` directory - docker

My config looks like this:
Dockerfile:
FROM node:current-alpine AS base
WORKDIR /base
COPY package*.json ./
RUN npm install
COPY . .
FROM base AS build
ENV NODE_ENV=production
WORKDIR /build
COPY --from=base /base ./
RUN npm run build
FROM node:current-alpine AS production
ENV NODE_ENV=production
WORKDIR /app
COPY --from=build /build/package*.json ./
COPY --from=build /build/build ./build
COPY --from=build /build/public ./public
RUN npm install next
EXPOSE 3000
CMD npm run start
next.config.js
const withPWA = require('next-pwa');
const runtimeCaching = require('next-pwa/cache');
module.exports = withPWA({
distDir: 'build',
pwa: {
dest: 'public',
runtimeCaching,
},
experimental: {
outputStandalone: true,
},
staticPageGenerationTimeout: 10000,
});
this works fine if I use docker build -t [my project name] . command, but fails with the gcloud. I'm using it like this gcloud builds submit --tag gcr.io/my project name] --project my project name] --timeout=900s but keep getting the error:
Error: > Couldn't find a `pages` directory. Please create one under the project root
Not sure what am I doing

Related

Dockerfile: How do I reflect the right path?

I keep having an issue where I get the error: Cannot find module '/mfa/main.js'.
However, the main.js is inside of /mfa/dist/apps/api
This is the latest configuration of Dockerfile I have:
FROM node:14
WORKDIR /mfa/
COPY package.json .
COPY decorate-angular-cli.js .
COPY yarn.lock .
# Configure NPM with the group access token
ENV GROUP_NPM_TOKEN="asdfghjkiuy"
RUN npm config set #my-web:registry http://git.hoosiers.com/api/v4/packages/npm
RUN npm config set //git.hoosiers.com/api/v4/packages/npm/:_authToken=${GROUP_NPM_TOKEN}
RUN npm config set //git.hoosiers.com/api/v4/packages/projects/:_authToken=${GROUP_NPM_TOKEN}
RUN yarn add typescript
RUN yarn install --frozen-lockfile
COPY ./dist .
CMD ["node", "apps/api/main.js"]
So now docker run <image-hash> runs just fine, but when I attempt docker-compose up is when I once again get Cannot find module '/mfa/main.js'.
This is my docker-compose.yml file:
version: '3.9'
services:
web-app:
build:
context: .
dockerfile: mostly-failed-apps.Dockerfile
ports:
- "3000:3000"
You have difine your WORKDIR is /mfa and you execute your main.js in apps/api/main.js
And tip for copy it's not mandatory to write /mfa/ you can just write dot (.) because your in WORKDIR
`
FROM node:14
# Go on /mfa if is dosen't exist WORKDIR create it and go in
WORKDIR /mfa/
# Copy of package.json where we are so we are with the dot so in /mfa/ it's the same for all copy
COPY package.json .
COPY decorate-angular-cli.js .
COPY yarn.lock .
# Configure NPM with the group access token
ENV GROUP_NPM_TOKEN="token"
RUN npm config set #my-web:registry http://git.hoosiers.com/api/v4/packages/npm
RUN npm config set //git.hoosiers.com/api/v4/packages/npm/:_authToken=${GROUP_NPM_TOKEN}
RUN npm config set //git.hoosiers.com/api/v4/packages/projects/:_authToken=${GROUP_NPM_TOKEN}
RUN yarn add typescript
RUN yarn install --frozen-lockfile
COPY ./dist .
# You have create your docker with /mfa/ so you need to excute it in /mfa/
CMD ["node", "/mfa/dist/apps/api/main.js"]
`

How run next.js with node-sharp for docker

I have a problem implementing sharp for Dockerfile.
Error: 'sharp' is required to be installed in standalone mode for the image
optimization to function correctly
Next.js with sharp works fine for local developing:
next 12.0.1
sharp 0.30.2
node 16.xx
npm 8.xx
OS - macOS Monterey - 12.2.1, M1 PRO
next.config.js
module.exports = {
experimental: {
outputStandalone: true,
},
}
Dockerfile:
FROM node:16-alpine AS deps
# Check https://github.com/nodejs/docker-node/tree/b4117f9333da4138b03a546ec926ef50a31506c3#nodealpine to understand why libc6-compat might be needed.
RUN apk add --no-cache libc6-compat
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
# Rebuild the source code only when needed
FROM node:16-alpine AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
# Next.js collects completely anonymous telemetry data about general usage.
# Learn more here: https://nextjs.org/telemetry
# Uncomment the following line in case you want to disable telemetry during the build.
# ENV NEXT_TELEMETRY_DISABLED 1
RUN npm run build
# Production image, copy all the files and run next
FROM node:16-alpine AS runner
WORKDIR /app
ENV NODE_ENV production
# Uncomment the following line in case you want to disable telemetry during runtime.
# ENV NEXT_TELEMETRY_DISABLED 1
RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs
# You only need to copy next.config.js if you are NOT using the default configuration
COPY --from=builder /app/public ./public
COPY --from=builder /app/package.json ./package.json
COPY --from=builder /app/package.json ./package.json
COPY --from=builder /app/next.config.js ./
COPY --from=builder /app/next-i18next.config.js ./
COPY --from=builder /app/next-sitemap.js ./cd
COPY --from=builder /app/jsconfig.json ./jsconfig.json
COPY --from=builder /app/data/ ./data
COPY --from=builder /app/components/ ./components
COPY --from=builder /app/utils/ ./utils
COPY --from=builder /app/assets/ ./assets
# Automatically leverage output traces to reduce image size
# https://nextjs.org/docs/advanced-features/output-file-tracing
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static
USER nextjs
EXPOSE 3000
ENV PORT 3000
CMD ["node", "server.js"]
.env file:
NEXT_SHARP_PATH=/tmp/node_modules/sharp next start
Sharp is installed in package.json
I checked both Next/Vercel tuts:
https://nextjs.org/docs/messages/install-sharp
https://nextjs.org/docs/messages/sharp-missing-in-production
RUN Docker:
docker build --no-cache . -t website-app && docker run --name website -p 3000:3000 website-app
I figured it out.
I removed NEXT_SHARP_PATH=/tmp/node_modules/sharp next start from .env
Local docker doesn't work with node-sharp without NEXT_SHARP_PATH and that is weird.
But I deployed it in my K8s Cluster with Docker and it works as is expected.
npm i sharp
this fix the problem for me
What works for me, is to change the version of Alpine Linux in the Dockerfile:
FROM node:18-alpine AS deps (instead of node:16)
Adding or removing node-sharp in my .env file didn't work for me.
For the final step (the one labeled "runner") in your Dockerfile, replace the base image with node:16-slim. This image is Debian-based, so it is approximately 20 MB larger than the alpine variant, but it has the binaries required to run sharp.
When using a similar Dockerfile to yours, I found that the NEXT_SHARP_PATH environment variable was not needed when using a Debian-based Node image.
And for reference, here is the NextJS documentation about the error message: https://nextjs.org/docs/messages/sharp-missing-in-production
Update: You may also be able to specify the libc implementation found in your base Docker image by using the following flags:
RUN npm_config_platform=linux npm_config_arch=x64 npm_config_libc=glibc npm ci
For more information about what these flags are, see the documentation for sharp.

Dockerize NextJS Application with Prisma

I have created a NextJS application, to connect to the database I use Prisma. When I start the application on my computer everything works. Unfortunately, I get error messages when I try to run the application in a Docker container. The container can be created and started. The start page of the application can also be shown (there are no database queries there yet). However, when I click on the first page where there is a database query I get error code 500 - Initial Server Error and the following error message in the console:
PrismaClientInitializationError: Unknown PRISMA_QUERY_ENGINE_LIBRARY undefined. Possible binaryTargets: darwin, darwin-arm64, debian-openssl-1.0.x, debian-openssl-1.1.x, rhel-openssl-1.0.x, rhel-openssl-1.1.x, linux-arm64-openssl-1.1.x, linux-arm64-openssl-1.0.x, linux-arm-openssl-1.1.x, linux-arm-openssl-1.0.x, linux-musl, linux-nixos, windows, freebsd11, freebsd12, openbsd, netbsd, arm, native or a path to the query engine library.
You may have to run prisma generate for your changes to take effect.
at cb (/usr/src/node_modules/#prisma/client/runtime/index.js:38689:17)
at async getServerSideProps (/usr/src/.next/server/pages/admin/admin.js:199:20)
at async Object.renderToHTML (/usr/src/node_modules/next/dist/server/render.js:428:24)
at async doRender (/usr/src/node_modules/next/dist/server/next-server.js:1144:38)
at async /usr/src/node_modules/next/dist/server/next-server.js:1236:28
at async /usr/src/node_modules/next/dist/server/response-cache.js:64:36 {
clientVersion: '3.6.0',
errorCode: undefined
}
My Dockerfile:
# Dockerfile
# base image
FROM node:16-alpine3.12
# create & set working directory
RUN mkdir -p /usr/src
WORKDIR /usr/src
# copy source files
COPY . /usr/src
COPY package*.json ./
COPY prisma ./prisma/
# install dependencies
RUN npm install
COPY . .
# start app
RUN npm run build
EXPOSE 3000
CMD npm run start
My docker-compose.yaml:
version: "3"
services:
web:
build:
context: .
dockerfile: Dockerfile
container_name: web
restart: always
volumes:
- ./:/usr/src/app
ports:
- "3000:3000"
env_file:
- .env
My package.json:
{
"name": "supermarket",
"version": "0.1.0",
"private": true,
"scripts": {
"dev": "next dev",
"build": "next build",
"start": "next start",
"lint": "next lint"
},
"prisma": {
"schema": "prisma/schema.prisma"
},
"dependencies": {
"#prisma/client": "^3.6.0",
"axios": "^0.22.0",
"cookie": "^0.4.1",
"next": "latest",
"nodemailer": "^6.6.5",
"react": "17.0.2",
"react-cookie": "^4.1.1",
"react-dom": "17.0.2"
},
"devDependencies": {
"eslint": "7.32.0",
"eslint-config-next": "11.1.2",
"prisma": "^3.6.0"
}
}
I've found the error. I think it's a problem with the M1 Chip.
I changed node:16-alpine3.12 to node:lts and added some commands to the Dockerfile which looks like this now:
# base image
FROM node:lts
# create & set working directory
RUN mkdir -p /usr/src
WORKDIR /usr/src
# copy source files
COPY . /usr/src
COPY package*.json ./
COPY prisma ./prisma/
RUN apt-get -qy update && apt-get -qy install openssl
# install dependencies
RUN npm install
RUN npm install #prisma/client
COPY . .
RUN npx prisma generate --schema ./prisma/schema.prisma
# start app
RUN npm run build
EXPOSE 3000
CMD npm run start
I hope this can also help other people 😊
I have been having a similar issue, which I have just solved.
I think what you need to do is change the last block in your docker file to this
# start app
RUN npm run build
RUN npx prism generate
EXPOSE 3000
CMD npm run start
I think that will solve your issue.
I've found this solution with some workarounds:
https://gist.github.com/malteneuss/a7fafae22ea81e778654f72c16fe58d3
In short:
# Dockerfile
...
FROM node:16-alpine AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
RUN npx prisma generate # <---important to support Prisma query engine in Alpine Linux in final image
RUN npm run build
# Production image, copy all the files and run next
FROM node:16-alpine AS runner
WORKDIR /app
...
COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static
COPY --chown=nextjs:nodejs prisma ./prisma/ # <---important to support Prisma DB migrations in bootstrap.sh
COPY --chown=nextjs:nodejs bootstrap.sh ./
...
CMD ["./bootstrap.sh"]
This Dockerfile is based on the official Nextjs with Docker example project and adapted to include Prisma. To run migrations on app start we can add a bash script that does so:
# bootstrap.sh
#!/bin/sh
# Run migrations
DATABASE_URL="postgres://postgres:postgres#db:5432/appdb?sslmode=disable" npx prisma migrate deploy
# start app
DATABASE_URL="postgres://postgres:postgres#db:5432/workler?sslmode=disable" node server.js
Unfortunately, we need to explicitly set the DATABASE_URL here, otherwise migrations don't work, because Prisma can't find the environment variable (e.g. from a docker-compose file).
And last but not least, because Alpine Linux base image uses a Musl C-library, the Prisma client has to be compiled in the builder image against that. So, to get the correct version, we need to add this info to Prisma's schema.prisma file:
# schema.prisma
generator client {
provider = "prisma-client-js"
binaryTargets = ["native", "linux-musl"] # <---- important to support Prisma Query engine in Alpine linux, otherwise "PrismaClientInitializationError2 [PrismaClientInitializationError]: Query engine binary for current platform "linux-musl" could not be found."
}
I had a luck this way:
FROM node:17-slim as dependencies
# set working directory
WORKDIR /usr/src/app
# Copy package and lockfile
COPY package.json ./
COPY yarn.lock ./
COPY prisma ./prisma/
RUN apt-get -qy update && apt-get -qy install openssl
# install dependencies
RUN yarn --frozen-lockfile
COPY . .
# ---- Build ----
FROM dependencies as build
# install all dependencies
# build project
RUN yarn build
# ---- Release ----
FROM dependencies as release
# copy build
COPY --from=build /usr/src/app/.next ./.next
COPY --from=build /usr/src/app/public ./public
# dont run as root
USER node
# expose and set port number to 3000
EXPOSE 3000
ENV PORT 3000
# enable run as production
ENV NODE_ENV=production
# start app
CMD ["yarn", "start"]

How to avoid node_modules folder being deleted

I'm trying to create a Docker container to act as a test environment for my application. I am using the following Dockerfile:
FROM node:14.4.0-alpine
WORKDIR /test
COPY package*.json ./
RUN npm install .
CMD [ "npm", "test" ]
As you can see, it's pretty simple. I only want to install all dependencies but NOT copy the code, because I will run that container with the following command:
docker run -v `pwd`:/test -t <image-name>
But the problem is that node_modules directory is deleted when I mount the volume with -v. Any workaround to fix this?
When you bind mount test directory with $PWD, you container test directory will be overridden/mounted with $PWD. So you will not get your node_modules in test directory anymore.
To fix this issue you can use two options.
You can run npm install in separate directory like /node and mount your code in test directory and export node_path env like export NODE_PATH=/node/node_modules
then Dockerfile will be like:
FROM node:14.4.0-alpine
WORKDIR /node
COPY package*.json ./
RUN npm install .
WORKDIR /test
CMD [ "npm", "test" ]
Or you can write a entrypoint.sh script that will copy the node_modules folder to the test directory at the container runtime.
FROM node:14.4.0-alpine
WORKDIR /node
COPY package*.json ./
RUN npm install .
WORKDIR /test
COPY Entrypoint.sh ./
ENTRYPOINT ["Entrypoint.sh"]
and Entrypoint.sh is something like
#!/bin/bash
cp -r /node/node_modules /test/.
npm test
Approach 1
A workaround is you can do
CMD npm install && npm run dev
Approach 2
Have docker install node_modules on docker-compose build and run the app on docker-compose up.
Folder Structure
docker-compose.yml
version: '3.5'
services:
api:
container_name: /$CONTAINER_FOLDER
build: ./$LOCAL_FOLDER
hostname: api
volumes:
# map local to remote folder, exclude node_modules
- ./$LOCAL_FOLDER:/$CONTAINER_FOLDER
- /$CONTAINER_FOLDER/node_modules
expose:
- 88
Dockerfile
FROM node:14.4.0-alpine
WORKDIR /test
COPY ./package.json .
RUN npm install
# run command
CMD npm run dev

Why webpack bundle is successfully built and run without multi-stage docker build?

I have the following Dockerfile:
FROM node:12.4.0 as builder
COPY ./package.json /src/package.json
ENV PORT 3000
WORKDIR /src
RUN npm install
COPY ./lerna.json /src/lerna.json
COPY ./packages/package-api/package.json /src/packages/package-api/package.json
COPY ./packages/package-config/package.json /src/packages/package-config/package.json
COPY ./packages/package-sitemap/package.json /src/packages/package-sitemap/package.json
COPY ./packages/package-utils/package.json /src/packages/package-utils/package.json
RUN npm run clean
COPY . /src
WORKDIR /src/packages/package-sitemap
RUN npm run build
FROM node:12.4.0
RUN echo "starting 2nd stage"
WORKDIR /src/packages/package-sitemap
COPY --from=builder /src/packages/package-sitemap/build/bundle.js .
EXPOSE ${PORT}
CMD ["node" , "bundle.js"]
running the bundle built from such Dockerfile results in following error: Error: Cannot find module 'debug'.
However if I remove the second stage and build the bundle in one stage there's no error and the bundle runs successfully, that is with the following Dockerfile:
FROM node:12.4.0 as builder
COPY ./package.json /src/package.json
ENV PORT 3000
WORKDIR /src
RUN npm install
COPY ./lerna.json /src/lerna.json
COPY ./packages/package-api/package.json /src/packages/package-api/package.json
COPY ./packages/package-config/package.json /src/packages/package-config/package.json
COPY ./packages/package-sitemap/package.json /src/packages/package-sitemap/package.json
COPY ./packages/package-utils/package.json /src/packages/package-utils/package.json
RUN npm run clean
COPY . /src
WORKDIR /src/packages/leaf-sitemap
RUN npm run build
EXPOSE ${PORT}
CMD ["node" , "build/bundle.js"]
I'm wondering what the difference can be? Is it because in the 2nd Dockerfile node_modules exist as well and webpack somehow sets paths to some modules in the bundle to the node_modules?

Resources