Docker image build fails: "protoc-gen-grpc-web: program not found or is not executable" - docker

i inherited a project with several microservices running on kubernetes. after copying the repo and running the steps that the previous team outlined, i have an issue building one of the images that i need to deploy. the script for the build is:
cd graph_endpoint
cp ../../Protobufs/Graph_Endpoint/graph_endpoint.proto .
protoc -I. graph_endpoint.proto --js_out=import_style=commonjs:.
protoc -I. graph_endpoint.proto --grpc-web_out=import_style=commonjs,mode=grpcwebtext:.
export NODE_OPTIONS=--openssl-legacy-provider
npx webpack ./test.js --mode development
cp ./dist/graph_endpoint.js ../public/graph_endpoint.js
cd ..
docker build . -t $1/canvas-lti-frontend:v2
docker push $1/canvas-lti-frontend:v2
i'm getting an error from line 4:
protoc-gen-grpc-web: program not found or is not executable
--grpc-web_out: protoc-gen-grpc-web: Plugin failed with status code 1.
any idea how to fix it? i have no prior experience with docker.
here's the Dockerfile:
FROM node:16
# Install app dependencies
COPY package*.json /frontend-app/
WORKDIR /frontend-app
RUN npm install
COPY server.js /frontend-app/
# Bundle app source
COPY public /frontend-app/public
COPY routes /frontend-app/routes
COPY controllers /frontend-app/controllers
WORKDIR /frontend-app
EXPOSE 3000
CMD [ "node", "server.js"]
and package.json:
{
"name": "frontend",
"version": "1.0.0",
"description": "The user-facing application for the Canvas LTI Student Climate Dashboard",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "",
"license": "ISC",
"dependencies": {
"#okta/oidc-middleware": "^4.3.0",
"#okta/okta-signin-widget": "^5.14.0",
"express": "^4.18.2",
"express-session": "^1.17.2",
"vue": "^2.6.14"
},
"devDependencies": {
"nodemon": "^2.0.20",
"protoc-gen-grpc-web": "^1.4.1"
}
}

You don't have protoc-gen-grpc-web installed on the machine on which you're running the build script.
You can download the plugins from the grpc-web repo's releases page.
protoc has a plugin mechanism.
protoc looks for its plugins in the path and expects these binaries to be prefixed protoc-gen-{foo}.
However, when you reference the plugin from protoc, you simply use {foo} and generally this is suffixed with _out and sometimes _opt, i.e. protoc ... --{foo}_out --{foo}_opt.
The plugin protoc-gen-grpc-web (once installed and accessible in the host's path) is thus referenced with protoc ... --grpc_web_out=...

Related

Building devcontainer with --ssh key for GitHub repository in build process fails on VS Code for ARM Mac

We are trying to run a python application using a devcontainer.json with VS Code.
The Dockerfile includes the installation of GitHub repositories with pip that require an ssh key. To build the images, we usually use the --ssh flag to pass the required key. We then use this key to run pip inside the Dockerfile as follows:
RUN --mount=type=ssh,id=ssh_key python3.9 -m pip install --no-cache-dir -r pip-requirements.txt
We now want to run a devcontainer.json inside VS Code. We have been trying many different ways.
1. Passing the --ssh key using the build arg variable:
Since you can not directly pass the --ssh key, we tried a workaround:
"args": {"kek":"kek --platform=linux/amd64 --ssh ssh_key=/Users/user/.ssh/id_rsa"}
This produces an OK looking build command that works in a normal terminal, but inside VS Code the key is not being passed and the build fails. (Both on Windows & Mac)
2. Putting an initial build command into the initializeCommand parameter and then a simple build command that should use the cached results:
We run a first build inside the initializeCommand parameter:
"initializeCommand": "docker buildx build --platform=linux/amd64 --ssh ssh_key=/Users/user/.ssh/id_rsa ."
and then we have a second build in the regular parameter:
"build": {
"dockerfile": "../Dockerfile",
"context": "..",
"args": {"kek":"kek --platform=linux/amd64"}
}
This solution is a nice workaround and works flawlessly on Windows. On the ARM Mac, however, only the initializeCommand build stage runs well, the actual build fails, as it does not use the cached version of the images. The crucial step when the --ssh key is used, fails just like described before.
We have no idea why the Mac VS Code ignores the already created images. In a regular terminal, again, the second build command generated by VS Code works flawlessly.
The problem is reproducible on different ARM Macs, and on different repositories.
Here is the entire devcontainer:
{
"name": "Dockername",
"build": {
"dockerfile": "../Dockerfile",
"context": "..",
"args": {"kek":"kek --platform=linux/amd64"}
},
"initializeCommand": "docker buildx build --platform=linux/amd64 --ssh ssh_key=/Users/user/.ssh/id_rsa .",
"runArgs": ["--env-file", "configuration.env", "-t"],
"customizations": {
"vscode": {
"extensions": [
"ms-python.python"
]
}
}
}
So, we finally found a work around:
We add a target to the initialize command:
"initializeCommand": "docker buildx build --platform=linux/amd64 --ssh ssh_key=/Users/user/.ssh/id_rsa -t dev-image ."
We create a new Dockerfile Dockerfile-devcontainer that only uses one line:
FROM --platform=linux/amd64 docker.io/library/dev-image:latest
In the build command of the devcontainer use that Dockerfile:
"name": "Docker",
"initializeCommand": "docker buildx build --platform=linux/amd64 --ssh ssh_key=/Users/user/.ssh/id_rsa -t dev-image:latest .",
"build": {
"dockerfile": "Dockerfile-devcontainer",
"context": "..",
"args": {"kek":"kek --platform=linux/amd64"}
},
"runArgs": ["--env-file", "configuration.env"],
"customizations": {
"vscode": {
"extensions": [
"ms-python.python"
]
}
}
}
In this way we can use the .ssh key and the docker image created in the initializeCommand (Tested on MacOS and Windows).

Combined JSON & Mochawesome test report not generating during Cypress tests in Docker container?

I am running Cypress tests inside a Docker container to generate a HTML test report.
Here is my folder structure:
As you can see in the cypress/reports/mocha folder, there are some JSON test results generated.
All tests are passing & the 3 JSON files there are populated.
Also, notice the empty cypress/reports/mochareports folder. This should contain the combined JSON of all test results, & a HTML test report.
Here is my package.json:
{
"name": "cypress-docker",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"clean:reports": "mkdir -p cypress/reports && rm -R -f cypress/reports/* && mkdir cypress/reports/mochareports",
"pretest": "npm run clean:reports",
"scripts": "cypress run",
"chrome:scripts": "cypress run --browser chrome ",
"firefox:scripts": "cypress run --browser firefox ",
"combine-reports": "mochawesome-merge cypress/reports/mocha/*.json > cypress/reports/mochareports/report.json",
"generate-report": "marge cypress/reports/mochareports/report.json -f report -o cypress/reports/mochareports",
"posttest": "npm run combine-reports && npm run generate-report",
"test": "npm run scripts || npm run posttest",
"chrome:test": "npm run pretest && npm run chrome:scripts || npm run posttest",
"firefox:test": "npm run pretest && npm run firefox:scripts || npm run posttest"
},
"keywords": [],
"author": "QA BOX <qabox#gmail.com>",
"license": "MIT",
"dependencies": {
"cypress": "^6.8.0",
"cypress-multi-reporters": "^1.4.0",
"mocha": "^8.2.1",
"mochawesome": "^6.2.1",
"mochawesome-merge": "^4.2.0",
"mochawesome-report-generator": "^5.1.0"
}
}
Here is my cypress.json:
{
"reporter": "cypress-multi-reporters",
"reporterOptions": {
"reporterEnabled": "mochawesome",
"mochawesomeReporterOptions": {
"reportDir": "cypress/reports/mocha",
"quite": true,
"overwrite": false,
"html": false,
"json": true
}
}
}
Here are the commands I use to run the tests:
To build the image - docker build -t cyp-dock-mocha-report .
docker-compose run e2e-chrome
Here is my Dockerfile:
FROM cypress/included:6.8.0
RUN mkdir /cypress-docker
WORKDIR /cypress-docker
COPY ./package.json .
COPY ./package-lock.json .
COPY ./cypress.json .
COPY ./cypress ./cypress
RUN npm install
ENTRYPOINT ["npm", "run"]
Here is my docker-compose.yml:
version: "3"
services:
# this container will run Cypress test using built-in Electron browser
e2e-electron:
image: "cyp-dock-mocha-report"
command: "test"
volumes:
- ./cypress/videos:/cypress-docker/cypress/videos
- ./cypress/reports:/cypress-docker/cypress/reports
# this container will run Cypress test using Chrome browser
e2e-chrome:
image: "cyp-dock-mocha-report"
command: "chrome:test"
volumes:
- ./cypress/videos:/cypress-docker/cypress/videos
- ./cypress/reports:/cypress-docker/cypress/reports
# this container will run Cypress test using Firefox browser
# note that both Chrome and Firefox browsers were pre-installed in the Docker image
e2e-firefox:
image: "cyp-dock-mocha-report"
command: "firefox:test"
# if you want to debug FF run, pass DEBUG variable like
environment:
- DEBUG=cypress:server:browsers:firefox-util,cypress:server:util:process_profiler
volumes:
- ./cypress/videos:/cypress-docker/cypress/videos
- ./cypress/reports:/cypress-docker/cypress/reports
All tests are passing as you can see below:
I don't know why the Mochawesome HTML test report isn't generating, or the merged JSO
Can someone please tell me why the merged JSON & the HTML test report aren't being generated in mochareports folder, & how I can get them to?
thanks for giving me a hint on how to use docker compose with this image! I think I see where the issue is: in the package.json file, under scripts, instead of "merge", you wrote "marge":
"generate-report": "marge cypress/reports/mochareports/report.json -f report -o cypress/reports/mochareports"

It seems you are running Vue CLI inside a container

I m trying to run my vuejs app using vs-code remote-containers. Its deployed and I can access it via the url: localhost:8080/ but If I update some js file, its not re-compiling and even not hot-reloading.
devcontainer.json
{
"name": "Aquawebvue",
"dockerFile": "Dockerfile",
"appPort": [3000],
"runArgs": ["-u", "node"],
"settings": {
"workbench.colorTheme": "Cobalt2",
"terminal.integrated.automationShell.linux": "/bin/bash"
},
"postCreateCommand": "yarn",
"extensions": [
"esbenp.prettier-vscode",
"wesbos.theme-cobalt2",
]
}
Dockerfile
FROM node:12.13.0
RUN npm install -g prettier
After opening container and running cmd 'yarn serve' in terminal it builds and deploy successfully but I got this warning:
It seems you are running Vue CLI inside a container.
Since you are using a non-root publicPath, the hot-reload socket
will not be able to infer the correct URL to connect. You should
explicitly specify the URL via devServer.public.
VSCode has a pre-defined .devcontainer directory for Vue projects. It can be found on GitHub. You can install it automatically by running the command Add Development Container Configuration Files... > Show All Definitions > Vue.
Dockerfile
# [Choice] Node.js version: 14, 12, 10
ARG VARIANT=14
FROM mcr.microsoft.com/vscode/devcontainers/javascript-node:0-${VARIANT}
RUN su node -c "umask 0002 && npm install -g http-server #vue/cli #vue/cli-service-global"
WORKDIR /app
EXPOSE 8080
# [Optional] Uncomment this section to install additional OS packages.
# RUN apt-get update && export DEBIAN_FRONTEND=noninteractive \
# && apt-get -y install --no-install-recommends <your-package-list-here>
devcontainer.json
{
"name": "Vue (Community)",
"build": {
"dockerfile": "Dockerfile",
"context": "..",
// Update 'VARIANT' to pick a Node version: 10, 12, 14
"args": { "VARIANT": "14" }
},
// Set *default* container specific settings.json values on container create.
"settings": {
"terminal.integrated.shell.linux": "/bin/zsh"
},
// Add the IDs of extensions you want installed when the container is created.
"extensions": [
"dbaeumer.vscode-eslint",
"octref.vetur"
],
// Use 'forwardPorts' to make a list of ports inside the container available locally.
"forwardPorts": [
8080
],
// Use 'postCreateCommand' to run commands after the container is created.
// "postCreateCommand": "uname -a",
// Comment out connect as root instead. More info: https://aka.ms/vscode-remote/containers/non-root.
"remoteUser": "node"
}

Rails/React/Yarn: How can I run "yarn start" from the root of my application

I'm creating a rails application from scratch using the following blog which needs me to run rails s and then yarn start in the /client folder of the application (where all the JS and React components will live).
The /client/package.json file has some scripts configured to run a server that detects changes in my react components and reloads the components automatically:
client/package.json:
{
"name": "client",
"version": "0.1.0",
"private": true,
"dependencies": {
"react": "^16.8.6",
"react-dom": "^16.8.6",
"react-scripts": "3.0.1"
},
"scripts": {
"start": "react-scripts start",
"build": "react-scripts build",
"test": "react-scripts test",
"eject": "react-scripts eject"
},
"eslintConfig": {
"extends": "react-app"
},
"browserslist": {
"production": [
">0.2%",
"not dead",
"not op_mini all"
],
"development": [
"last 1 chrome version",
"last 1 firefox version",
"last 1 safari version"
]
},
"proxy": "http://localhost:3001"
}
I use a program called hivemind which is a Procfile manager and I would like both the rails server process and the yarn process to run together in the same terminal.
I'd like to do something like this in my Profile:
server: bin/rails server
react: yarn start
The problem is that Profile lives in the root of my application and I have to change directory into client and then run yarn start.
TL;DR:
Is there an option with yarn run that you can tell it to run from another folder or read a package.json file in another folder and have it run the script in that file?
You can write cmd arguments in Procfile
try changing react command to
react: cd client && yarn start
It will change the current directory and runs your script
When you run yarn [command] yarn looks for a file locally called package.json. You can add yarn commands inside the "scripts" section of this file. In my own rails app root directory I added a "start" command that did what Sumanth's answer does but now I can just have
react: yarn start
in my Procfile.
Below is the edited version of my package.json file in my app's root path:
{
"name": "create-repack-app",
"version": "1.0.0",
"scripts": {
"build": "cd client && npm install --only=dev && npm install && npm run build && cd ..",
"deploy": "cp -a client/build/. public/",
"heroku-postbuild": "npm run build && npm run deploy && echo 'Client Built'",
"start": "cd client && yarn start"
},
"keywords": [],
"author": "",
"license": "ISC"
}

Packer fails my docker build with error "sudo: not found" despite sudo being present

I'm trying to build a packer image with docker on it and I want docker to create a docker image with a custom script. The relevant portion of my code is (note that the top builder double-checks that sudo is installed):
{
"type": "shell",
"inline": [
"apt-get install sudo"
]
},
{
"type": "docker",
"image": "python:3",
"commit": true,
"changes": [
"RUN pip install Flask",
"CMD [\"python\", \"echo.py\"]"
]
}
The relevant portion of my screen output is:
==> docker: provisioning with shell script: /var/folders/s8/g1_gobbldygook/T/packer-shell23453453245
docker: /temp/script_1234.sh: 3: /tmp/script_1234.sh: sudo: not found
==> docker: killing the contaner: 34234hashvomit234234
Build 'docker' errored: Scipt exited with non-zero exit status: 127
The script in question is not one of mine. It's some randomly generated script that has a different series of four numbers every time I build. I'm new to both packer and docker, so maybe it's obvious what the problem is, but it's not to me.
There seem to be a few problems with your packer input. Since you haven't included the complete input file it's hard to tell, but notice a couple of things:
You probably need to run apt-get update before calling apt-get install sudo. Without that, even if the image has cached package metadata it is probably stale. If I try to build an image using your input it fails with:
E: Unable to locate package sudo
While not a problem in this context, it's good to explicitly include -y on the apt-get command line when you're running it non-interactively:
apt-get -y install sudo
In situations where apt-get is attached to a terminal, this will prevent it from prompting for confirmation. This is not a necessary change to your input, but I figure it's good to be explicit.
Based on the docs and on my testing, you can't include a RUN statement in the changes block of a docker builder. That fails with:
Stderr: Error response from daemon: run is not a valid change command
Fortunately, we can move that pip install command into a shell provisioner.
With those changes, the following input successfully builds an image:
{
"builders": [{
"type": "docker",
"image": "python:3",
"commit": true
}],
"provisioners": [{
"type": "shell",
"inline": [
"apt-get update",
"apt-get -y install sudo",
"pip install Flask"
]
}],
"post-processors": [[ {
"type": "docker-tag",
"repository": "packer-test",
"tag": "latest"
} ]]
}
(NB: Tested using Packer v1.3.5)

Resources