Google cloud ruby deployment and ruby-docker - ruby-on-rails

I am trying to put my rails project on the google cloud engine for the first time and I have a lot of trouble.
I've wanted to upload my project with a custom runtime app.yaml (because I would like yarn to install the dependencies as well), but the deployment command fails with this error:
Error Response: [4] Your deployment has failed to become healthy in the allotted time and therefore was rolled back. If you believe this was an error, try adjusting the 'app_start_timeout_sec' setting in the 'readiness_check' section.
PS: the app runs locally (development and production env).
My app.yaml looks like this:
entrypoint: bundle exec rails s -b '0.0.0.0' --port $PORT
env: flex
runtime: custom
env_variables:
My Environment variables
beta_settings:
cloud_sql_instances: ekoma-app:us-central1:ekoma-db
readiness_check:
path: "/_ah/health"
check_interval_sec: 5
timeout_sec: 4
failure_threshold: 2
success_threshold: 1
app_start_timeout_sec: 120
And my Dockerfile looks like this:
FROM l.gcr.io/google/ruby:latest
RUN apt-get update -qq && apt-get install apt-transport-https
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
RUN echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
RUN apt-get update -qq && apt-get install -y build-essential libpq-dev imagemagick yarn
WORKDIR /app
COPY Gemfile /app/Gemfile
COPY Gemfile.lock /app/Gemfile.lock
COPY package.json /app/package.json
COPY yarn.lock /app/yarn.lock
RUN gem install pkg-config -v "~> 1.1"
RUN bundle install && npm install
COPY . /app
When deploying with a ruby runtime I realized that the dockerfile generated was much more complex and probably complete and google provide a repo to generate it.
So, I tried to look into the ruby-docker public repo that google shared but I don't know how to use their generated docker images and therefore fix my Dockerfile issue
https://github.com/GoogleCloudPlatform/ruby-docker
Could someone help me figure what's wrong in my setup and how to run these ruby-docker image (seems very useful!)?
Thank you!

The "entrypoint" field in app.yaml is not used when a custom runtime is in play. Instead, set the CMD in your Dockerfile. e.g.:
CMD ["bundle", "exec", "rails", "s", "-b", "0.0.0.0", "--port", "8080"]
That probably will get your application running. (Remember that environment variables are not interpolated in exec form, so I replaced your $PORT with the hard-coded port 8080, which is the port App Engine expects.)
As an alternative:
It may be possible to use the Ruby runtime images in the ruby-docker repo, and not have to use a custom runtime (i.e. you may not need to write your own Dockerfile), even if you have custom build steps like doing yarn installs. Most of the build process in runtime: ruby is customizable, but it's not well-documented. If you want to try this path, the TL;DR is:
Use runtime: ruby in your app.yaml and don't provide your own Dockerfile. (And reinstate the entrypoint of course.)
If you want to install ubuntu packages not normally present in runtime: ruby, list them in app.yaml under runtime_config:packages. For example:
runtime_config:
packages:
- libgeos-dev
- libproj-dev
If you want to run custom build steps, list them in app.yaml under runtime_config:build. They get executed in the Dockerfile after the bundle install step (which cannot itself be modified). For example:
runtime_config:
build:
- npm install
- bundle exec rake assets:precompile
- bundle exec rake setup_my_stuff
Note that by default, if you don't provide custom build steps, the ruby runtime behaves as if there is one build step: bundle exec rake assets:precompile || true. That is, by default, runtime: ruby will attempt to compile your assets during app engine deployment. If you do modify the build steps and you want to keep this behavior, make sure you include that rake task as part of your custom build steps.

Related

Dockerizing Nuxt 3 app for development purposes

I'm trying to dockerize Nuxt 3 app, but I have strange issue.
This Dockerfile is working with this docker run command:
docker run -v /Users/my_name/developer/nuxt-app:/app -it -p 3000:3000 nuxt-app
# Dockerfile
FROM node:16-alpine3.14
# create destination directory
RUN mkdir -p /usr/src/nuxt-app
WORKDIR /usr/src/nuxt-app
# update and install dependency
RUN apk update && apk upgrade
RUN apk add git
# copy the app, note .dockerignore
COPY . /usr/src/nuxt-app/
RUN npm install
# RUN npm run build
EXPOSE 3000
# ENV NUXT_HOST=0.0.0.0
# ENV NUXT_PORT=3000
CMD [ "npm", "run", "dev"]
I don't understand why despite mounting it to /app folder in the container and declaring /usr/src/nuxt-app in Dockerfile it works.
When I try to match them then I get this error:
ERROR (node:18) PromiseRejectionHandledWarning: Promise rejection was handled asynchronously (rejection id: 3) 20:09:42
(Use `node --trace-warnings ...` to show where the warning was created)
✔ Nitro built in 571 ms nitro 20:09:43
ERROR [unhandledRejection]
You installed esbuild for another platform than the one you're currently using.
This won't work because esbuild is written with native code and needs to
install a platform-specific binary executable.
Specifically the "#esbuild/darwin-arm64" package is present but this platform
needs the "#esbuild/linux-arm64" package instead. People often get into this
situation by installing esbuild on Windows or macOS and copying "node_modules"
into a Docker image that runs Linux, or by copying "node_modules" between
Windows and WSL environments.
If you are installing with npm, you can try not copying the "node_modules"
directory when you copy the files over, and running "npm ci" or "npm install"
on the destination platform after the copy. Or you could consider using yarn
instead of npm which has built-in support for installing a package on multiple
platforms simultaneously.
If you are installing with yarn, you can try listing both this platform and the
other platform in your ".yarnrc.yml" file using the "supportedArchitectures"
feature: https://yarnpkg.com/configuration/yarnrc/#supportedArchitectures
Keep in mind that this means multiple copies of esbuild will be present.
Another alternative is to use the "esbuild-wasm" package instead, which works
the same way on all platforms. But it comes with a heavy performance cost and
can sometimes be 10x slower than the "esbuild" package, so you may also not
want to do that.
at generateBinPath (node_modules/vite/node_modules/esbuild/lib/main.js:1841:17)
at esbuildCommandAndArgs (node_modules/vite/node_modules/esbuild/lib/main.js:1922:33)
at ensureServiceIsRunning (node_modules/vite/node_modules/esbuild/lib/main.js:2087:25)
at build (node_modules/vite/node_modules/esbuild/lib/main.js:1978:26)
at runOptimizeDeps (node_modules/vite/dist/node/chunks/dep-3007b26d.js:42941:26)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
I have absolutely no clue what is going on here except architecture mismatch (that doesn't seem to be the case with working version - I'm on MacBook Air M1).
The second issue is that mounting doesn't update the page.
Okay, found the way. The issue was with Vite because HMR module is using port 24678 and I didn't expose it so it couldn't reload the page. This is how it should be looking:
docker run --rm -it \
-v /path/to/your/app/locally:/usr/src/app \
-p 3000:3000 \
-p 24678:24678 \
nuxt-app
Dockerfile
FROM node:lts-slim
WORKDIR /usr/src/app
CMD ["sh", "-c", "npm install && npm run dev"]

How to get wkhtmltopdf with a Dockerized Rails App with running ruby:3.0.4-alpine for use with Wicked PDF

I'm trying to upgrade a Dockerized Rails app to Ruby 3+ and Rails 7. Due to other project dependencies, I've landed on this version of Ruby Docker image ruby:3.0.4-alpine. Here's my Dockerfile:
FROM ruby:3.0.4-alpine
RUN apk --update add --no-cache build-base bash git vim curl postgresql-dev openssl-dev nodejs npm yarn tzdata less \
imagemagick postgresql-client gcompat
RUN mkdir /app
WORKDIR /app
COPY Gemfile Gemfile.lock package.json yarn.lock ./
RUN yarn set version berry
RUN bundle update --bundler
RUN bundle install --jobs 5
ADD . /app
RUN yarn install
RUN yarn install --frozen-lockfile \
&& RAILS_SECRET_KEY_BASE=secret_key_base RAILS_ENV=production bundle exec rails assets:precompile apipie:cache \
&& rm -rf node_modules
EXPOSE 5000
VOLUME ["/app/public", "/usr/local/bundle"]
CMD bash -c "bundle exec puma -C config/puma.rb"
At this point the only dependency in my Rails app that doesn't work with this Dockerfile is wkhtmltopdf.
I would prefer to install wkhtmltopdf with the Alpine Package Keeper (apk) as part of RUN apk --update add --no-cache.
But, it appears that the latest version of Alpine that has the wkhtmltopdf package available is Alpine 3.14. The release notes for Alpine 3.15 state "qt5-qtwebkit, kdewebkit, wkhtmltopdf, and py3-pdfkit have been removed due to known vulnerabilities and lack of upstream support for qtwebkit."
So other than switching off of Alpine Linux, which would present other dependency issues. I would prefer not to do as that seems to be the version used by most Dockerized Rails apps for the foreseeable future, what are my options?
I also tried using the wkhtmltopdf_binary gem, but that was last updated in 2016. When I try to export a PDF using wicked_pdf with wkhtmltopdf installed by that gem, I get this error:
/usr/local/bundle/gems/wkhtmltopdf-binary-0.12.6.5/bin/wkhtmltopdf:61:in `<top (required)>': Invalid platform, must be running on Ubuntu
16.04/18.04/20.04 CentOS 6/7/8, Debian 9/10, archlinux amd64, or intel-based Cocoa macOS (missing binary: /usr/local/bundle/gems/wkhtmltopdf-binary-0.12.6.5/bin/wkhtmltopdf_alpine_3.16.0_i386). (RuntimeError) from /usr/local/bundle/bin/wkhtmltopdf:25:in `load' from /usr/local/bundle/bin/wkhtmltopdf:25:in `<main>'
I believe I'm getting this error because the docker image ruby:3.0.4-alpine is running Alpine 3.16 and Alpine dropped support for wkhtmltopdf on version 3.14. The wkhtmltopdf_binary gem hasn't been updated since 2016, so it wouldn't target more recent versions of Alpine.
So, what are my options to work around this? Is there a way to script a compatible installation of the wkhtmltopdf in my Dockerfile? If so, how would I do that?
I haven't been able to find another gem that targets more recent versions of Alpine that install wkhtmltopdf. Is there one out there that I'm missing?
This has to be a common problem (Rails app running on Docker Alpine image with Ruby 3+ that needs to export PDFs using wkhtmltopdf).
Or, is there another way to export PDFs in ruby that doesn't depend on wkhtmltopdf that I haven't found? (I've found other NodeJS options, but that introduces a whole other set of dependencies, so a Docker/Ruby solution is preferable.)

Unable to provide config files to puma in kubernetes using command/args section of kubernets Deployment yam file

What could be the reason of Deployment not being able to see config files?
This is a part from Deployment
command: ["bundle", "exec", "puma", "-C", "config/puma.rb"]
already tried with ./config/.. and using args instead of command
I'm getting Errno::ENOENT: No such file or directory # rb_sysopen - config/puma.rb
Everything used to work fine with docker-compose
When I keep the last line (CMD) from the Dockerfile below and omit the command: in Deployment, everything works fine but, to reuse the image for sidekiq, I need to provide config files.
Dockerfile
FROM ruby:2.7.2
RUN apt-get update -qq && apt-get install -y build-essential ca-certificates libpq-dev nodejs postgresql-client yarn vim -y
ENV APP_ROOT /var/www/app
RUN mkdir -p $APP_ROOT
WORKDIR $APP_ROOT
COPY Gemfile Gemfile
COPY Gemfile.lock Gemfile.lock
COPY public public/
RUN gem install bundler
RUN bundle install
# tried this
COPY config config/
COPY . .
EXPOSE 9292
# used to have this line but I want to reuse the image
# CMD ["bundle", "exec", "puma", "-C", "config/puma.rb"]
error message
bundler: failed to load command: puma (/usr/local/bundle/bin/puma)
Errno::ENOENT: No such file or directory # rb_sysopen - config/puma.rb
upd
It seems that the issue was related to wrong paths and misunderstanding of command and args fields. The following config worked for me. It also possible there were cache issues with docker(happened to me earlier)
command:
- bundle
- exec
- puma
args:
- "-C"
- "config/puma.rb"
For some reason providing commands inside of values.yaml doesn't seem to work properly.
But it works when commands are provided through template.
There's the following section in the app/templates/deployment.yaml of my app. Everything works fine now.
containers:
- name: {{ .Values.app.name }}
image: {{ .Values.app.container.image }}
command:
- bundle
- exec
- puma
args:
- "-C"
- "config/puma.rb"
I have also found this k8s rails demo https://github.com/lewagon/rails-k8s-demo/blob/master/helm/templates/deployments/sidekiq.yaml
As you can see the commans section is provided through templates/../name.yaml rather than values.yaml

Rails bundle "Can't find GEM in any of the sources" in Docker Container only

I'm using Rails in a Docker container, and every once in a while I run into this issue that I have no idea how to solve. When adding a new gem to the Gemfile, upon rebuilding the Docker Image + Container, the build will fail with the common bundler error Could not find [GEM_NAME] in any of the sources; Run 'bundle install' to install missing gems. This only occurs to me when I try to build the image in Docker, if I run a regular bundle install on my local machine, the Gemfile gets installed correctly and everything works as expected.
I have a fairly standard Dockerfile & docker-compose file.
Dockerfile:
FROM ruby:2.6.3
ARG PG_MAJOR
ARG BUNDLER_VERSION
ARG UID
ARG MODE
# Add POSTGRESQL to the source list using the right version
RUN curl -sSL https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add - \
&& echo 'deb http://apt.postgresql.org/pub/repos/apt/ stretch-pgdg main' $PG_MAJOR > /etc/apt/sources.list.d/pgdg.list
ENV RAILS_ENV $MODE
RUN apt-get update -qq && apt-get install -y postgresql-client-$PG_MAJOR vim
RUN apt-get -y install sudo
RUN mkdir /usr/src/app
WORKDIR /usr/src/app
COPY Gemfile /usr/src/app/Gemfile
COPY Gemfile.lock /usr/src/app/Gemfile.lock
ENV BUNDLER_VERSION $BUNDLER_VERSION
RUN gem install bundler:$BUNDLER_VERSION
RUN bundle install
COPY . /usr/src/app
# Add a script to be executed every time the container starts.
COPY entrypoint.sh /usr/bin/
RUN chmod +x /usr/bin/entrypoint.sh
ENTRYPOINT ["entrypoint.sh"]
EXPOSE 3000
# Start the main process.
CMD ["rails", "server", "-b", "0.0.0.0"]
docker-compose.yml:
version: '3'
services:
backend:
build:
dockerfile: Dockerfile
args:
UID: ${UID:-1001}
BUNDLER_VERSION: 2.0.2
PG_MAJOR: 10
mode: development
tty: true
stdin_open: true
volumes:
- ./[REDACTED]:/usr/src/app
- gem_data_api:/usr/local/bundle:cached
ports:
- "3000:3000"
user: root
I've tried docker system prune -a, docker builder prune -a, reinstalling Docker, multiple rebuilds in a row, restarting my machine and so on, to no avail. The weird part is that it doesn't happen with every new Gem that I decide to add, only for some specific gems. For example I got this issue once again when trying to add gem 'sendgrid-ruby' to my Gemfile. This is the repo for the gem for reference, and the specific error I get with sendgrid-ruby is Could not find ruby_http_client-3.5.1 in any of the sources. I tried specifying ruby_http_client in my Gemfile, and I also tried sshing into the Docker container and running gem install ruby_http_client, but I get the same errors.
What might be happening here?
You're mounting a named volume over the container's /usr/local/bundle directory. The named volume will get populated from the image, but only the very first time you run the container. After that the old contents of the named volume will take precedence over the content of the image: using a volume this way will cause Docker to completely ignore any changes you make in the Gemfile.
You should be able to delete that volumes: line from the docker-compose.yml file. I'm not clear what benefit you would get from keeping the installed gems in a named volume.

Bundling from Github in a Dockerfile

I'm trying to move our Rails app over to a Docker deployment, however I can't manage to get bundle to install from a Github reference.
With the below Dockerfile:
FROM ruby:2.3.0-slim
MAINTAINER Chris Jewell <chrisjohnjewell#gmail.com>
# Install dependencies:
# - build-essential: To ensure certain gems can be compiled
# - nodejs: Compile assets
# - libpq-dev: Communicate with postgres through the postgres gem
# - postgresql-client-9.4: In case you want to talk directly to postgres
RUN apt-get update && apt-get install -qq -y build-essential nodejs libpq-dev postgresql-client-9.4 --fix-missing --no-install-recommends
# Set an environment variable to store where the app is installed to inside
# of the Docker image.
ENV INSTALL_PATH /ventbackend
RUN mkdir -p $INSTALL_PATH
# This sets the context of where commands will be ran in and is documented
# on Docker's website extensively.
WORKDIR $INSTALL_PATH
# Ensure gems are cached and only get updated when they change. This will
# drastically increase build times when your gems do not change.
COPY Gemfile Gemfile
RUN bundle install
# Copy in the application code from your work station at the current directory
# over to the working directory.
COPY . .
# Provide dummy data to Rails so it can pre-compile assets.
RUN bundle exec rake RAILS_ENV=production DATABASE_URL=postgresql://user:pass#127.0.0.1/dbname SECRET_TOKEN=pickasecuretoken assets:precompile
# Expose a volume so that nginx will be able to read in assets in production.
VOLUME ["$INSTALL_PATH/public"]
# The default command that gets ran will be to start the Unicorn server.
CMD bundle exec unicorn -c config/unicorn.rb
I get the following error when trying to run docker-compose up:
You need to install git to be able to use gems from git repositories. For help
installing git, please refer to GitHub's tutorial at
https://help.github.com/articles/set-up-git
I'm assuming that this is because of lines in the Gemfile like:
gem 'logstasher', github: 'MarkMurphy/logstasher', ref: 'be3e871385bde7b1897ec2a1831f868a843d8000'
However, we also use some private Gems as well.
Is installing Git on the container the way to go? How will this authenticate with Github?
Is installing Git on the container the way to go?
In that case, yes: you can see an example at "Using Docker to maintain a Ruby gem". It Dockerfile does include a:
# ~~~~ OS Maintenance ~~~~
RUN apt-get update && apt-get install -y git
How will this authenticate with Github?
It does not have to authenticate to GitHub in order to read, ie clone.
It you needed to push back a gem (to publish it), then you would need for instance your ssh keys (mounted through a volume).
But that is not needed here.

Resources