ElasticSearch in Docker Compose - ruby-on-rails

Question

Please go easy on me here, My experience with ElasticSearch is pretty much solely with use of SearchKick and my own Rails app.
I have a Rails app which I have been running locally on my machine using SearchKick and ElasticSearch for quite some time now.
Recently I’ve been reading about how to convert my dev environment into a Dockerised form.
I have got my Rails app and its database working nicely but then I came to add Search back in .
My Docker Compose file is below:
version: '3.7'
services:
app:
build: .
command: bundle exec rails s -p 3000 -b '0.0.0.0'
volumes:
- .:/app
- bundle-volume:/usr/local/bundle
- yarn-volume:/app/node_modules
ports:
- "3000:3000"
depends_on:
- db
- search
db:
image: 'postgres:9.6'
env_file:
- .env.development
volumes:
- db-volume:/var/lib/postgresql/data
ports:
- "5432:5432"
# environment:
# - POSTGRES_HOST_AUTH_METHOD=trust
search:
image: 'elasticsearch'
env_file:
- .env.development
volumes:
- 'search-volume:/usr/share/elasticsearch/data'
ports:
- '9200:9200'
volumes:
bundle-volume:
yarn-volume:
db-volume:
search-volume:
When I start the container and head to the Rails console to try and run <Model>.reindex I get the following error:
Traceback (most recent call last): 1: from (irb):1
Faraday::ConnectionFailed (Failed to open TCP connection to
localhost:9200 (Cannot assign requested address - connect(2) for
“localhost” port 9200))
I have set an Environment var like so in my .env.development file:
ELASTICSEARCH_URL=http://search:9200
The error shows it’s trying to connect to localhost which is exactly where I think the issue lays.
It doesn’t look like the variable i’m setting in my .env.development file is being used in the way it should be.
Anyone with an in-depth knowledge of Docker able to help me out here?
Thanks,
Neil
EDIT:
Dockerfile:
FROM ruby:2.6.5
# Install 3rd party dependencies.
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add && \
echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list && \
apt update && \
apt -y install nodejs \
yarn \
build-essential \
libpq-dev
# Sets Environment variables
ENV RAILS_ROOT /app
ENV PORT 3000
# Working directory.
RUN mkdir $RAILS_ROOT
WORKDIR $RAILS_ROOT
# Script run when container first starts.
COPY docker-entrypoint.sh docker-entrypoint.sh
RUN chmod +x docker-entrypoint.sh
ENTRYPOINT [ "/app/docker-entrypoint.sh" ]
docker-entrypoint.sh:
#!/bin/bash
set -e
bundle install
yarn install --check-files --frozen-lockfile
# Remove a potentially pre-existing server.pid for Rails.
rm -f /app/tmp/pids/server.pid
# Run the command.
exec "$#"
.env:
ELASTICSEARCH_URL=http://search:9200
DATABASE_HOST=db
DATABASE_USER=postgres
DATABASE_PASSWORD=postgres
DATABASE_NAME=mysite_development
DATABASE_PORT=5432
Gemfile:
source 'https://rubygems.org'
git_source(:github) { |repo| "https://github.com/#{repo}.git" }
ruby '2.6.5'
# Bundle edge Rails instead: gem 'rails', github: 'rails/rails'
gem 'rails', '~> 6.1.3'
# Use postgresql as the database for Active Record
gem 'pg', '~> 1.2.3'
# Use Puma as the app server
gem 'puma', '~> 4.3.5'
# Use SCSS for stylesheets
gem 'sass-rails', '>= 6'
# Transpile app-like JavaScript. Read more: https://github.com/rails/webpacker
gem 'webpacker', '~> 6.x'
# Turbolinks makes navigating your web application faster. Read more: https://github.com/turbolinks/turbolinks
gem 'turbo-rails', '~> 0.5.11'
# Build JSON APIs with ease. Read more: https://github.com/rails/jbuilder
gem 'jbuilder', '~> 2.7'
# Use Redis adapter to run Action Cable in production
gem 'redis', '~> 4'
# Use Active Model has_secure_password
# gem 'bcrypt', '~> 3.1.7'
# Use Active Storage variant
# gem 'image_processing', '~> 1.2'
# Use Capistrano for deployment
# gem 'capistrano-rails', group: :development
# Reduces boot times through caching; required in config/boot.rb
gem 'bootsnap', '>= 1.4.2', require: false
gem 'devise' # Authorisation and Authentication
gem 'httparty' # Make HTTP requests
gem 'searchkick', '~> 4.5.0' # ElasticSearch searching
gem 'geocoder' # Allows models to have a location
gem 'gretel' #Breadcrumb functionality
gem 'nav_lynx' # Navigation links
gem 'inline_svg', '~> 1.7.2' # Inlines SVG files
gem 'flickraw' # Integration with the Flickr API
gem 'countries', require: 'countries/global' # Country list (with helper to omit ISO3166:: module prefix)
gem 'kaminari' # Pagination
gem 'flutie' # Utility classes
gem 'acts-as-taggable-on', '~> 7.0.0' # Tagging
gem 'friendly_id', '~> 5.4'
gem 'acts_as_list' # Ordering
gem 'delayed_job_active_record' # Background Jobs
gem 'foreman'
gem 'daemons'
gem 'delayed_job_web'
gem 'activerecord-hierarchical_query', '~> 1.3.0'
gem 'roo', '~> 2.8.3'
gem 'faker'
group :development, :test do
gem 'dotenv-rails'
# Call 'byebug' anywhere in the code to stop execution and get a debugger console
gem 'byebug', platforms: [:mri, :mingw, :x64_mingw]
gem 'rspec-rails'
gem "bullet"
end
group :development do
# Access an interactive console on exception pages or by calling 'console' anywhere in the code.
gem 'web-console', '>= 3.3.0'
gem 'listen', '~> 3.4'
# Spring speeds up development by keeping your application running in the background. Read more: https://github.com/rails/spring
gem 'spring'
gem 'spring-watcher-listen', '~> 2.0.1'
end
group :test do
# Adds support for Capybara system testing and selenium driver
gem 'capybara', '>= 2.15'
gem 'selenium-webdriver'
# Easy installation and use of web drivers to run system tests with browsers
gem 'webdrivers'
end
# Windows does not include zoneinfo files, so bundle the tzinfo-data gem
gem 'tzinfo-data', platforms: [:mingw, :mswin, :x64_mingw, :jruby]
gem 'capistrano', '~> 3.11'
gem 'capistrano-rails', '~> 1.4'
gem 'capistrano-passenger', '~> 0.2.0'
gem 'capistrano-rbenv', '~> 2.1', '>= 2.1.4'
gem 'ed25519'
gem 'bcrypt_pbkdf'
gem 'rubocop', require: false
gem 'rubocop-rails', require: false
EDIT 2:
Latest error message:
Traceback (most recent call last): 1: from (irb):1
Faraday::ConnectionFailed (Failed to open TCP connection to
search:9200 (getaddrinfo: No address associated with hostname))
EDIT 3:
So, after getting my machine back today, I tried your suggestions. It turns out the search container was erroring. The error is pasted below:
search_1 | [2021-08-04T20:44:40,178][WARN
][o.e.b.ElasticsearchUncaughtExceptionHandler] [unknown] uncaught
exception in thread [main] search_1 |
org.elasticsearch.bootstrap.StartupException:
ElasticsearchException[java.io.IOException: failed to read [id:7,
file:/usr/share/elasticsearch/data/nodes/0/_state/node-7.st]]; nested:
IOException[failed to read [id:7,
file:/usr/share/elasticsearch/data/nodes/0/_state/node-7.st]]; nested:
XContentParseException[[-1:36] [node_meta_data] unknown field
[node_version], parser not found];
It looks like it's failing due to the location the Volume is set to. I temporarily commented out the Volume configuration for that service and my Rails model now gets indexed.
I'm assuming now though, that if I close the container and start it up again that the Index will have been removed because it's no longer stored in a Volume?
Any ideas how I can get the index data stored in a volume and let the container work?