Elasticsearch not ready on CircleCI - docker

Question

When I run CircleCI, the first few tests fail due to Elasticsearch not being fully set up yet.
Usually I would use the dockerize library to wait for Elasticsearch to be finished, however this does not seem to detect Elasticsearch. Any ideas why? The dockerize command simply times out. However, the Elasticsearch container seems to be running because if I do not wait, eventually Elasticsearch starts to work with the tests.
Here is my docker file
version: 2
jobs:
build:
parallelism: 3
working_directory: ~/export-opportunities
docker:
- image: circleci/ruby:2.5.5-node
environment:
BUNDLE_JOBS: 3
BUNDLE_RETRY: 3
BUNDLE_PATH: vendor/bundle
PGHOST: localhost
PGUSER: user
RAILS_ENV: test
- image: circleci/postgres:latest
environment:
POSTGRES_USER: user
POSTGRES_DB: circle_test
POSTGRES_PASSWORD: $POSTGRES_PASSWORD
- image: circleci/redis:4.0.9
environment:
REDIS_URL: "redis://localhost:6379/"
- image: docker.elastic.co/elasticsearch/elasticsearch:6.3.2
environment:
cluster.name: elasticsearch
xpack.security.enabled: false
transport.host: localhost
network.host: 127.0.0.1
http.port: 9200
discovery.type: single-node
branches:
only: chore/XOT-597-circleci
steps:
- checkout # check out the code in the project directory
# restore bundle cache
- restore_cache:
keys:
- exops-{{ checksum "Gemfile.lock" }}
- run:
name: Bundle Install
command: bundle check || bundle install
# store bundle cache
- save_cache:
key: exops-{{ checksum "Gemfile.lock" }}
paths:
- vendor/bundle
# Database setup
- run:
name: install dockerize
command: wget https://github.com/jwilder/dockerize/releases/download/$DOCKERIZE_VERSION/dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz && sudo tar -C /usr/local/bin -xzvf dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz && rm dockerize-linux-amd64-$DOCKERIZE_VERSION.tar.gz
environment:
DOCKERIZE_VERSION: v0.3.0
- run:
name: Wait for DB
command: dockerize -wait tcp://localhost:5432 -timeout 1m
- run:
name: Database setup
command: |
bundle exec rake db:create
bundle exec rake db:migrate
# Redis setup
- run:
name: Wait for Redis
command: dockerize -wait tcp://localhost:6379 -timeout 1m
# DOES NOT WORK:
- run:
name: Wait for Elasticsearch
command: dockerize -wait http://localhost:9200 -timeout 2m
# Run rspec in parallel
- run: |
echo Running test ...
bundle exec rspec --profile 10 \
--format RspecJunitFormatter \
--out test_results/rspec.xml \
--format progress \
$(circleci tests glob "spec/**/*_spec.rb" | circleci tests split --split-by=timings)
# Save test results for timing analysis
- store_test_results:
path: test_results
Note I've also tried dockerize -wait tcp://localhost:9200 -timeout 2m, and dockerize -wait http://127.0.0.1:9200 -timeout 2m, and dockerize -wait tcp://127.0.0.1:9200 -timeout 2m to no effect.

Answer 1

I tried adding sleep 10 and sleep 100 however the issue persisted.
The issue was that the tests were running before the index was created. The index creation was being triggered by the first test to run, but took a few seconds, so the first few tests always failed.
My solution with this was to add the following code to trigger building the index, if not present, in rails_helper.rb which runs on starting the rails environment. For most other environments, the indices do exist so it does not slow down other processes.
# Build initial indices if not present, e.g. CircleCI
[Opportunity, Subscription].each do |model|
unless model.__elasticsearch__.index_exists? index: model.__elasticsearch__.index_name
model.__elasticsearch__.create_index!(force: true)
sleep 2
end
end