Want to stop noisy FATAL [404] for index checking on Elasticsearch::Persistence::Repository

[aeroastro]

Summary

We are using Elasticsearch::Persistence::Repository, and we are happy with the handiness compared with directly using Elasticsearch::Client.

However, we're now concerned about the noisy FATAL error [404] the repository emits when checking the index's existence.

• Elasticsearch::Persistence::Repository#index_exists?
• Elasticsearch::Persistence::Repository#create_index!

Reproduction Code

elasticsearch-persistence version: 7.1.0 (current master)
ruby version: 2.6.2

require 'logger'
require 'elasticsearch/persistence'

class MyRepository
  include Elasticsearch::Persistence::Repository
end

repository = MyRepository.create(
  index_name: 'new_index',
  client: Elasticsearch::Client.new(logger: Logger.new(STDOUT, level: :error))
)
repository.client.indices.delete(index: '*') # initialization for reproduction code

repository.index_exists? # This is the problem
# F, [2021-03-02T00:24:33.962978 #28878] FATAL -- : [404]
# => false
repository.create_index! # This is also the problem
# F, [2021-03-02T00:24:57.883461 #28878] FATAL -- : [404]
# => {"acknowledged"=>true, "shards_acknowledged"=>true, "index"=>"new_index"}

Root Cause

Both of the above codes are using the following method.

elasticsearch-rails/elasticsearch-model/lib/elasticsearch/model/indexing.rb

Lines 270 to 274 in 16a297a

	def index_exists?(options={})
	target_index = options[:index] || self.index_name
	self.client.indices.exists(index: target_index) rescue false
	end

Because Elasticsearch::Client regards all the HTTP status codes >= 300 as FATAL, the above code self.client.indices.exists(index: target_index) generates FATAL log.

Based on the objective of HEAD request and index_exists?, the log is noisy.

Proposed Solution

Based on the objective of exists?, where both 200 and 404 are the expected status code, IMHO, we should avoid fatal logs in these situations.

Now I propose the following two solutions to this issue.

Difficult but fundamental fix:

• Treat 404 for HEAD request as non-fatal in Elasticsearch::Client.

As this touches the internal design of elasticsearch-ruby, it takes time for the maintainers to decide.

Easy but workaround fix:

• Pass ignore: 404 as the optional arguments to Elasticsearch::API::Indices::Actions#exists?

This can stop FATAL log in addition to stop exception.

[aeroastro]

I have created a Pull Request for this issue.

Since I would like to suppress unnecessary logs on our development and production servers in the near future, I have chosen the latter workaround option for this patch.