HEAD / and GET / in spring data ES 4.4.1 - spring-data-elasticsearch

Question

ES cloud is hosted in Azure. It works in spring data ES 4.1.5(ES client 7.9.3). But spring data ES 4.4.1 (ES client 7.17.4) requires cluster:monitor/main permission.
My admin doesn't want to grant such permission.
"root_cause":[{"type":"security_exception","reason":"action [cluster:monitor/main] is unauthorized for user [xxxx] with roles
I first asked question in this post. Val pointed out / end point cluster level API requires this permission.
why does it need cluster:monitor/main permission
I did some debug and found out more details. Spring data ES is sending HEAD / and GET / during SimpleElasticsearchRepository initialization.
Sorry I can't get the format working. So I attach screenshot here
There is error during SimpleElasticsearchRepository initialization. SimpleElasticsearchRepository -> RestIndexTemplate -> RestHighLevelClient
During SimpleElasticsearchRepository initialization, indices are empty. So the normal HEAD request on actual index becomes HEAD /. This user account doesn't have permission on sending HEAD /
Since the purpose is to check if index exists, is it possible to skip checking with server if indices are empty? Because it turns into request to check cluster level info. And below requests require total different permission.
HEAD /
HEAD /actual_index
I also find this error in the log. I couldn't find out how it was being sent out in spring data ES. This use can't run GET / neither.
It works in spring data ES 4.1.5 Why didn't Spring data ES 4.1.5 send out above HEAD / and GET / requests?
Updated to include my original post.

Answer 1

I found out my details about the issues.
HEAD /
During SimpleElasticsearchRepository initialization, it does below. My index has createIndex = true. So it goes to check exists(). But the index is empty here, so HEAD / my_index becomes HEAD /.
I don't know why index is empty here. But I set createIndex = false to skip it.
if (this.shouldCreateIndexAndMapping() && !this.indexOperations.exists()) {
this.indexOperations.createWithMapping();
}
GET /
Based on my testing, GET / is being sent out once per session. It acts like network connectivity testing. If there is any error, it will be ignored.
//MainClientExec - [exchange: 1] start execution -- GET / HTTP/1.1[\r][\n] -- once per session
11:58:46.648 [main] DEBUG org.apache.http.impl.nio.client.MainClientExec - [exchange: 1] start execution
11:58:46.967 [I/O dispatcher 1] DEBUG org.apache.http.headers - http-outgoing-0 >> GET / HTTP/1.1
11:58:46.967 [I/O dispatcher 1] DEBUG org.apache.http.headers - http-outgoing-0 >> X-Elastic-Client-Meta: es=7.17.4p,jv=1.8,t=7.17.4p,hc=4.1.4,kt=1.6,gy=2.4
11:58:46.967 [I/O dispatcher 1] DEBUG org.apache.http.headers - http-outgoing-0 >> Content-Length: 0
11:58:46.967 [I/O dispatcher 1] DEBUG org.apache.http.headers - http-outgoing-0 >> Host: xxx.azure.elastic-cloud.com:9243
11:58:46.967 [I/O dispatcher 1] DEBUG org.apache.http.headers - http-outgoing-0 >> Connection: Keep-Alive
11:58:46.967 [I/O dispatcher 1] DEBUG org.apache.http.headers - http-outgoing-0 >> User-Agent: elasticsearch-java/7.17.4-SNAPSHOT (Java/1.8.0_331)
11:58:46.967 [I/O dispatcher 1] DEBUG org.apache.http.headers - http-outgoing-0 >> Authorization: Basic xxx
11:58:47.105 [I/O dispatcher 1] DEBUG org.apache.http.wire - http-outgoing-0 << "HTTP/1.1 403 Forbidden[\r][\n]"
11:58:47.105 [I/O dispatcher 1] DEBUG org.apache.http.wire - http-outgoing-0 << "Content-Length: 4157[\r][\n]"
11:58:47.105 [I/O dispatcher 1] DEBUG org.apache.http.wire - http-outgoing-0 << "Content-Type: application/json; charset=UTF-8[\r][\n]"
11:58:47.105 [I/O dispatcher 1] DEBUG org.apache.http.wire - http-outgoing-0 << "X-Cloud-Request-Id: 3TA-1pHlTXuLbAoue0rKZw[\r][\n]"
11:58:47.105 [I/O dispatcher 1] DEBUG org.apache.http.wire - http-outgoing-0 << "X-Elastic-Product: Elasticsearch[\r][\n]"
11:58:47.105 [I/O dispatcher 1] DEBUG org.apache.http.wire - http-outgoing-0 << "X-Found-Handling-Cluster: aaf9bff719c349bbad7a015615a994f7[\r][\n]"
11:58:47.105 [I/O dispatcher 1] DEBUG org.apache.http.wire - http-outgoing-0 << "X-Found-Handling-Instance: instance-0000000001[\r][\n]"
11:58:47.105 [I/O dispatcher 1] DEBUG org.apache.http.wire - http-outgoing-0 << "Date: Fri, 22 Jul 2022 16:58:46 GMT[\r][\n]"
11:58:47.105 [I/O dispatcher 1] DEBUG org.apache.http.wire - http-outgoing-0 << "[\r][\n]"
11:58:47.105 [I/O dispatcher 1] DEBUG org.apache.http.wire - http-outgoing-0 << "{"error":{"root_cause":[{"type":"security_exception","reason":"action [cluster:monitor/main] is unauthorized for user
11:58:47.115 [I/O dispatcher 1] DEBUG org.apache.http.headers - http-outgoing-0 << HTTP/1.1 403 Forbidden
11:58:47.115 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.client.MainClientExec - [exchange: 1] Response received HTTP/1.1 403 Forbidden
11:58:47.126 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.client.InternalIODispatch - http-outgoing-0 [ACTIVE(4157)] Input ready
11:58:47.126 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.client.MainClientExec - [exchange: 1] Consume content
11:58:47.131 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.client.InternalHttpAsyncClient - [exchange: 1] Connection can be kept alive indefinitely
11:58:47.131 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.client.MainClientExec - [exchange: 1] Response processed
11:58:47.131 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.client.InternalHttpAsyncClient - [exchange: 1] releasing connection
11:58:47.131 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.conn.ManagedNHttpClientConnectionImpl - http-outgoing-0
11:58:47.131 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager - Releasing connection: [id: http-outgoing-0][route: {s}->https://
11:58:47.131 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager - Connection [id: http-outgoing-0][route: {s}->https://
11:58:47.132 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.conn.ManagedNHttpClientConnectionImpl - http-outgoing-0 10.2
11:58:47.133 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager - Connection released: [id: http-outgoing-0][
After above, the real query is sent.
//MainClientExec - [exchange: 2] start execution - GET /my_index/_doc/asdf HTTP/1.1
11:58:47.137 [main] DEBUG org.apache.http.impl.nio.client.MainClientExec - [exchange: 2] start execution
11:58:47.147 [I/O dispatcher 1] DEBUG org.apache.http.headers - http-outgoing-0 >> GET /my_index/_doc/asdf HTTP/1.1
11:58:47.147 [I/O dispatcher 1] DEBUG org.apache.http.headers - http-outgoing-0 >> X-Elastic-Client-Meta: es=7.17.4p,jv=1.8,t=7.17.4p,hc=4.1.4,kt=1.6,gy=2.4
11:58:47.147 [I/O dispatcher 1] DEBUG org.apache.http.headers - http-outgoing-0 >> Content-Length: 0
11:58:47.147 [I/O dispatcher 1] DEBUG org.apache.http.headers - http-outgoing-0 >> Host: xxx.azure.elastic-cloud.com:9243
11:58:47.147 [I/O dispatcher 1] DEBUG org.apache.http.headers - http-outgoing-0 >> Connection: Keep-Alive
11:58:47.147 [I/O dispatcher 1] DEBUG org.apache.http.headers - http-outgoing-0 >> User-Agent: elasticsearch-java/7.17.4-SNAPSHOT (Java/1.8.0_331)
11:58:47.147 [I/O dispatcher 1] DEBUG org.apache.http.headers - http-outgoing-0 >> Authorization: Basic xx==
11:58:47.148 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.conn.ManagedNHttpClientConnectionImpl - http-outgoing-0 10.24.15.131:58069<->52.158.162.229:9243[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][0][0][0]: Event set [w]
11:58:47.148 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.client.MainClientExec - [exchange: 2] Request completed
11:58:47.148 [I/O dispatcher 1] DEBUG org.apache.http.impl.nio.conn.ManagedNHttpClientConnectionImpl - http-outgoing-0 10.24.15.131:58069<->52.158.162.229:9243[ACTIVE][rw:w][ACTIVE][rw][NOT_HANDSHAKING][0][0][372]: 343 bytes written
11:58:47.148 [I/O dispatcher 1] DEBUG org.apache.http.wire - http-outgoing-0 >> "GET /my_index/_doc/asdf HTTP/1.1[\r][\n]"
11:58:47.148 [I/O dispatcher 1] DEBUG org.apache.http.wire - http-outgoing-0 >> "X-Elastic-Client-Meta: es=7.17.4p,jv=1.8,t=7.17.4p,hc=4.1.4,kt=1.6,gy=2.4[\r][\n]"