Environment

• Airbyte version: 0.40.15

• OS Version / Instance: Ubuntu 22.04.1 LTS

• Deployment: local Docker

• Source Connector and version: source-elasticsearch 0.1.0

• Step where error happened: Sync job

Current Behavior

An too_long_frame_exception error is return by elasticsearch when trying to get indexes metadata (io.airbyte.integrations.source.elasticsearch.ElasticsearchConnection L 148) when trying to sync our ES log cluster (there is one index per day).

Expected Behavior

It should build sustainable request index mapping requests.

Explanation

No matter how much indexes you have, the ElasticsearchConnection will build only one request to ES, and then may "hit" the 4096 bytes limit :

 String[] copiedIndices = indices.toArray(String[]::new); request.indices(copiedIndices); GetMappingsResponse getMappingResponse = client.indices().getMapping(request, RequestOptions.DEFAULT); 

Steps to Reproduce

1. create a lot of indexes (like ES configured for logs)
2. run the synchronization

Are you willing to submit a PR?

Yes. It is already... Ready.