Integrate CCS with new search_shards API #95894
Conversation
dnhatn force-pushed the ccs-can-match branch 10 times, most recently from 7db7d36 to 34bf797 Compare dnhatn force-pushed the ccs-can-match branch 6 times, most recently from faec02c to a77a3f7 Compare dnhatn force-pushed the ccs-can-match branch 3 times, most recently from 66551cd to f817652 Compare dnhatn changed the title dnhatn added :Search/Search 
| Hi @dnhatn, I've created a changelog YAML for you. |
server/src/internalClusterTest/java/org/elasticsearch/search/ccs/CCSCanMatchIT.java Show resolved Hide resolved
server/src/test/java/org/elasticsearch/action/search/TransportSearchActionTests.java Outdated Show resolved Hide resolved
| @quux00 Thanks for reviews. |
server/src/internalClusterTest/java/org/elasticsearch/search/ccs/CCSCanMatchIT.java Outdated Show resolved Hide resolved
server/src/internalClusterTest/java/org/elasticsearch/search/ccs/CCSCanMatchIT.java Show resolved Hide resolved
server/src/internalClusterTest/java/org/elasticsearch/search/ccs/CCSCanMatchIT.java Outdated Show resolved Hide resolved
server/src/main/java/org/elasticsearch/action/search/CanMatchPreFilterSearchPhase.java Outdated Show resolved Hide resolved
server/src/main/java/org/elasticsearch/action/search/TransportSearchShardsAction.java Outdated Show resolved Hide resolved
| * will invoke the listener immediately. | ||
| */ | ||
| void ensureConnected(String clusterAlias, ActionListener<Void> listener) { | ||
| public void ensureConnected(String clusterAlias, ActionListener<Void> listener) { |
There was a problem hiding this comment.
I am not entirely sure that this is a good idea, I tend to think that this is not public for a good reason, in that consumers should not worry about handling connection.
There was a problem hiding this comment.
I made an alternative to combine both ensureConnected and getConnection in 56571d8. WDYT? Here we need to replicate partially the logic in RemoteClusterAwareClient#doExecute. Any suggestion are welcome.
There was a problem hiding this comment.
I see thanks for trying that. I think that I am still missing why we can't use the remote cluster client like we did before, sorry if that's a silly question but it's not so obvious to me :)
There was a problem hiding this comment.
The reason that we can't use the remote cluster client is that we need to know the version the of the transport connection (between the local and remote clusters) before we decide if we can go with the new action.
server/src/main/java/org/elasticsearch/action/search/TransportSearchAction.java Show resolved Hide resolved
server/src/main/java/org/elasticsearch/action/search/TransportSearchAction.java Outdated Show resolved Hide resolved
server/src/main/java/org/elasticsearch/action/search/SearchShardsResponse.java Outdated Show resolved Hide resolved
server/src/test/java/org/elasticsearch/transport/RemoteClusterConnectionTests.java Show resolved Hide resolved
| @javanna Thanks for reviews. I think I have addressed all your comments except the |
server/src/main/java/org/elasticsearch/action/search/SearchShardsResponse.java Outdated Show resolved Hide resolved
server/src/test/java/org/elasticsearch/transport/RemoteClusterConnectionTests.java Show resolved Hide resolved
| * will invoke the listener immediately. | ||
| */ | ||
| void ensureConnected(String clusterAlias, ActionListener<Void> listener) { | ||
| public void ensureConnected(String clusterAlias, ActionListener<Void> listener) { |
There was a problem hiding this comment.
I see thanks for trying that. I think that I am still missing why we can't use the remote cluster client like we did before, sorry if that's a silly question but it's not so obvious to me :)
When security is enabled, CCS requests for a remote wildcard pointing data streams are not resolved correctly. This issue happens because search_shards requests do not override the includeDataStreams method, causing the RBAC engine to exclude data streams while resolving the index pattern. Relates #95894 Relates #94534
4 participants
This PR integrates CCS with the new search_shards API. With this change, we will be able to skip shards on the coordinator on remote clusters using the timestamps stored in the cluster state.
Relates #94534
Closes #93730