fix: transient error log-spam, add retry policy #1273
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
toddbaert commented Mar 11, 2025
.../src/main/java/dev/openfeature/contrib/providers/flagd/resolver/common/ChannelConnector.java Show resolved Hide resolved
toddbaert commented Mar 11, 2025
...gd/src/main/java/dev/openfeature/contrib/providers/flagd/resolver/common/ChannelBuilder.java Show resolved Hide resolved
toddbaert commented Mar 11, 2025
...e/contrib/providers/flagd/resolver/process/storage/connector/sync/SyncStreamQueueSource.java Show resolved Hide resolved
toddbaert commented Mar 11, 2025
...rs/flagd/src/main/java/dev/openfeature/contrib/providers/flagd/resolver/rpc/RpcResolver.java Show resolved Hide resolved
toddbaert commented Mar 11, 2025
providers/flagd/src/test/java/dev/openfeature/contrib/providers/flagd/FlagdProviderTest.java Outdated Show resolved Hide resolved
toddbaert changed the title toddbaert force-pushed the fix/logspam-with-retry-pol branch from ff4efc1 to aafa689 Compare toddbaert commented Mar 12, 2025
...gd/src/main/java/dev/openfeature/contrib/providers/flagd/resolver/common/ChannelBuilder.java Show resolved Hide resolved
Signed-off-by: Todd Baert <todd.baert@dynatrace.com>
toddbaert force-pushed the fix/logspam-with-retry-pol branch from aafa689 to 934e191 Compare Signed-off-by: Todd Baert <todd.baert@dynatrace.com>
toddbaert commented Mar 12, 2025
...ntrib/providers/flagd/resolver/process/storage/connector/sync/SyncStreamQueueSourceTest.java Show resolved Hide resolved
toddbaert commented Mar 12, 2025
...ntrib/providers/flagd/resolver/process/storage/connector/sync/SyncStreamQueueSourceTest.java Show resolved Hide resolved
aepfli approved these changes Mar 12, 2025
chrfwow reviewed Mar 12, 2025
...gd/src/main/java/dev/openfeature/contrib/providers/flagd/resolver/common/ChannelBuilder.java Outdated Show resolved Hide resolved
...e/contrib/providers/flagd/resolver/process/storage/connector/sync/SyncStreamQueueSource.java Show resolved Hide resolved
chrfwow approved these changes Mar 12, 2025
Signed-off-by: Todd Baert <todd.baert@dynatrace.com>
Signed-off-by: Todd Baert <todd.baert@dynatrace.com>
Signed-off-by: Todd Baert <todd.baert@dynatrace.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
Fixes an issue observed when intervening proxies are used (like Istio), where the proxy's upstream is disconnected, resulting in immediate error responses from the proxy (which we still have a healthy TCP connection to) which causes a tight loop in our retry logic for a brief time until the upstream is available.
This PR adds a retry policy to retry such problematic requests, which also slows down our retry loop. In general this will make the provider more robust to transient errors in streams and unary calls. I will be adding something similar to the flagd spec.
Other minor refactors as well.
There is one possible enhancement we could consider as a separate issue, mentioned here.