-
Couldn't load subscription status.
- Fork 259
fix: update network error msg to match kubelet expectations #3768
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: GitHub <noreply@github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR aligns the mtpnc is not ready error message with the kubelet’s expected “network is not ready” string and adds a comment explaining the rationale.
- Changed
errMTPNCNotReadymessage to include “network is not ready” prefix - Added comment referencing kubelet retry logic
- Existing unit tests and documentation adapters are included
Comments suppressed due to low confidence (3)
cns/middlewares/k8sSwiftV2.go:20
- Referencing a specific commit hash can become outdated. Consider linking to stable Kubernetes documentation or describing the relevant code path more generally to improve long-term maintainability.
// In AKS, for kubelet to retry faster it is particularly looking for "network is not ready" error string. cns/middlewares/k8sSwiftV2.go:22
- [nitpick] Consider renaming this error variable to
errNetworkNotReadyor similar to reflect the updated message and improve consistency between the variable name and its content.
errMTPNCNotReady = errors.New("network is not ready - mtpnc is not ready") cns/middlewares/k8sSwiftV2.go:22
- [nitpick] Ensure that the kubelet recognizes the exact prefix it expects (e.g.,
"network is not ready"). If only that substring matters for retry behavior, you may simplify the error string to avoid potential mismatches.
errMTPNCNotReady = errors.New("network is not ready - mtpnc is not ready") There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it looks like from upstream code this will reduce the retry interval from ~10 seconds to ~1 second-- wondering if there are concerns if the cns never comes up or if this fast retry will exhaust resources (ex: x10 more fail logs if cns never comes up)? Was there a particular incident or reasoning for needing the 10 sec improvement?
To clarify this doesn't happen when "CNS doesn't come up" instead this error message is thrown when MTPNC is not found by CNS. Yes this will make 10x more error logs in CNS and CNI, but this is the only way to address perf bottle neck related to swiftv2 podStartupLatency. It is not acceptable for customers to be waiting for a pod for > 30 secs , but today we are at > 50 secs. This fix will shave off 10 to 15 secs and we must take it. This is in swiftv2 multitenant code path only which is exceedingly small number of clusters in AKS today, so not all CNI error logs will be noisy. |
| /azp run Azure Container Networking PR |
| Azure Pipelines successfully started running 1 pipeline(s). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
update network error msg to match kubelet expectations Signed-off-by: GitHub <noreply@github.com>
update network error msg to match kubelet expectations Signed-off-by: GitHub <noreply@github.com>
…found and not ready in CNS (#3775) * fix: update network error msg to match kubelet expectations (#3768) update network error msg to match kubelet expectations Signed-off-by: GitHub <noreply@github.com> * fix: update error message for MTPNC not found and not ready in CNS (#3776) squash commit to prevent github CLA issue Signed-off-by: GitHub <noreply@github.com> --------- Signed-off-by: GitHub <noreply@github.com>
update network error msg to match kubelet expectations Signed-off-by: GitHub <noreply@github.com>
update network error msg to match kubelet expectations Signed-off-by: GitHub <noreply@github.com>
Signed-off-by: GitHub noreply@github.com
Reason for Change:
This is a perf fix. The particular error message is retries with faster backOffInterval than default error messages by the kubelet.
Issue Fixed:
Requirements:
Notes: