Define own WatchErrorHandler for toolscache.Reflector #2987
Description
client-go sets a default handler for watching errors func DefaultWatchErrorHandler(r *Reflector, err error), see https://github.com/kubernetes/client-go/blob/23900f492969db045e8a36396852381fd189569b/tools/cache/reflector.go#L154. Right now, if it is included in the controller-runtime it will use its logger klog by default. This leads to inconsistent log output by default. Therefore, I propose that the controller-runtime should overwrite this method, i.e., use its own handler.
This handler could look something like this:
func WatchErrorHandler(r *toolscache.Reflector, err error) { log := ctrl.Log.WithValues("name", r.Name(), "description", r.TypeDescription()) switch { case apierrors.IsResourceExpired(err) || apierrors.IsGone(err): // Don't set LastSyncResourceVersionUnavailable - LIST call with ResourceVersion=RV already // has a semantic that it returns data at least as fresh as provided RV. // So first try to LIST with setting RV to resource version of last observed object. log.Error(err, "watch closed") case err == io.EOF: // watch closed normally case err == io.ErrUnexpectedEOF: log.Error(err, "watch closed with unexpected EOF") default: utilruntime.HandleError(fmt.Errorf("%s: Failed to watch %v: %v", r.Name(), r.TypeDescription(), err)) } }As you can see, we would use a logger with two fields of the reflector now. I exposed these fields recently (see kubernetes/client-go@ce42c29), so the controller-runtime can essentially redefine the default handler of client-go, as the default handler also logs these fields.
I would like to implement this, if it gets your approval.