Skip to content

Conversation

@arjan-bal
Copy link
Contributor

The test verifies that lazy's child get the latest resolver state followed by the latest resolver error. The test waits for an RPC to succeed before verifying the child has received the resolver error. The RPC can complete before the child get the resolver error. This PR uses a grpcsync.Event to wait till the child receives the resolver error.

Failing test run: https://github.com/grpc/grpc-go/actions/runs/13314475263/attempts/2?pr=8086

Failure stack trace
WARNING: DATA RACE Read at 0x00c00019c57f by goroutine 191: google.golang.org/grpc/balancer/lazy_test.s.TestGoodUpdateThenResolverError() /home/runner/work/grpc-go/grpc-go/balancer/lazy/lazy_ext_test.go:289 +0x13b7 google.golang.org/grpc/balancer/lazy_test.(*s).TestGoodUpdateThenResolverError() <autogenerated>:1 +0x3e runtime.call16() /opt/hostedtoolcache/go/1.23.6/x[64](https://github.com/grpc/grpc-go/actions/runs/13314475263/job/37208920450?pr=8086#step:8:65)/src/runtime/asm_amd64.s:775 +0x42 reflect.callMethod() <autogenerated>:1 +0x56 reflect.methodValueCall() /opt/hostedtoolcache/go/1.23.6/x64/src/reflect/asm_amd64.s:76 +0x6d testing.tRunner() /opt/hostedtoolcache/go/1.23.6/x64/src/testing/testing.go:1690 +0x226 testing.(*T).Run.gowrap1() /opt/hostedtoolcache/go/1.23.6/x64/src/testing/testing.go:1743 +0x44 Previous write at 0x00c00019c57f by goroutine 200: google.golang.org/grpc/balancer/lazy_test.s.TestGoodUpdateThenResolverError.func3() /home/runner/work/grpc-go/grpc-go/balancer/lazy/lazy_ext_test.go:216 +0xa4 google.golang.org/grpc/internal/balancer/stub.(*bal).ResolverError() /home/runner/work/grpc-go/grpc-go/internal/balancer/stub/stub.go:70 +0x94 google.golang.org/grpc/balancer/lazy.(*lazyBalancer).ExitIdle() /home/runner/work/grpc-go/grpc-go/balancer/lazy/lazy.go:145 +0x586 google.golang.org/grpc/balancer/lazy.NewBalancer.func1.gowrap1() /home/runner/work/grpc-go/grpc-go/balancer/lazy/lazy.go:[65](https://github.com/grpc/grpc-go/actions/runs/13314475263/job/37208920450?pr=8086#step:8:66) +0x33 Goroutine 191 (running) created at: testing.(*T).Run() /opt/hostedtoolcache/go/1.23.6/x64/src/testing/testing.go:1743 +0x825 google.golang.org/grpc/internal/grpctest.RunSubTests() /home/runner/work/grpc-go/grpc-go/internal/grpctest/grpctest.go:114 +0x351 google.golang.org/grpc/balancer/lazy_test.Test() /home/runner/work/grpc-go/grpc-go/balancer/lazy/lazy_ext_test.go:60 +0x34 testing.tRunner() /opt/hostedtoolcache/go/1.23.6/x64/src/testing/testing.go:1690 +0x226 testing.(*T).Run.gowrap1() /opt/hostedtoolcache/go/1.23.6/x64/src/testing/testing.go:1743 +0x44 Goroutine 200 (finished) created at: google.golang.org/grpc/balancer/lazy.NewBalancer.func1() /home/runner/work/grpc-go/grpc-go/balancer/lazy/lazy.go:65 +0x8d sync.OnceFunc.func1() /opt/hostedtoolcache/go/1.23.6/x64/src/sync/oncefunc.go:27 +0x7b sync.(*Once).doSlow() /opt/hostedtoolcache/go/1.23.6/x64/src/sync/once.go:76 +0xe1 sync.(*Once).Do() /opt/hostedtoolcache/go/1.23.6/x64/src/sync/once.go:67 +0x44 sync.OnceFunc.func2() /opt/hostedtoolcache/go/1.23.6/x64/src/sync/oncefunc.go:32 +0x4a google.golang.org/grpc/balancer/lazy.(*idlePicker).Pick() /home/runner/work/grpc-go/grpc-go/balancer/lazy/lazy.go:157 +0x35 google.golang.org/grpc.(*pickerWrapper).pick() /home/runner/work/grpc-go/grpc-go/picker_wrapper.go:156 +0x948 google.golang.org/grpc.(*ClientConn).getTransport() /home/runner/work/grpc-go/grpc-go/clientconn.go:1051 +0xae google.golang.org/grpc.(*csAttempt).getTransport() /home/runner/work/grpc-go/grpc-go/stream.go:470 +0x144 google.golang.org/grpc.newClientStreamWithParams.func2() /home/runner/work/grpc-go/grpc-go/stream.go:353 +0x33 google.golang.org/grpc.(*clientStream).withRetry() /home/runner/work/grpc-go/grpc-go/stream.go:790 +0x197 google.golang.org/grpc.newClientStreamWithParams() /home/runner/work/grpc-go/grpc-go/stream.go:365 +0x1797 google.golang.org/grpc.newClientStream.func3() /home/runner/work/grpc-go/grpc-go/stream.go:222 +0x191 google.golang.org/grpc.newClientStream() /home/runner/work/grpc-go/grpc-go/stream.go:257 +0xc69 google.golang.org/grpc.invoke() /home/runner/work/grpc-go/grpc-go/call.go:[66](https://github.com/grpc/grpc-go/actions/runs/13314475263/job/37208920450?pr=8086#step:8:67) +0xc6 google.golang.org/grpc.(*ClientConn).Invoke() /home/runner/work/grpc-go/grpc-go/call.go:37 +0x2f0 google.golang.org/grpc/interop/grpc_testing.(*testServiceClient).EmptyCall() /home/runner/work/grpc-go/grpc-go/interop/grpc_testing/test_grpc.pb.go:95 +0x1af google.golang.org/grpc/balancer/lazy_test.s.TestGoodUpdateThenResolverError() /home/runner/work/grpc-go/grpc-go/balancer/lazy/lazy_ext_test.go:281 +0x12dc google.golang.org/grpc/balancer/lazy_test.(*s).TestGoodUpdateThenResolverError() <autogenerated>:1 +0x3e runtime.call16() /opt/hostedtoolcache/go/1.23.6/x64/src/runtime/asm_amd64.s:775 +0x42 reflect.callMethod() <autogenerated>:1 +0x56 reflect.methodValueCall() /opt/hostedtoolcache/go/1.23.6/x64/src/reflect/asm_amd64.s:76 +0x6d testing.tRunner() /opt/hostedtoolcache/go/1.23.6/x64/src/testing/testing.go:1[69](https://github.com/grpc/grpc-go/actions/runs/13314475263/job/37208920450?pr=8086#step:8:70)0 +0x226 testing.(*T).Run.gowrap1() /opt/hostedtoolcache/go/1.23.6/x64/src/testing/testing.go:1743 +0x44 

RELEASE NOTES: N/A

@arjan-bal arjan-bal added Type: Testing Area: Resolvers/Balancers Includes LB policy & NR APIs, resolver/balancer/picker wrappers, LB policy impls and utilities. labels Feb 14, 2025
@arjan-bal arjan-bal added this to the 1.71 Release milestone Feb 14, 2025
@arjan-bal arjan-bal requested review from dfawley and easwars February 14, 2025 04:29
@codecov
Copy link

codecov bot commented Feb 14, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.18%. Comparing base (fabe274) to head (87a41ab).
Report is 5 commits behind head on master.

Additional details and impacted files
@@ Coverage Diff @@ ## master #8088 +/- ## ========================================== - Coverage 82.20% 82.18% -0.02%  ========================================== Files 387 387 Lines 38928 38943 +15 ========================================== + Hits 31999 32005 +6  - Misses 5606 5615 +9  Partials 1323 1323 

see 23 files with indirect coverage changes

@dfawley dfawley assigned arjan-bal and unassigned easwars and dfawley Feb 14, 2025
@arjan-bal arjan-bal merged commit e55819e into grpc:master Feb 17, 2025
15 checks passed
@arjan-bal arjan-bal deleted the fix-race-lazy-test branch February 17, 2025 03:32
janardhanvissa pushed a commit to janardhanvissa/grpc-go that referenced this pull request Mar 24, 2025
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 17, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Area: Resolvers/Balancers Includes LB policy & NR APIs, resolver/balancer/picker wrappers, LB policy impls and utilities. Type: Bug

3 participants