Improve memory diagnoser accuracy #2562
Conversation
timcassell force-pushed the stabilize-memorydiagnoser branch from d351f0e to c864594 Compare 
| It looks like I may have resolved the zero-alloc issue! New results (using the same benchmark from #1543):
The resolution ended up being moving the I was unable to repro the issue in a plain console application, so I'm not sure why PTAL @adamsitnik |
timcassell force-pushed the stabilize-memorydiagnoser branch from 3518d50 to ce134ab Compare | I added the thread sleep from #1543 in Core runtimes 3.0 to 6.0, and now I'm getting 0 measurements across the board:
|
timcassell force-pushed the stabilize-memorydiagnoser branch 3 times, most recently from 0624a8c to 917988c Compare 
There was a problem hiding this comment.
Big thanks for working on improving the accuracy of the memory diagnoser @timcassell !
Let's wait for the response from @Maoni0.
timcassell force-pushed the stabilize-memorydiagnoser branch from c192cb5 to 07d4c4c Compare This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
| Update: I found the culprit and opened an issue in dotnet/runtime. dotnet/runtime#101536 |
timcassell force-pushed the stabilize-memorydiagnoser branch from 4d86cd1 to 31f280d Compare | I disabled the flaky tests and left a todo to re-enable them when that issue is fixed. This is as accurate as we can make the memory diagnoser from our side. I think this should be good to merge now, unless you have any further feedback @adamsitnik. |
timcassell force-pushed the stabilize-memorydiagnoser branch 2 times, most recently from 060ce34 to 63b62a9 Compare | Ok, I re-enabled the tests and added a finalizer thread blocker to make the tests less flaky (nice idea @jkotas). This does not have any effect on real world measurements, though. |
timcassell force-pushed the stabilize-memorydiagnoser branch from 2548395 to f6a4194 Compare | Wow, even with tiered jit disabled and finalizer thread blocked, the CI is still flaky. 2024-04-28T00:50:55.0136465Z Failed AllocationQuantumIsNotAnIssueForNetCore21Plus(toolchain: .NET 8.0) [9 s] 2024-04-28T00:50:55.0137358Z Error Message: 2024-04-28T00:50:55.0137773Z Assert.Equal() Failure: Values differ 2024-04-28T00:50:55.0138293Z Expected: 88 2024-04-28T00:50:55.0138634Z Actual: 176I don't know what causes that behavior, but I just moved the MemoryDiagnoserTests to ManualRunning to stabilize the CI. |
My guess is that the next problem are the event source background threads. |
Can you provide more details? I couldn't find information about it from google. |
| EventSource and EventPipe create background threads that may allocate. These background threads should be only doing work if there is something listening to the events. It turns out that there is pretty much always something listening to .NET runtime events in typical machine configs these days. |
| You mentioned disabling that in the other issue. How can we do that? |
5 participants
This doesn't fix the zero-alloc measurement, but it does fix a bug from the threading diagnoser interfering with the results. The other changes give us the highest confidence that any errant measurements are not our fault. The rest is up to the runtime.Fixes #1542