Add a UTF-8 statistics handler

[verdie-g]

Closes #2412.

The statistics payload can be quite large (several MB) and currently it can be copied twice in the managed code:

1. The payload is received from the C driver
2. A .NET string is allocated for the JSON
3. The statistics handler is called
4. The subscriber will likely call System.Text.Json.Serialize.Deserialize which cause the string to be converted back to UTF-8

This round-trip causes important memory fragmentation which puts pressure on the GC. In .NET 6, this can be avoided using Span.

Here is a benchmark showing the potential gains. It uses a 8 MB JSON from a production Kafka producer:

[MemoryDiagnoser] public unsafe class JsonStats { private byte* _jsonBytesPtr; private int _jsonBytesLen; [GlobalSetup] public void GlobalSetup() { byte[] jsonBytes = File.ReadAllBytes("kafka.json"); _jsonBytesPtr = (byte*)NativeMemory.Alloc((UIntPtr)jsonBytes.Length); _jsonBytesLen = jsonBytes.Length; jsonBytes.AsSpan().CopyTo(new Span<byte>(_jsonBytesPtr, _jsonBytesLen)); } [GlobalCleanup] public void GlobalCleanup() { NativeMemory.Free(_jsonBytesPtr); } [Benchmark] public void Old() { string json = Encoding.UTF8.GetString(_jsonBytesPtr, _jsonBytesLen); using var doc = JsonSerializer.Deserialize<JsonDocument>(json); } [Benchmark] public void New() { ReadOnlySpan<byte> json = new(_jsonBytesPtr, _jsonBytesLen); using var doc = JsonSerializer.Deserialize<JsonDocument>(json); } }

Method	Mean	Error	StdDev	Gen0	Gen1	Gen2	Allocated
Old	34.58 ms	0.511 ms	0.453 ms	200.0000	200.0000	200.0000	21742.48 KB
New	29.25 ms	0.032 ms	0.028 ms	-	-	-	256.2 KB

One uses 21 MB of memory while the other 256 KB, but most importantly, the new solution does not promote objects to the LOH.

[cla-assistant]

All committers have signed the CLA.

[cla-assistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[confluent-cla-assistant]

🎉 All Contributor License Agreements have been signed. Ready to merge.
✅ verdie-g
_{Please push an empty commit if you would like to re-run the checks to verify CLA status for all contributors.}

[verdie-g]

@emasab @rayokota could you have a look at this PR, it's been open for more than a year.