Update mono to support shuffle for constant inputs

[tannergooding]

Mono wasn't accelerating shuffle for anything other than byte/sbyte on x64. This was in part because doing so required detecting inputs as constant which didn't exist when that support was first brought online.

However, I had since added basic support for constants last year in #81902. This PR adds support that mirrors most of the RyuJIT Shuffle importation logic and so we can now do the right thing. As a side effect, the general support for generating such constants for Create calls when all inputs are constants was also expanded which should have some indirect benefits in other places.

Shuffle having not being accelerated is very impactful to the entire BCL and I would expect this to have a broad impact across many core benchmarks including HexConverter, XxHash, TensorPrimitives, ReverseEndianness, Base64Decoder, Guid, Matrix4x4, Quaternion, IndexOfAny, ProbablisticMap, various SpanHelpers APIs and more.

[dotnet-policy-service]

Tagging subscribers to this area: @lambdageek, @steveisok
See info in area-owners.md if you want to be subscribed.

[tannergooding]

No diffs for x64 is expected above since the x64 changes are MonoLLVM only.

For arm64 it should be both MonoJIT and MonoLLVM

[EgorBo]

@EgorBot -arm64 -mono --envvars "MONO_VERBOSE_METHOD:ShuffleBench"

using System.Numerics; using BenchmarkDotNet.Attributes; using System.Runtime.Intrinsics; public class StrengthReduction { Vector128<int> srcVec = Vector128.Create(1, 2, 3, 4); Vector128<int> dstVec; [Benchmark] public void ShuffleBench() => dstVec = Vector128.Shuffle(srcVec, Vector128.Create(2, 2, 2, 2)); [Benchmark] public Quaternion ConcatenateBenchmark() => Quaternion.Concatenate(Quaternion.Identity, Quaternion.Identity); }

[EgorBot]

Benchmark results on Arm64

BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish) Unknown processor Job-DMCBWN : .NET 9.0.0 (42.42.42.42424) using MonoVM, Arm64 ArmBase Job-EBPPKC : .NET 9.0.0 (42.42.42.42424) using MonoVM, Arm64 ArmBase EnvironmentVariables=MONO_VERBOSE_METHOD=ShuffleBench 

Method	Toolchain	Mean	Error	Ratio
ShuffleBench	Main	17.3048 ns	0.0015 ns	1.00
ShuffleBench	PR	0.2047 ns	0.0075 ns	0.01
ConcatenateBenchmark	Main	120.1574 ns	0.0091 ns	1.00
ConcatenateBenchmark	PR	NA	NA	?
Benchmarks with issues:
StrengthReduction.ConcatenateBenchmark: Job-EBPPKC(EnvironmentVariables=MONO_VERBOSE_METHOD=ShuffleBench, Toolchain=PR)

BDN_Artifacts.zip

[tannergooding]

Need to fix a MonoLLVM assert, but...

Before:

il_seq_point intr il: 0x0 move R33 <- R32 move R34 <- R32 loadx_membase R35 <- [R32 + 0x10] [System.Runtime.Intrinsics.Vector128`1<int>] iconst R36 <- [2] iconst R37 <- [2] iconst R38 <- [2] iconst R39 <- [2] xzero R40 <- [System.Runtime.Intrinsics.Vector128`1<int>] insert_i4 R41 <- R40 R36 insert_i4 R42 <- R41 R37 insert_i4 R43 <- R42 R38 insert_i4 R44 <- R43 R39 il_seq_point il: 0x10, nonempty-stack outarg_vtretaddr R46 <- R45 move R47 <- R46 outarg_vt R35 outarg_vt R44 xcall R45 <- [System.Runtime.Intrinsics.Vector128`1<int> System.Runtime.Intrinsics.Vector128:Shuffle (System.Runtime.Intrinsics.Vector128`1<int>,System.Runtime.Intrinsics.Vector128`1<int>)] il_seq_point il: 0x15, nonempty-stack compare_imm R32 [0] cond_exc_eq NullReferenceException storex_membase [R32 + 0x20] <- R45 [System.Runtime.Intrinsics.Vector128`1<int>] il_seq_point il: 0x1a nop 

After:

il_seq_point intr il: 0x0 move R33 <- R32 move R34 <- R32 loadx_membase R35 <- [R34 + 0x10] [System.Runtime.Intrinsics.Vector128`1<int>] iconst R36 <- [2] iconst R37 <- [2] iconst R38 <- [2] iconst R39 <- [2] xzero R40 <- [System.Runtime.Intrinsics.Vector128`1<int>] xconst R41 <- il_seq_point il: 0x10, nonempty-stack xones R42 <- xop_ovr_x_x_x R43 <- R35 R42 il_seq_point il: 0x15, nonempty-stack compare_imm R33 [0] cond_exc_eq NullReferenceException storex_membase [R33 + 0x20] <- R43 [System.Runtime.Intrinsics.Vector128`1<int>] il_seq_point il: 0x1a br [B1] 

[tannergooding]

@matouskozak or @fanyang-mono, is there an easy way to remove MonoInst * that are known to be dead or is it perhaps handled elsewhere?

In a case like this, element will be dead because we're just folding it directly into the constant. In the case that ins is xzero or xones we will likewise be replacing it with an xconst that includes the underlying constant from element.

So in the example given above, the following IR is "dead":

iconst R36 <- [2] iconst R37 <- [2] iconst R38 <- [2] iconst R39 <- [2] xzero R40 <- [System.Runtime.Intrinsics.Vector128`1<int>] 

[matouskozak]

I haven't tried it but I think MONO_REMOVE_INS (or MONO_DELETE_INS) could work for this.

[tannergooding]

This should now be ready for review.

-- Please feel free to call out anywhere I messed up the styling. Tried to follow the conventions, but it's entirely possible I made some mistakes.

[EgorBo]

@EgorBot -arm64 -mono --envvars "MONO_VERBOSE_METHOD:ShuffleBench"

using System.Numerics; using BenchmarkDotNet.Attributes; using System.Runtime.Intrinsics; public class StrengthReduction { Vector128<int> srcVec = Vector128.Create(1, 2, 3, 4); Vector128<int> dstVec; [Benchmark] public void ShuffleBench() => dstVec = Vector128.Shuffle(srcVec, Vector128.Create(2, 2, 2, 2)); [Benchmark] public Quaternion ConcatenateBenchmark() => Quaternion.Concatenate(Quaternion.Identity, Quaternion.Identity); }

[EgorBot]

Benchmark results on Arm64

BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish) Unknown processor Job-OKJKWC : .NET 9.0.0 (42.42.42.42424) using MonoVM, Arm64 ArmBase Job-CZGEXJ : .NET 9.0.0 (42.42.42.42424) using MonoVM, Arm64 ArmBase EnvironmentVariables=MONO_VERBOSE_METHOD=ShuffleBench 

Method	Toolchain	Mean	Error	Ratio
ShuffleBench	Main	17.3735 ns	0.0047 ns	1.00
ShuffleBench	PR	0.2321 ns	0.0052 ns	0.01
ConcatenateBenchmark	Main	118.2752 ns	0.0251 ns	1.00
ConcatenateBenchmark	PR	7.3540 ns	0.0022 ns	0.06

BDN_Artifacts.zip

[tannergooding]

Both of those numbers for MonoJIT are very good improvements:

BenchmarkDotNet v0.13.12, Ubuntu 22.04.4 LTS (Jammy Jellyfish) Unknown processor Job-OKJKWC : .NET 9.0.0 (42.42.42.42424) using MonoVM, Arm64 ArmBase Job-CZGEXJ : .NET 9.0.0 (42.42.42.42424) using MonoVM, Arm64 ArmBase EnvironmentVariables=MONO_VERBOSE_METHOD=ShuffleBench 

Method	Toolchain	Mean	Error	Ratio
ShuffleBench	Main	17.3735 ns	0.0047 ns	1.00
ShuffleBench	PR	0.2321 ns	0.0052 ns	0.01
ConcatenateBenchmark	Main	118.2752 ns	0.0251 ns	1.00
ConcatenateBenchmark	PR	7.3540 ns	0.0022 ns	0.06

I had done some local testing for a few other scenarios and seen similar improvements to APIs using Shuffle. The IR from MONO_VERBOSE_METHOD generally looks good too, but I think there's a broader possibility to trim out some MonoInst in a later phase if they're unused. Such trimming I think is better suited for .NET 10, however.

[tannergooding]

For ShuffleBench the codegen is as below. There's still a few places that could be optimized further, but I don't think those are as prevalent for .NET 9 and are likely more involved/general-purpose.

Before

 0:	a9ba7bfd	stp	x29, x30, [sp, #-96]!  4:	910003fd movx29, sp  8:	f9000bba strx26, [x29, #16]  c:	aa0003fa movx26, x0  10:	d29c1000 movx0, #0xe080	// #57472  14:	f2b6d620	movk	x0, #0xb6b1, lsl #16  18:	f2defa00	movk	x0, #0xf7d0, lsl #32  1c:	f9400011	ldr	x17, [x0]  20:	b4000051	cbz	x17, 0x28  24:	954b13d1 bl0x52c4f68  28:	3dc00740	ldr	q0, [x26, #16]  2c:	3d800fa0 strq0, [x29, #48]  30:	d2800044 movx4, #0x2	// #2  34:	d2800043 movx3, #0x2	// #2  38:	d2800042 movx2, #0x2	// #2  3c:	d2800041 movx1, #0x2	// #2  40:	4f00e400	movi	v0.16b, #0x0  44:	4e041c80 movv0.s[0], w4  48:	4e0c1c60 movv0.s[1], w3  4c:	4e141c40 movv0.s[2], w2  50:	4e1c1c20 movv0.s[3], w1  54:	3d8013a0 strq0, [x29, #64]  58:	910083a0 addx0, x29, #0x20  5c:	f9002ba0 strx0, [x29, #80]  60:	f9401ba0	ldr	x0, [x29, #48]  64:	f9401fa1	ldr	x1, [x29, #56]  68:	f94023a2	ldr	x2, [x29, #64]  6c:	f94027a3	ldr	x3, [x29, #72]  70:	9400001c bl0xe0  74:	f9402bbe	ldr	x30, [x29, #80]  78:	a90007c0	stp	x0, x1, [x30]  7c:	eb1f035f cmpx26, xzr  80:10000011	adr	x17, 0x80  84:	540000e0	b.eq0xa0 // b.none  88:	3dc00ba0	ldr	q0, [x29, #32]  8c:	3d800b40 strq0, [x26, #32]  90:	f9400bba	ldr	x26, [x29, #16]  94:	910003bf movsp, x29  98:	a8c67bfd	ldp	x29, x30, [sp], #96  9c:	d65f03c0 ret  a0:	d28027c0 movx0, #0x13e	// #318  a4:	aa1103e1 movx1, x17  a8:	954b50ac bl0x52d4358

After:

 0:	a9be7bfd	stp	x29, x30, [sp, #-32]!  4:	910003fd movx29, sp  8:	f9000bba strx26, [x29, #16]  c:	aa0003fa movx26, x0  10:	3dc00740	ldr	q0, [x26, #16]  14:	9c0001e1	ldr	q1, 0x50  18:	4e010000	tbl	v0.16b, {v0.16b}, v1.16b  1c:	eb1f035f cmpx26, xzr  20:10000011	adr	x17, 0x20  24:	540000c0	b.eq0x3c // b.none  28:	3d800b40 strq0, [x26, #32]  2c:	f9400bba	ldr	x26, [x29, #16]  30:	910003bf movsp, x29  34:	a8c27bfd	ldp	x29, x30, [sp], #32  38:	d65f03c0 ret  3c:	d28027c0 movx0, #0x13e	// #318  40:	aa1103e1 movx1, x17  44:94000007 bl0x60  48:	b9006740 strw0, [x26, #100]  4c:	b9806740	ldrsw	x0, [x26, #100]  50:	0b0a0908 addw8, w8, w10, lsl #2  54:	0b0a0908 addw8, w8, w10, lsl #2  58:	0b0a0908 addw8, w8, w10, lsl #2  5c:	0b0a0908 addw8, w8, w10, lsl #2

[fanyang-mono]

LGTM! Just some minor format adjustment needed. Thanks for adding this for mono.

[kg]

I'm not sure whether interp does this kind of optimization, but jiterpreter does, just FYI. If it becomes important we could probably move this optimization from the jiterpreter up into the interpreter, but I think the only target where interpreter performance really matters right now is WASM.