IntroArgumentsSource sample does not work for me #2195
Description
Ran into trouble adding ArgumentsSource to my benchmark and found that even the sample code does not generate the expected results (note the "?")
| Method | time | x | y | Mean | Error | StdDev | | --------------- | ----------------- | --- | --- | -------------------:| ----------------:| ----------------:| | SingleArgument | 00:00:00.0100000 | ? | ? | 15,780,658.958 ns | 53,493.3152 ns | 50,037.6802 ns | | SingleArgument | 00:00:00.1000000 | ? | ? | 110,181,308.000 ns | 517,614.3512 ns | 484,176.7852 ns | | ManyArguments | ? | 1 | 1 | 3.135 ns | 0.0852 ns | 0.1326 ns | | ManyArguments | ? | 2 | 2 | 13.571 ns | 0.2180 ns | 0.1933 ns | | ManyArguments | ? | 4 | 4 | 13.478 ns | 0.2188 ns | 0.1940 ns | | ManyArguments | ? | 10 | 10 | 13.471 ns | 0.2294 ns | 0.2034 ns | The following warning is shown:
// * Warnings * MultimodalDistribution IntroArgumentsSource.ManyArguments: Default -> It seems that the distribution is bimodal (mValue = 3.44) This occurs with .net5 or .net6 console app, built with VS 2022, using BenchmarkDotNet 0.13.2 (currently the latest stable release)
using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Running; using CommandLine; using System; using System.Collections.Generic; using System.Threading; using System.Timers; namespace BenchmarkBug { public class IntroArgumentsSource { [Benchmark] [ArgumentsSource(nameof(Numbers))] public double ManyArguments(double x, double y) => Math.Pow(x, y); public IEnumerable<object[]> Numbers() // for multiple arguments it's an IEnumerable of array of objects (object[]) { yield return new object[] { 1.0, 1.0 }; yield return new object[] { 2.0, 2.0 }; yield return new object[] { 4.0, 4.0 }; yield return new object[] { 10.0, 10.0 }; } [Benchmark] [ArgumentsSource(nameof(TimeSpans))] public void SingleArgument(TimeSpan time) => Thread.Sleep(time); public IEnumerable<object> TimeSpans() // for single argument it's an IEnumerable of objects (object) { yield return TimeSpan.FromMilliseconds(10); yield return TimeSpan.FromMilliseconds(100); } } internal class Program { static void Main(string[] args) { BenchmarkRunner.Run<IntroArgumentsSource>(); } } }