How to use WithCustomBuildConfiguration?

[SebastianStehle]

Hi,

I want to benchmark with local version of the library with previous nuget versions. Therefore I have created the following setup:

Code for that can be found at:
https://github.com/SebastianStehle/mjml-net/blob/main/Mjml.Net.Benchmark/TemplateBenchmarks.cs

1. Define multiple jobs:

var baseJob = Job.ShortRun;

            AddJob(baseJob
                .WithId("Dev").WithBaseline(true));

            AddJob(baseJob.WithCustomBuildConfiguration("V1_24")
                .WithId("1.24.0"));

            AddJob(baseJob.WithCustomBuildConfiguration("V2_0")
                .WithId("2.0.0"));

            AddJob(baseJob.WithCustomBuildConfiguration("V2_1")
                .WithId("2.1.0"));

2. Load the nuget version based on configuration:

<ItemGroup Condition="'$(Configuration)'=='V1_24'">
	<PackageReference Include="Mjml.Net" Version="1.24.0" />
  </ItemGroup>
	
  <ItemGroup Condition="'$(Configuration)'=='V2_0'">
	<PackageReference Include="Mjml.Net" Version="2.0.0" />
  </ItemGroup>
	
  <ItemGroup Condition="'$(Configuration)'=='V2_1'">
	<PackageReference Include="Mjml.Net" Version="2.1.0-beta1" />
  </ItemGroup>

  <ItemGroup Condition="'$(Configuration)'=='Debug'">
    <ProjectReference Include="..\Mjml.Net\Mjml.Net.csproj" />
  </ItemGroup>

  <ItemGroup Condition="'$(Configuration)'=='Release'">
    <ProjectReference Include="..\Mjml.Net\Mjml.Net.csproj" />
  </ItemGroup>

But when I run it I get errors like this:

// Execute: dotnet 8013b23c-cfa0-4303-a97b-ce66f7c64415.dll --anonymousPipes 1412 4632 --benchmarkName "Mjml.Net.Benchmarking.TemplateBenchmarks.Render_Template_Minify(MjmlTemplateFilePath: \"./Templates/Amario.mjml\")" --job 1.24.0 --benchmarkId 1 in D:\mjml\mjml-net\Mjml.Net.Benchmark\bin\Release\net7.0\8013b23c-cfa0-4303-a97b-ce66f7c64415\bin\V1_24\net7.0
// BeforeAnythingElse

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.IO.FileNotFoundException: Could not load file or assembly 'Mjml.Net, Version=2.0.0.0, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.
File name: 'Mjml.Net, Version=2.0.0.0, Culture=neutral, PublicKeyToken=null'
   at Mjml.Net.Benchmarking.TemplateBenchmarks..ctor()
   at BenchmarkDotNet.Autogenerated.Runnable_1..ctor() in D:\mjml\mjml-net\Mjml.Net.Benchmark\bin\Release\net7.0\8013b23c-cfa0-4303-a97b-ce66f7c64415\8013b23c-cfa0-4303-a97b-ce66f7c64415.notcs:line 395
   at BenchmarkDotNet.Autogenerated.Runnable_1.Run(IHost host, String benchmarkName) in D:\mjml\mjml-net\Mjml.Net.Benchmark\bin\Release\net7.0\8013b23c-cfa0-4303-a97b-ce66f7c64415\8013b23c-cfa0-4303-a97b-ce66f7c64415.notcs:line 339
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
   --- End of inner exception stack trace ---
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
   at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args) in D:\mjml\mjml-net\Mjml.Net.Benchmark\bin\Release\net7.0\8013b23c-cfa0-4303-a97b-ce66f7c64415\8013b23c-cfa0-4303-a97b-ce66f7c64415.notcs:line 57
// AfterAll

For me it seems that it tries to build with the wrong version. If I check the bin folder, I actually see the 2.0.0 version of the dll. there, but if I build with dotnet build -c V1_24 the correct version is used.

It seems that i am doing something wrong.

[timcassell]

@SebastianStehle You're doing everything correctly, there is a bug on BDN side. I moved this to an issue #2425.