Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

WithCustomBuildConfiguration breaks build #2425

Open
timcassell opened this issue Sep 3, 2023 Discussed in #2424 · 7 comments · May be fixed by #2563
Open

WithCustomBuildConfiguration breaks build #2425

timcassell opened this issue Sep 3, 2023 Discussed in #2424 · 7 comments · May be fixed by #2563
Labels
bug

Comments

@timcassell
Copy link
Collaborator

Discussed in #2424

Originally posted by SebastianStehle September 2, 2023
Hi,

I want to benchmark with local version of the library with previous nuget versions. Therefore I have created the following setup:

Code for that can be found at:
https://github.com/SebastianStehle/mjml-net/blob/main/Mjml.Net.Benchmark/TemplateBenchmarks.cs

  1. Define multiple jobs:
var baseJob = Job.ShortRun;

            AddJob(baseJob
                .WithId("Dev").WithBaseline(true));

            AddJob(baseJob.WithCustomBuildConfiguration("V1_24")
                .WithId("1.24.0"));

            AddJob(baseJob.WithCustomBuildConfiguration("V2_0")
                .WithId("2.0.0"));

            AddJob(baseJob.WithCustomBuildConfiguration("V2_1")
                .WithId("2.1.0"));
  1. Load the nuget version based on configuration:
<ItemGroup Condition="'$(Configuration)'=='V1_24'">
	<PackageReference Include="Mjml.Net" Version="1.24.0" />
  </ItemGroup>
	
  <ItemGroup Condition="'$(Configuration)'=='V2_0'">
	<PackageReference Include="Mjml.Net" Version="2.0.0" />
  </ItemGroup>
	
  <ItemGroup Condition="'$(Configuration)'=='V2_1'">
	<PackageReference Include="Mjml.Net" Version="2.1.0-beta1" />
  </ItemGroup>

  <ItemGroup Condition="'$(Configuration)'=='Debug'">
    <ProjectReference Include="..\Mjml.Net\Mjml.Net.csproj" />
  </ItemGroup>

  <ItemGroup Condition="'$(Configuration)'=='Release'">
    <ProjectReference Include="..\Mjml.Net\Mjml.Net.csproj" />
  </ItemGroup>

But when I run it I get errors like this:

// Execute: dotnet 8013b23c-cfa0-4303-a97b-ce66f7c64415.dll --anonymousPipes 1412 4632 --benchmarkName "Mjml.Net.Benchmarking.TemplateBenchmarks.Render_Template_Minify(MjmlTemplateFilePath: \"./Templates/Amario.mjml\")" --job 1.24.0 --benchmarkId 1 in D:\mjml\mjml-net\Mjml.Net.Benchmark\bin\Release\net7.0\8013b23c-cfa0-4303-a97b-ce66f7c64415\bin\V1_24\net7.0
// BeforeAnythingElse

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.IO.FileNotFoundException: Could not load file or assembly 'Mjml.Net, Version=2.0.0.0, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.
File name: 'Mjml.Net, Version=2.0.0.0, Culture=neutral, PublicKeyToken=null'
   at Mjml.Net.Benchmarking.TemplateBenchmarks..ctor()
   at BenchmarkDotNet.Autogenerated.Runnable_1..ctor() in D:\mjml\mjml-net\Mjml.Net.Benchmark\bin\Release\net7.0\8013b23c-cfa0-4303-a97b-ce66f7c64415\8013b23c-cfa0-4303-a97b-ce66f7c64415.notcs:line 395
   at BenchmarkDotNet.Autogenerated.Runnable_1.Run(IHost host, String benchmarkName) in D:\mjml\mjml-net\Mjml.Net.Benchmark\bin\Release\net7.0\8013b23c-cfa0-4303-a97b-ce66f7c64415\8013b23c-cfa0-4303-a97b-ce66f7c64415.notcs:line 339
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
   --- End of inner exception stack trace ---
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
   at BenchmarkDotNet.Autogenerated.UniqueProgramName.AfterAssemblyLoadingAttached(String[] args) in D:\mjml\mjml-net\Mjml.Net.Benchmark\bin\Release\net7.0\8013b23c-cfa0-4303-a97b-ce66f7c64415\8013b23c-cfa0-4303-a97b-ce66f7c64415.notcs:line 57
// AfterAll

For me it seems that it tries to build with the wrong version. If I check the bin folder, I actually see the 2.0.0 version of the dll. there, but if I build with dotnet build -c V1_24 the correct version is used.

It seems that i am doing something wrong.
@timcassell timcassell added the bug label Sep 3, 2023
@timcassell timcassell mentioned this issue Sep 3, 2023
Closed
@timcassell
Copy link
Collaborator Author

@ViktorHofer Any idea why building these in parallel fails or is incorrect? Even with the changes in #2393 where binary/intermediate outputs are forced to different directories, it still fails.

@timcassell timcassell mentioned this issue Apr 13, 2024
Open
@rainersigwald
Copy link
Member

NuGet does not support PackageReferences that are conditional on anything other than TargetFramework.

This problem is the reason why: the project.assets.json file is written to the obj directory (not a config- or target-framework-specific subfolder thereof), so different configurations share the same assets file. That means that other conditions can work as long as they're restored and then built with no intermediate step. If you restore for one of the other configurations before building, you'll get the behavior you're describing.

There are two workarounds:

  1. Build and restore sequentially so the assets file always has the right contents for the current configuration.
  2. Move the assets file location to a place that is unique for your given configuration. The easiest way to do that is to set BaseIntermediateOutputPath in a Directory.Build.props.

Note that neither gets you to a supported configuration, but they can be made to work.

@timcassell
Copy link
Collaborator Author

@rainersigwald I tried setting BaseIntermediateOutputPath via the dotnet command, but when I do, I get errors.

// Build Error: Standard output:

 Standard error:
 MSBuild version 17.9.4+90725d08d for .NET
  Determining projects to restore...
  Restored C:\Users\Tim\Downloads\mjml-net-main\Mjml.Net.Benchmark\Mjml.Net.Benchmark.csproj (in 576 ms).
C:\Program Files\dotnet\sdk\8.0.201\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(266,5): error NETSDK1005: Assets file 'C:\Users\Tim\Downloads\mjml-net-main\Mjml.Net.Benchmark\bin\Release\net7.0\5f8e1062-55e6-4907-a09e-54dd45bb6b9e\obj\V2_1\net7.0\project.assets.json' doesn't have a target for 'netstandard2.0'. Ensure that restore has run and that you have included 'netstandard2.0' in the TargetFrameworks for your project. [C:\BenchmarkDotNet\src\BenchmarkDotNet.Annotations\BenchmarkDotNet.Annotations.csproj::TargetFramework=netstandard2.0]
Build FAILED.
C:\Program Files\dotnet\sdk\8.0.201\Sdks\Microsoft.NET.Sdk\targets\Microsoft.PackageDependencyResolution.targets(266,5): error NETSDK1005: Assets file 'C:\Users\Tim\Downloads\mjml-net-main\Mjml.Net.Benchmark\bin\Release\net7.0\5f8e1062-55e6-4907-a09e-54dd45bb6b9e\obj\V2_1\net7.0\project.assets.json' doesn't have a target for 'netstandard2.0'. Ensure that restore has run and that you have included 'netstandard2.0' in the TargetFrameworks for your project. [C:\BenchmarkDotNet\src\BenchmarkDotNet.Annotations\BenchmarkDotNet.Annotations.csproj::TargetFramework=netstandard2.0]
    0 Warning(s)
    1 Error(s)
Time Elapsed 00:00:04.00
.AppendArgument($"/p:IntermediateOutputPath=\"{artifactsPaths.IntermediateDirectoryPath}{Path.AltDirectorySeparatorChar}\"")
.AppendArgument($"/p:BaseIntermediateOutputPath=\"{artifactsPaths.IntermediateDirectoryPath}{Path.AltDirectorySeparatorChar}\"")
.AppendArgument($"/p:OutDir=\"{artifactsPaths.BinariesDirectoryPath}{Path.AltDirectorySeparatorChar}\"")
.AppendArgument($"/p:OutputPath=\"{artifactsPaths.BinariesDirectoryPath}{Path.AltDirectorySeparatorChar}\"")

Removing BaseIntermediateOutputPath and leaving IntermediateOutputPath, OutDir, and OutputPath it works fine when run sequentially. What's wrong with this method?

@timcassell
Copy link
Collaborator Author

Nevermind, I found my answer at dotnet/sdk#2003 (comment). Setting ArtifactsPath instead works (though it requires sdk 8+).

@timcassell
Copy link
Collaborator Author

timcassell commented Apr 15, 2024 •
edited

Well, setting ArtifactsPath seems to work for every runtime except wasm (and probably MonoAOTLLVM, but we don't have any tests for it). Might you have any insight on that @LoopedBard3? https://github.com/dotnet/BenchmarkDotNet/actions/runs/8696789893

@timcassell timcassell linked a pull request Apr 17, 2024 that will close this issue
Open
@matouskozak
Copy link
Member

Well, setting ArtifactsPath seems to work for every runtime except wasm (and probably MonoAOTLLVM, but we don't have any tests for it). Might you have any insight on that @LoopedBard3? https://github.com/dotnet/BenchmarkDotNet/actions/runs/8696789893

@timcassell do you have any more information about why the benchmarks are crashing. I took a look at the run you posted and there seems to be only mentions about "Toolchain Build Failure" without any specific information what failed during the build stage.

@timcassell
Copy link
Collaborator Author

@matouskozak Unfortunately I don't have any more details than what are in those logs. I can't run wasm benchmarks on my machine since it doesn't support windows.

@timcassell timcassell mentioned this issue May 22, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix parallel builds dotnet/BenchmarkDotNet
Only build sequentially dotnet/BenchmarkDotNet
3 participants
@rainersigwald @timcassell @matouskozak