vstest.console.exe cannot resolve assembly for extension with target framework .net8.0, however dotnet test (vstest.console.dll) resolves it correctly

[VladimirNilov]

Description

I'm trying to use the next extension in my test porject, when I'm running my tests via VS2022 or directly via the vstest.console.exe, it looks like it cannot correctly resolve the extension assembly via the vstest.console.exe:

TpTrace Verbose: 0 : 12032, 1, 2024/07/17, 19:29:47.635, 4868177793925, vstest.console.exe, MetadataReaderExtensionsHelper: Discovering extensions inside assembly 'ReportPortal.VSTest.TestLogger, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' file path 'C:\Users\test\source\repos\test_project\Tests\bin\Debug\net8.0\ReportPortal.VSTest.TestLogger.dll'

TpTrace Information: 0 : 12032, 1, 2024/07/17, 19:29:47.651, 4868177860464, vstest.console.exe, AssemblyResolver.OnResolve: System.Runtime, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a: Resolving assembly.

at the same time, when I'm running the tests via the dotnet test, it resolves everything correctly
TpTrace Verbose: 0 : 8448, 1, 2024/07/17, 19:29:57.585, 4868277158270, vstest.console.dll, MetadataReaderExtensionsHelper: Discovering extensions inside assembly 'ReportPortal.VSTest.TestLogger, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' file path 'C:\Users\test\source\repos\test_project\Tests\bin\Debug\net8.0\ReportPortal.VSTest.TestLogger.dll'

TpTrace Information: 0 : 8448, 1, 2024/07/17, 19:29:57.586, 4868277166350, vstest.console.dll, AssemblyResolver.OnResolve: ReportPortal.Shared: Resolving assembly.

the same issue was described for the package here and here
also GenerateBindingRedirectsOutputType true didn't help

Steps to reproduce

1. Add this extension and include it into your run settings (for TestAdaptersPaths please use the full path)
2. Try to run a test via the Test Explorer or the vstest.console.exe

Expected behavior

The assemblies should be resolved correctly, the same how it works via dotnet test

Actual behavior

TpTrace Warning: 0 : 14804, 1, 2024/07/18, 02:17:15.668, 5112670068026, vstest.console.exe, TestPluginDiscoverer-FileLoadException: Failed to load extensions from file 'C:\Users\test\source\repos\test_project\Tests\bin\Debug\net8.0\ReportPortal.VSTest.TestLogger.dll'. Skipping test extension scan for this file. Error: System.IO.FileLoadException: Could not load file or assembly 'System.Runtime, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
File name: 'System.Runtime, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' ---> System.IO.FileLoadException: Could not load file or assembly 'System.Runtime, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
File name: 'System.Runtime, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'
at System.Reflection.RuntimeAssembly.nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly locationHint, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
at System.Reflection.RuntimeAssembly.InternalLoadAssemblyName(AssemblyName assemblyRef, Evidence assemblySecurity, RuntimeAssembly reqAssembly, StackCrawlMark& stackMark, IntPtr pPrivHostBinder, Boolean throwOnFileNotFound, Boolean forIntrospection, Boolean suppressSecurityChecks)
at System.Reflection.Assembly.Load(AssemblyName assemblyRef)
at Microsoft.VisualStudio.TestPlatform.Common.ExtensionFramework.TestPluginCache.CurrentDomainAssemblyResolve(Object sender, AssemblyResolveEventArgs args) in //src/Microsoft.TestPlatform.Common/ExtensionFramework/TestPluginCache.cs:line 513
at System.AppDomain.OnAssemblyResolveEvent(RuntimeAssembly assembly, String assemblyFullName)

Diagnostic logs

Added the diagnostic message logs above

Environment

Local: VS2022 (recent Community and recent Preview Version 17.11.0 Preview 4.0), .net8.0,
CI: _tool\VsTest\17.10.0\x64\tools\net462\Common7\IDE\Extensions\TestPlatform

PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.10.0" />
PackageReference Include="MSTest.TestAdapter" Version="3.5.0" />
PackageReference Include="MSTest.TestFramework" Version="3.5.0" />

[nohwnd]

vstest.console.exe is a .NET Framework application. .NET Framework cannot load .NET assemblies.

The linked extension is targeting .NET Framework and .NET Standard, but the error suggests that you are in fact loading a dll that targets .NET 8. This won't work, you either have to retarget the extension, or not use it with vstest.console.exe.

[VladimirNilov]

vstest.console.exe is a .NET Framework application. .NET Framework cannot load .NET assemblies.

The linked extension is targeting .NET Framework and .NET Standard, but the error suggests that you are in fact loading a dll that targets .NET 8. This won't work, you either have to retarget the extension, or not use it with vstest.console.exe.

the logs that I've provided after the lib update into net8.0, it looks like the problem is related to the target framework for Microsoft.TestPlatform.ObjectModel that probably is not compatible with net8.0

[VladimirNilov]

ah yes, it's the same as you mentioned that vstest.console.exe is a .NET Framework application. .NET Framework cannot load .NET assemblies.

thanks for the details