Dependencies and targeting for a custom VS test provider

[andyjohnson0]

Hi. I'm looking at writing a custom test provider for VS2022. Briefly, I have two questions:

1. Should I target .NET or .NET Framework?
2. Which version of Microsoft.VisualStudio.TestPlatform.ObjectModel should I be using, and from where?

More details:

I originally just tried stubbing ITestRuntimeProvider in a .NET 8 dll project using v17.12.0 of Microsoft.VisualStudio.TestPlatform.ObjectModel, Using a test project with a custom .runsettings file I was able to see that my dll was being loaded but the provider wasn't being instantiated.

Looking at Microsoft.VisualStudio.TestWindow.dll that ships with VS I can see that its built for .NET framework 4.6.2 and gives its AssemblyVersion attribute as 15.0.0.0. So I hypothesized that maybe the test runtime was using the .net framework and couldn't load .NET assemblies. I tried building an ITestRuntimeProvider stub as a framework assembly with v15.0.0.0 of Microsoft.VisualStudio.TestPlatform.ObjectModel from nuget - but this doesn't appear to provide a definition of ITestRuntimeProvider (although VS's private version does).

So now I don't know whether my initial .NET 8 approach was correct,, or whether I should use the framework with a different reference source for the ObjectModel dll. Or something else that I'm currently unaware of.

I'd appreciate any clarification on this. Thanks.

[nohwnd]

Hello, what is your goal? I am assuming you want to integrate a new testing framework? Or a whole new way of running tests?

You most likely don't want to implement ITestRuntimeProvider. Instead you want to implement test discoverer and test executor. Example of that is here: https://github.com/nohwnd/intent

You should ship it as netstandard2.0. And don't forget to name the dll *.TestAdapter.dll to be discovered.

For object model you should choose the version that is equal to the oldest VS you want to support, and use it as ref (not ship it).

[andyjohnson0]

Hello, what is your goal? I am assuming you want to integrate a new testing framework? Or a whole new way of running tests?

Its something like implementing a new testing framework

You most likely don't want to implement ITestRuntimeProvider. Instead you want to implement test discoverer and test executor. Example of that is here: https://github.com/nohwnd/intent

Thanks, but I need the TestProcessStartInfo so as far as I can tell I need to implement ITestRuntimeProvider.

You should ship it as netstandard2.0. And don't forget to name the dll *.TestAdapter.dll to be discovered.

For object model you should choose the version that is equal to the oldest VS you want to support, and use it as ref (not ship it).

I've tried that, with v17 of the obj model (for vs 2022), but still don't see my provider being instantiated.

Anyway, I suspect that what I'm seeing isn't an issue with the test platform - but is something I'm either not doing or doing wrong, so I'll close this ticket. Thanks!