Vstest loads Dlls in system folder before the ones in the application folder

[danyaalk123]

Describe the issue
I'm facing an issue when using the onnxruntime.ml.microsoft package with native MSTest unit tests. When I run the tests, it seems the package loads the onnxruntime from the System32 directory rather than the application's output folder. This behavior is not observed with Google Tests.

Workaround:
I discovered that by disabling SafeDllSearchMode, the behavior can be corrected. The process is outlined in the official Microsoft documentation here.

Possible Root Cause:
I suspect the underlying cause might be how vstest starts up within the Visual Studio environment. It seems like it might be starting from a Visual Studio program folder, which causes it to prioritize the loading of DLLs from System32 over the project's output folder.

To reproduce
Create a new MSTest project and add the Microsoft.ML.OnnxRuntime package.
Write a test case that utilizes onnxruntime functionality.
Run the test case.
Urgency
No response

Platform
Windows

OS Version
Windows 10

ONNX Runtime Installation
Released Package

ONNX Runtime Version or Commit ID
1.16.0

ONNX Runtime API
C++

Architecture
X64

Execution Provider
Default CPU

Execution Provider Library Version
No response

[nohwnd]

@Evangelink this looks like a mstest issue rather than vstest. Please have a look.

[Evangelink]

Hi @danyaalk123,

Would you mind providing a simple repro? I have created the project but I don't know what you are doing with the ML model to have your error.

[danyaalk123]

@Evangelink
Thanks for looking into it

https://drive.google.com/file/d/1nLqX2MEGBZXra8jvWxrPX8FPlFeykwWR/view?usp=sharing

Old version so might have to update onnxruntime nuget package version

.Net Tests will run and pass but the native test 1 wont pass due to the wrong onnxruntime.dll being loaded.

If I was to swtich to google tests it would pass or do the regedit

[Evangelink]

@danyaalk123, the link is not valid (it's saying that it doesn't respect Google's conditions).

[avivhu]

I'm having the same problem with vstest.

onnxruntime.dll in System32 takes precedence over the onnxruntime.dll in the directory of the tested DLL.
The SafeDllSearchMode reg edit workaround also works for me.
The DLL loading order in vstest is not as I would expect, why is that?

[Evangelink]

Hi @avivhu, would it be possible for you to provide us with a repro project?

[danyaalk123]

@Evangelink
https://gofile.io/d/Plyo7b

[avivhu]

@Evangelink I tried to make an isolated repro but got bogged down in the complexity. Try @danyaalk123 's repro first.

[avivhu]

In my code I was able to work around it by calling the Win32 API
SetDllDirectory(".");

I call it before the onnxruntime.dll is loaded, and thus it will load from the working directory instead of System32, as I wanted.

The effect is similar to setting the registry key mentioned by @danyaalk123:
Set-ItemProperty -Path "HKLM:\System\CurrentControlSet\Control\Session Manager" -Name SafeDllSearchMode -Value 0

I couldn't call SetDllDirectory in @danyaalk123 sample code, because in that code the onnxruntime.dll is loaded implicitly before SetDllDirectory can run.

[danyaalk123]

@Evangelink Any Updates? Are you able to recreate the issue

[Evangelink]

I am able to see an issue with UnitTest1.TestMethod:

Doing some debugging with Process Monitor, I see that the search pattern is correct according to https://learn.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-search-order#standard-search-order-for-packaged-apps. Steps 1 to 6 nothing is specified, for step 7 the calling process is started from VS folder so resolution is tried from this folder

and then we get to step 8, loading from system32
.

FYI, you are not using MSTest test framework but cpp unit testing framework and there is no bug on vstest.console.exe logic either. This means that if something is indeed wrong, this is on cpp unit testing side so you would need to open a bug in https://developercommunity.visualstudio.com/home to have it triaged to the correct team.