Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ReleaseMutex failed. WaitOne Id: {originalThreadId} Release Id: {releaseThreadId} in Microsoft.Extensions.Logging.Generators.Roslyn4.0 #63863

Open
carlossanlop opened this issue Sep 8, 2022 · 20 comments
Open

ReleaseMutex failed. WaitOne Id: {originalThreadId} Release Id: {releaseThreadId} in Microsoft.Extensions.Logging.Generators.Roslyn4.0 #63863

carlossanlop opened this issue Sep 8, 2022 · 20 comments
Labels
Area-Infrastructure
Milestone
Backlog

Comments

@carlossanlop
Copy link
Member

Found in a release/7.0 backport PR: dotnet/runtime#75284

Please help determine if this needs a fix to get backported to 7.0.

Microsoft.Extensions.Configuration.Abstractions -> /__w/1/s/artifacts/bin/Microsoft.Extensions.Configuration.Abstractions/ref/Release/net7.0/Microsoft.Extensions.Configuration.Abstractions.dll
/__w/1/s/.dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error MSB3883: Unexpected exception:  [/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/Microsoft.Extensions.Logging.Generators.Roslyn4.0.csproj]
##[error].dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error MSB3883: (NETCORE_ENGINEERING_TELEMETRY=Build) Unexpected exception: 
/__w/1/s/.dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error : One or more errors occurred. (ReleaseMutex failed. WaitOne Id: 12 Release Id: 12) [/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/Microsoft.Extensions.Logging.Generators.Roslyn4.0.csproj]
##[error].dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error : (NETCORE_ENGINEERING_TELEMETRY=Build) One or more errors occurred. (ReleaseMutex failed. WaitOne Id: 12 Release Id: 12)
  Microsoft.Extensions.Configuration.Abstractions -> /__w/1/s/artifacts/bin/Microsoft.Extensions.Configuration.Abstractions/Release/net7.0/Microsoft.Extensions.Configuration.Abstractions.dll
@ghost
Copy link

ghost commented Sep 8, 2022

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

Found in a release/7.0 backport PR: dotnet/runtime#75284

Please help determine if this needs a fix to get backported to 7.0.

Microsoft.Extensions.Configuration.Abstractions -> /__w/1/s/artifacts/bin/Microsoft.Extensions.Configuration.Abstractions/ref/Release/net7.0/Microsoft.Extensions.Configuration.Abstractions.dll
/__w/1/s/.dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error MSB3883: Unexpected exception:  [/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/Microsoft.Extensions.Logging.Generators.Roslyn4.0.csproj]
##[error].dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error MSB3883: (NETCORE_ENGINEERING_TELEMETRY=Build) Unexpected exception: 
/__w/1/s/.dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error : One or more errors occurred. (ReleaseMutex failed. WaitOne Id: 12 Release Id: 12) [/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/Microsoft.Extensions.Logging.Generators.Roslyn4.0.csproj]
##[error].dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error : (NETCORE_ENGINEERING_TELEMETRY=Build) One or more errors occurred. (ReleaseMutex failed. WaitOne Id: 12 Release Id: 12)
  Microsoft.Extensions.Configuration.Abstractions -> /__w/1/s/artifacts/bin/Microsoft.Extensions.Configuration.Abstractions/Release/net7.0/Microsoft.Extensions.Configuration.Abstractions.dll
Author: carlossanlop
Assignees: -
Labels:

arch-wasm, area-Extensions-Logging, source-generator
Milestone: -

@ghost ghost added the untriaged Issues and PRs which have not yet been triaged by a lead label Sep 8, 2022
@ghost
Copy link

ghost commented Sep 8, 2022

Tagging subscribers to this area: @dotnet/area-extensions-logging
See info in area-owners.md if you want to be subscribed.

Issue Details

Found in a release/7.0 backport PR: dotnet/runtime#75284

Please help determine if this needs a fix to get backported to 7.0.

Microsoft.Extensions.Configuration.Abstractions -> /__w/1/s/artifacts/bin/Microsoft.Extensions.Configuration.Abstractions/ref/Release/net7.0/Microsoft.Extensions.Configuration.Abstractions.dll
/__w/1/s/.dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error MSB3883: Unexpected exception:  [/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/Microsoft.Extensions.Logging.Generators.Roslyn4.0.csproj]
##[error].dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error MSB3883: (NETCORE_ENGINEERING_TELEMETRY=Build) Unexpected exception: 
/__w/1/s/.dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error : One or more errors occurred. (ReleaseMutex failed. WaitOne Id: 12 Release Id: 12) [/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/Microsoft.Extensions.Logging.Generators.Roslyn4.0.csproj]
##[error].dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error : (NETCORE_ENGINEERING_TELEMETRY=Build) One or more errors occurred. (ReleaseMutex failed. WaitOne Id: 12 Release Id: 12)
  Microsoft.Extensions.Configuration.Abstractions -> /__w/1/s/artifacts/bin/Microsoft.Extensions.Configuration.Abstractions/Release/net7.0/Microsoft.Extensions.Configuration.Abstractions.dll
Author: carlossanlop
Assignees: -
Labels:

arch-wasm, area-Extensions-Logging, source-generator
Milestone: -

@carlossanlop
Copy link
Member Author

@lewing @radical @steveisok @akoeplinger

@radical
Copy link
Member

radical commented Sep 8, 2022

Looks similar to dotnet/runtime#70836 .

@steveisok
Copy link
Member

Should we run these tests on wasm/mobile at all?

@radical
Copy link
Member

radical commented Sep 8, 2022

This is probably getting triggered because src/libraries has changes.

@radical
Copy link
Member

radical commented Sep 8, 2022

This is not even running any tests, it's failing while building. And the will only build - Build Browser wasm Linux Release _Threading_PerfTracing_BuildOnly.

@lewing
Copy link
Member

lewing commented Sep 8, 2022

yeah this is a CSC crash it isn't really wasm related

@tarekgh
Copy link
Member

tarekgh commented Sep 8, 2022 

error : One or more errors occurred. (ReleaseMutex failed. WaitOne Id: 12 Release Id: 12) [/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/Microsoft.Extensions.Logging.Generators.Roslyn4.0.csproj]

Does this suggest something not supported on WASM?

@lewing
Copy link
Member

lewing commented Sep 8, 2022

There is no wasm running at all at this point, it looks like the generator failed for some reason.

@eerhardt
Copy link
Member

eerhardt commented Sep 8, 2022

Looks like it is coming from this line in Roslyn:

roslyn/src/Compilers/Shared/BuildServerConnection.cs

Line 245 in 3e39dd3

var message = $"ReleaseMutex failed. WaitOne Id: {originalThreadId} Release Id: {releaseThreadId}";

@radical
Copy link
Member

radical commented Sep 8, 2022 

"/__w/1/s/.packages/microsoft.dotnet.arcade.sdk/7.0.0-beta.22418.4/tools/Build.proj" (default target) (1) ->
"/__w/1/s/Build.proj" (Build target) (3:2) ->
"/__w/1/s/src/libraries/oob.proj" (default target) (333) ->
"/__w/1/s/src/libraries/shims.proj" (default target) (334) ->
"/__w/1/s/src/libraries/shims/src/WindowsBase.csproj" (default target) (362) ->
"/__w/1/s/src/libraries/oob-src.proj" (default target) (339:23) ->
"/__w/1/s/src/libraries/Microsoft.Extensions.Caching.Memory/src/Microsoft.Extensions.Caching.Memory.csproj" (default target) (366:6) ->
"/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/src/Microsoft.Extensions.Logging.Abstractions.csproj" (default target) (389:7) ->
"/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/Microsoft.Extensions.Logging.Generators.Roslyn4.0.csproj" (default target) (462:2) ->
(CoreGenerateSatelliteAssemblies target) ->
  /__w/1/s/.dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error MSB3883: Unexpected exception:  [/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/Microsoft.Extensions.Logging.Generators.Roslyn4.0.csproj]
  /__w/1/s/.dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error : One or more errors occurred. (ReleaseMutex failed. WaitOne Id: 12 Release Id: 12) [/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/Microsoft.Extensions.Logging.Generators.Roslyn4.0.csproj]

@tarekgh
Copy link
Member

tarekgh commented Sep 8, 2022 •
edited
Loading

Make sense @lewing.

Looking at the build analysis results too and I am seeing https://github.com/dotnet/runtime/pull/75284/checks?check_run_id=8258063425 all failures are on Mono only. This suggests specific to Mono.

@ghost
Copy link

ghost commented Sep 8, 2022

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

Issue Details

Found in a release/7.0 backport PR: dotnet/runtime#75284

Please help determine if this needs a fix to get backported to 7.0.

Microsoft.Extensions.Configuration.Abstractions -> /__w/1/s/artifacts/bin/Microsoft.Extensions.Configuration.Abstractions/ref/Release/net7.0/Microsoft.Extensions.Configuration.Abstractions.dll
/__w/1/s/.dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error MSB3883: Unexpected exception:  [/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/Microsoft.Extensions.Logging.Generators.Roslyn4.0.csproj]
##[error].dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error MSB3883: (NETCORE_ENGINEERING_TELEMETRY=Build) Unexpected exception: 
/__w/1/s/.dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error : One or more errors occurred. (ReleaseMutex failed. WaitOne Id: 12 Release Id: 12) [/__w/1/s/src/libraries/Microsoft.Extensions.Logging.Abstractions/gen/Microsoft.Extensions.Logging.Generators.Roslyn4.0.csproj]
##[error].dotnet/sdk/7.0.100-preview.7.22377.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(952,5): error : (NETCORE_ENGINEERING_TELEMETRY=Build) One or more errors occurred. (ReleaseMutex failed. WaitOne Id: 12 Release Id: 12)
  Microsoft.Extensions.Configuration.Abstractions -> /__w/1/s/artifacts/bin/Microsoft.Extensions.Configuration.Abstractions/Release/net7.0/Microsoft.Extensions.Configuration.Abstractions.dll
Author: carlossanlop
Assignees: -
Labels:

arch-wasm, untriaged, area-Infrastructure-mono, source-generator
Milestone: -

@lewing
Copy link
Member

lewing commented Sep 8, 2022

There is no mono running at this point either, perhaps it is a linker bug that is breaking the generator?

@tarekgh
Copy link
Member

tarekgh commented Sep 8, 2022

Ok, it looks make sense to move this to Roslyn repo then.

@tarekgh tarekgh transferred this issue from dotnet/runtime Sep 8, 2022
@lewing
Copy link
Member

lewing commented Sep 9, 2022

Actually looking at this as @eerhardt points out it appears to be the compiler service breaking

@lewing lewing changed the title WASM CI build failure: "ReleaseMutex failed" in Microsoft.Extensions.Logging.Generators.Roslyn4.0 "ReleaseMutex failed. WaitOne Id: {originalThreadId} Release Id: {releaseThreadId} in Microsoft.Extensions.Logging.Generators.Roslyn4.0 Sep 10, 2022
@lewing lewing changed the title "ReleaseMutex failed. WaitOne Id: {originalThreadId} Release Id: {releaseThreadId} in Microsoft.Extensions.Logging.Generators.Roslyn4.0 ReleaseMutex failed. WaitOne Id: {originalThreadId} Release Id: {releaseThreadId} in Microsoft.Extensions.Logging.Generators.Roslyn4.0 Sep 10, 2022
@lewing
Copy link
Member

lewing commented Sep 13, 2022

cc @MattGal

        Unhandled exception: System.IO.IOException: Connection timed out : 'Global\msbuild-server-launch-NDSZDEInz8s3G_evUwEeKheECB+4j1X89xWLxyQAqoU'
           at System.Threading.Mutex.CreateMutexCore(Boolean initiallyOwned, String name, Boolean& createdNew)
           at Microsoft.Build.Experimental.MSBuildClient.TryLaunchServer()

@MattGal
Copy link
Member

MattGal commented Sep 13, 2022

cc @MattGal

        Unhandled exception: System.IO.IOException: Connection timed out : 'Global\msbuild-server-launch-NDSZDEInz8s3G_evUwEeKheECB+4j1X89xWLxyQAqoU'
           at System.Threading.Mutex.CreateMutexCore(Boolean initiallyOwned, String name, Boolean& createdNew)
           at Microsoft.Build.Experimental.MSBuildClient.TryLaunchServer()

I noted yesterday on an email thread that we'd seen problems that led to finding out that "MSBuild Server by default" was a new behavior on Linux, but I'd ping someone like @rokonec for subject-matter-expertise as I only know about this in the context of investigating build failures.

If it happens to be related to the server running where it didn't previously and you can get a local repro, setting DOTNET_CLI_DO_NOT_USE_MSBUILD_SERVER = true and seeing it stop reproducing will confirm it. It's possible this is a red herring, of course.

@arunchndr arunchndr removed the untriaged Issues and PRs which have not yet been triaged by a lead label Sep 14, 2022
@akoeplinger
Copy link
Member

akoeplinger commented Nov 25, 2022 •
edited
Loading

We actually have a quite long running tracking issue in runtime here: dotnet/runtime#53420

According to runfo it fails about twice a week. Given how old the issue is and afaik we disabled the msbuild server in arcade recently I don't think it is related to that.

@arunchndr arunchndr added this to the Backlog milestone Nov 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Area-Infrastructure
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
10 participants
@radical @lewing @steveisok @carlossanlop @akoeplinger @MattGal @eerhardt @tarekgh @arunchndr and others