Process is terminated due to StackOverflowException.

[JaCraig]

So it looks like the newest version of ccdocgen is throwing a stackoverflowexception. No stacktrace, just the message in the title. Happens on any project that I've built with any sort of complexity that has the "Emit contracts into XML doc files" checked. Killed many a build today.

Also, I figured that I could spend the night putting in a fix. Did a fork, pulled down, opened up Visual Studio... And I got stuck. I have no idea where that app lives in the code base. I found ccdoc10 but I can't figure out which solution that is housed in. Nor when opening up the project file can I find DataStructures10 or ContractMSInternal. Also it's late and I'm getting tired... If you can point me in the right direction, I'll take a crack at it but otherwise I guess I'll just say glhf (good luck, have fun).

[sharwell]

Hi @JaCraig,

Thanks for taking the time to report this, and even to investigate it! We are all living with confusing code for now I think. Like many of the executables in Code Contracts, ccdocgen.exe is created by an ILMerge pass that runs separately from the normal build you see in Visual Studio - hence why you weren't able to find it. The primary ILMerge operations are defined here, and ccdocgen is created from ccdoc.exe plus these libraries.

Unfortunately the source code for all of this is not in this repository. All I see is a big group of binaries.

[ContentsMayVary]

Just to confirm, I am also seeing this issue.

For the time being I'm going to do a global change of
"True"
to
"False"
in all our ".csproj" files.

(I added the so I can globally change them all back once this is fixed.)

[JaCraig]

@sharwell I say this with all love and affection as someone who genuinely likes many of the things that you've worked on over the years (ANTLR, Code Contracts, DartVS, etc): But damn, you crazy.

That is one heck of a build process and debugging must be interesting... Are they binaries for a reason? For instance, if I were to say disassemble those DLLs, set up a project tree, etc. for those things would that be bad? Or are they just binaries because the code has been lost to time and the merge is really the only option without some work?

[mike-barnett]

All of those assemblies are from CCI2: https://cciast.codeplex.com/. They should be replaced by a NuGet dependence, but I don't believe there is a NuGet package for them yet. We created the tools with ILMerge just because it made deployment easier (or at least, we thought it did --- it also reduces startup time because .NET seems to perform better with fewer assemblies). It would certainly be possible to change this decision and stop merging the assemblies to create the different tools.

[sharwell]

@JaCraig 😆 I had basically nothing to do with the build process in Code Contracts. In all fairness, my educated guess as to why it's so crazy is the project was started in the very early days of .NET and had access to internal projects (e.g. Spec#/Singularity/Midori) that never panned out. For anything where you think "that would be a bad idea", the item was probably in place in this code a few years before anyone actually decided the idea was bad.

[JaCraig]

@mike-barnett No, you guys can keep doing you. If the code for the assemblies is available then that means debugging isn't that bad. I just saw his comment of "All I see is a big group of binaries" and I audibly gasped. I've worked with research projects in the past where certain items were binaries because the person who started the project kicked it to a couple of grad students who didn't understand source control... So instantly my mind went to the scene from planet of the apes. One of you comes to a beach where you find the shattered remains of a CD with the original code. You drop to your knees and shout angrily at the sky "You Maniacs! You blew it up! Ah, damn you! God damn you all to hell!" Good to know that isn't the case.

So I guess I'm also downloading CCI2 tonight. Well actually I'll try debugging without it first but good to know what those items are before I make the attempt.

[tom-englert]

Just for completeness: My builds in 2015 fail, too, but with this message:
C:\Program Files (x86)\Microsoft\Contracts\MsBuild\v14.0\Microsoft.CodeContracts.targets(642,5): error MSB3073: The command ""C:\Program Files (x86)\Microsoft\Contracts\Bin\ccdocgen.exe" "@obj\Debug\AVL.Coreccdocgen.rsp"" exited with code -1073741571.
(indeed this is the stack overflow error code)
The same projects built with 2013 succeed.

[tom-englert]

@JaCraig A simple way to debug this without the need to have CCI2 or fiddle around with the build is to use JetBrains DotPeek and just generate PDB and source code directly from ccdocgen.exe.
The bug seems to be in CCI2:

namespace Microsoft.Cci.ILToCodeModel
{
  internal class ClosureFinder : CodeTraverser
  {
    ...

    private void AddOuterClosures(ITypeDefinition typeDefinition)
    {
      foreach (IFieldReference fieldReference in typeDefinition.Fields)
      {
        ITypeDefinition resolvedType = fieldReference.Type.ResolvedType;
        if (TypeHelper.IsCompilerGenerated(resolvedType) && !TypeHelper.IsEmbeddedInteropType(resolvedType))
        {
          this.closures.Add(resolvedType.InternedKey, (object) resolvedType);
          // => here the recursion happens:
          this.AddOuterClosures(resolvedType);
        }
      }
    }
  }
}

[tom-englert]

@mike-barnett this bug seems to be old - just that with assemblies generated by VS2013 this method get's never invoked.

[mike-barnett]

Hmm, it probably has to do with the way that Roslyn has changed their code generation pattern for static closures (those that don't capture anything). Anyone have a repro that I can use to track it down?

[tom-englert]

@mike-barnett https://tomstoolbox.codeplex.com/SourceControl/latest
Just download and rebuild the Core module with VS2015

[tom-englert]

@mike-barnett if you think about making CCI2 an official package, there might be other requirements to consider. E.g. there is already a shadow copy for CCI around (https://www.nuget.org/packages/SymbolSource.Microsoft.Cci.Metadata/) because too few interfaces are public, making basic tasks like getting the list of source file names impossible (see e.g. NuGet/Home#304 or NuGet/Home#529). Relaxing the access of some classes would highly increase the usability of this library.

[krk]

I got a StackOverflowException from VS2015 itself in the same class, ClosureFinder.

https://connect.microsoft.com/VisualStudio/feedback/details/1634045

[SergeyTeplyakov]

Can someone please provide a repo example?

[tom-englert]

Reproducible with:
https://tomstoolbox.codeplex.com/SourceControl/latest
Just download and rebuild the Core module with VS2015

[SergeyTeplyakov]

@tom-englert Thanks, will try.

I just thought, that maybe someone already analyzed the issue and got small repo that could be easily used in unit tests for checking this issue.

[tom-englert]

@SergeyTeplyakov you would need to store a binary and run ccdocgen.exe against this.
Are there already unit tests covering ccdocgen?

[Dennis-Petrov]

The same problem. Emitting contracts into XML doc works using VS 2013, and fails with StackoverflowException using VS 2015. Turning off "Emit contracts..." is a workaround.

[tom-englert]

@mike-barnett this bug is a real show stopper, but I don't see any progress.
Are there any plans to fix CCI2? Or are we supposed to clone the CCI2 sources an do our own fix?

[tom-englert]

@SergeyTeplyakov do you know anything new about this?

[mike-barnett]

Sorry! I will try to get to this on the weekend.

[kkm000]

@SergeyTeplyakov, @mike-barnett, @tom-englert So is it possible to fix this bug in any way outside of decompiling the binary?

I am working around the bug by checking in generated doc files in hope that not a lot of discrepancy would accumulate before the bug is fixed, but that becomes tedious to maintain 😢

[tom-englert]

@mike-barnett any news? Can we help in some way?

[tom-englert]

@mike-barnett I have created a patch for https://cciast.codeplex.com (17904) that should fix this issue.

There is another patch for https://ccimetadata.codeplex.com (17902) that adds a simple function to pdb reader to enhace its usage scope. It would be great it that could be published, too.

Just came across this: https://www.nuget.org/packages/Microsoft.Cci - will it replace the above?

[Worthaboutapig]

Any chance of getting this fixed? It's a major head-ache and seems a pretty serious bug to me.Given the CCIAST has been reported, with a fix since Jun 11, 2013, I guess that's never going to be fixed.

[mike-barnett]

Okay, sorry about the long delay. I have checked in the fixed sources to CCI (including the two patches 17904 and 17902). But I have not built new CCI2 binaries and checked them in to the Code Contracts repo, so if you can take care of that, then that would be great.

I haven't done anything yet about distributing CCI through NuGet, but that would be great. I'm still waiting to hear about getting both CCI and Code Contracts moved to the .NET Foundation organization. Hope to have news about that soon.

[Worthaboutapig]

@mike-barnett Thanks! @tom-englert Are you able to sort this out?

[tom-englert]

I'll take a look at this

[ghost]

@hubuk I also tested your msi above and had the same problem as @psmacchia: There is no Code Contracts tab in the Project Properties in VS2015 (started the msi from within an administrative command prompt).

[hubuk]

I am looking into it.

[hubuk]

@taschna @psmacchia CodeToolsUpdate project required some changes. I have submit a PR #281.
I will create an installer for you tomorrow.

[hubuk]

@psmacchia @taschna I was able to build a new installer. However there is a one more problem which need to be addressed in addition to the previous one. I will be working on a fix and in a meantime you can check a new installer. It is available here:
https://ci.appveyor.com/project/hubuk/codecontracts/build/1.9.11028.16/artifacts
One additional thing you need to do is to create a CodeContractsAfter.targets file in C:\Program Files (x86)\MSBuild\14.0\Microsoft.Common.Targets\ImportAfter for VS2015 with the following content:

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <!-- Begin CodeTools: CodeContracts: After -->
  <PropertyGroup>
    <CodeContractsInstallDir Condition="'$(CodeContractsInstallDir)'==''">C:\Program Files (x86)\Microsoft\Contracts\</CodeContractsInstallDir>
  </PropertyGroup>
  <Import Condition="'$(CodeContractsImported)' != 'true' AND '$(DontImportCodeContracts)' != 'true'" Project="$(CodeContractsInstallDir)MsBuild\v14.0\Microsoft.CodeContracts.targets" />

  <!-- End CodeTools: CodeContracts: After -->
</Project>

For some reason installer is not creating this file out of box.

[sharwell]

@hubuk Have you created an issue for that installer bug?

[hubuk]

@sharwell Unfortunately no. I do not know if it is still required. I have already submit a PR #282 for the issue.

[ghost]

@hubuk: The new installer worked exactly as promised: the project tab is visible and the XML file is emmitted. Thank you for the work!

[hubuk]

@taschna I was only fixing my own defects:) @tom-englert provided the fix in title.

[ndepend]

Ok, I finally found some time to test and all bugs I reported seem fixed, thanks. I am using this version https://ci.appveyor.com/project/hubuk/codecontracts/build/1.9.11111.33/artifacts

However I found a new bug, not a blocker one as previous ones, yet it prevents a pro usage.

We have a lot of require not null contracts and I noticed that the contract formatting in the XML doc that used to be csharp="XYZ != null" is now csharp="!(XYZ <= null)". If I look in the xml generated I get:

// XML Generated Now     csharp="!(initialSeq <= null)"
<requires description="initialSeq must not be null" csharp="!(initialSeq &lt;= null)" vb="!(initialSeq &lt;= Nothing)">!(initialSeq &lt;= null)</requires>

// XML Generated Before  csharp="initialSeq != null"
<requires description="initialSeq must not be null" csharp="initialSeq != null" vb="initialSeq &lt;&gt; Nothing">initialSeq != null</requires>

Here is the contract declaration in the C# code:

// Contract declaration in the C# code
Contract.Requires(initialSeq != null, "initialSeq must not be null");

[yaakov-h]

At a guess, that's due to the way Roslyn now emits != null checks.

What's the IL for your contract declaration?

[ndepend]

Please investigate.

Sorry I cannot take the time to install CC 1.9 / deinstall CC 1.9 / reinstall CC 1.7 to work, it takes some time to apply this very tedious procedure http://stackoverflow.com/a/27914825/27194

Also why Roslyn would be the cause? It is the job of CC to weave the contract IL, isn't it? Does it uses Roslyn for that?

Here is the IL when compiled with VS2013 + Code Contract 1.7 if it can help:

L_0000: ldarg.0 
L_0001: ldnull 
L_0002: ceq 
L_0004: ldc.i4.0 
L_0005: ceq 
L_0007: ldstr "initialSeq must not be null"
L_000c: ldstr "initialSeq != null"
L_0011: call void System.Diagnostics.Contracts.__ContractsRuntime::Requires(bool, string, string)
L_0016: nop 
L_0017: ldarg.1 
L_0018: ldnull 
L_0019: ceq 
L_001b: ldc.i4.0 
L_001c: ceq

[yaakov-h]

Yep. Here's the IL from VS2015 (Update 1 RC) after being rewritten with 1.9.1:

    IL_0000: ldarg.0
    IL_0001: ldnull
    IL_0002: cgt.un
    IL_0004: ldstr "initialSeq must not be null"
    IL_0009: ldstr "initialSeq != null"
    IL_000e: call void System.Diagnostics.Contracts.__ContractsRuntime::Requires(bool, string, string)
    IL_0013: nop
    IL_0014: nop
    IL_0015: br IL_001a

[ndepend]

Thanks Yaakov, did you fix anything? is there a build I can try from https://ci.appveyor.com/project/hubuk/codecontracts ?

Your IL looks fine isn't it? "initialSeq != null"

Or is the bug still unfixed and you get: !(initialSeq <= null)" in the XML doc rewritten by CC?

[yaakov-h]

No, I don't have much expertise in actually modifying Code Contracts. I can edit the system contracts and that's about it.

But this is likely cause by Roslyn - Roslyn compiler emits x != null as x > null, whereas the old <= VS2013 compiler emitted it as (x == null) == 0 as you can see from your IL above.

ccdocgen likely needs to be updated to understand that x > null also means x != null, but I'd have no idea how to go about doing that.

[ndepend]

Thanks yaakov-h, I opened a new issue, hopefully someone able to modify CC will handle it, thank

#307

[mike-barnett]

The best thing (I think, not having looked at the code…) would be to fix the CCI2 decompiler to understand this idiom and produce the x != null expression from the IL. Then all downstream tools, such as CCDoc, will automatically do the right thing.

Mike

From: psmacchia [mailto:[email protected]]
Sent: Wednesday, November 18, 2015 9:14 PM
To: Microsoft/CodeContracts [email protected]
Cc: Mike Barnett [email protected]
Subject: Re: [CodeContracts] Process is terminated due to StackOverflowException. (#169)

Thanks yaakov-h, I opened a new issue, hopefully someone able to modify CC will handle it, thank

#307#307

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/169#issuecomment-157952976.

[yaakov-h]

I had a brief look at the code and don't understand it fully, but I also think CCI is where the fix ought to go.

[CreepyGnome]

Is there any status on this stack overflow issue and when it is expected to be addressed?

It just started happening out of the blue in a project and I undid all the changes I made before it occurred but it seems once it occurs it doesn't go away.

I tried to read through this open issue and maybe I missed it but I don't see any workarounds either, are we screwed once this occurs and we have to disable code contracts to move forward? I mean if we cannot compile a project it kind of screws everything else that is dependent on that project.

[hubuk]

@CreepyGnome StackOverflowException should be already fixed, but fixed version has not been relrease yet. You can clone repository and build an installer from master branch.
You can also use an installer produced by one of my latest builds. Unfortunately due to some problems with versioning scheme used by the project it has lower version number than the released one. As a result Visual Studio will keep telling you that it is outdated.

[CreepyGnome]

Code Contracts seem fragile enough right now as it is (crashes at least once every hour or so but seems to recover) that I don't think I should be experimenting with unofficial releases. I like them, but right now I have a whole solution that have them disabled because of this stack overflow issue. Everything was fine and then it just started out of the blue and there appears to be no workaround once it starts. This seems like a fatal/critical bug and should prompted a release just for it as it is completely blocking.

I am aware of the version issue and have put in place the work around I posted on that issue. So it does not tell me I am out of date, and I hope in the next official release that fix will be included and should just overwrite my workaround.

[kkm000]

We rolled out build 1.9.11111.33 from appveyor, and the SO in CCDoc is certainly fixed, thank you @hubuk!

[caractacus]

It's six months since the last release of Code Contracts - and we are STILL unable to work with Visual Studio 2015, build the git repo, or otherwise make any progress.

I am writing off CodeContracts as there is obviously no official support for this track.

[Update: I am testing the RC... ]

[hubuk]

@caractacus Is there anything wrong with current release candidate? https://github.com/Microsoft/CodeContracts/releases/tag/v1.10.10126.2

[caractacus]

Hi @hubuk, the release candidate works in VS2015, and the builds are completing successfully. Thank you, now I'll eat my hat.

However, Visual Studio continues to report v1.8.xxx installed, with an updated to v1.9.xxx available.

[hubuk]

@caractacus I am seeing the same issue. I will investigate and submit a separate issue for that.

[ContentsMayVary]

@hubuk I think the issue about the wrong version number being shown has already been reported as issue #377

[LoganDavidTalbot]

@anyone is there a fix for this issue now?

[tom-englert]

The original issue (stack overflow) was fixed with PR #276.
@SergeyTeplyakov I think this issue can be closed.