[RFC] Full extensibility

[amis92]

Summary

This project is quite useful. Its extensibility is highly restricted, however. As an enhancement, I propose to add more extension points allowing consumers to make use of the core functionality of this project on their own.

Analysis

The following discusses this project's current state:

Core value

This project's core value is plugging into an MSBuild build pipeline a task that generates code based on the already existing code in the project and adds this generated code to the core compilation.

Current restrictions

• By design, an attribute on a syntax node in source code is the trigger.
• The action is an invocation of the generator (external plugin) which is supposed to return a part of a generated code.
• The result, for a single source document, is another (single) source document.

As it stands it's already a couple restrictions.

Promises

• This project should handle everything that's needed to compile the project with Roslyn and pass the result to the generators.
• The project should handle writing the files to disc.
• The project should include written files in core compilation.
• The project should optimize away generation of unmodified files (?)

---

Proposal

Taking into account all of the above, I'd suggest the following extensibility model.

Layers

Given command line arguments, the tool builds up the execution pipeline:

1. First layer, the CLI tool, is handling the inputs and outputs, which involves:

• loading referenced plugins (custom engines and generators)
• gathering Roslyn arguments (defines, references, source files)
• creating Roslyn Compilation instance
• controlling generated files

2. Second layer is running the engines in the given EngineContext, which contains:

• Compilation instance

• additional parameters (eg command line parameters, project directory)

• generated file list

• generated file manager (that provides ability to save generated files)

  Given Engine defines EngineContext transformations/actions that are run on a context. Resulting context is passed to the next Engine.

3. Third layer is for the Generators. In the current state, the CodeGeneration.Roslyn.Engine would run generators implementing the ICodeGenerator/IRichCodeGenerator (as it does now).

Design

What this essentially means is that we separate out responsibilities from CodeGeneration.Roslyn.Engine which is a workhorse:

1. Generated file management
2. Compilation creation
3. Compilation processing (resulting in new files)

Most importantly we open up ourselves for other ways of Compilation processing than Attribute-based.

I'd imagine that our current DocumentTransform and CompilationGenerator would result in an AttributeBasedEngine which would invoke IAttributeBasedCodeGenerator implementations (it would in turn be a replacement for both ICodeGenerator and IRichCodeGenerator).

Also, the new AttributeBasedEngine could be designed so that deriving from it is easy, thus allowing for easier extensibility.

Summary

I'd like to make CodeGeneration.Roslyn a universal extensibility point for Roslyn-based Code Generation, providing a default Attribute-based Engine that runs currently implemented ICodeGenerators and allows for a much greater flexibility in terms of triggering generation and storing its results.

[hypervtechnics]

Is there a fork/wip branch regarding this proposal? Or anything planned?

[amis92]

No, only this design draft.

[hypervtechnics]

How much effort would be needed to rework the existing library into the draft? The draft looks quite promising. What I'd like to had is some less error-prune way of adding it to projects, an example and providing code generators as a NuGet package.

[amis92]

Estimate: hard to say. It's probably a rewrite of most of the content.

What you ask for, though, is already here:

• less error-prone way of adding it to projects: README provides a guide. Other problems: raise an issue. Maybe with the SDK (Create SDK package #113) we can make the experience better.
• an example: see samples folder. Want more? Raise an issue for a specific example.
• Providing code generators via nuget: already possible, instructions are in README. You can also see an example generator (distributed via NuGet) at https://github.com/amis92/RecordGenerator

[hypervtechnics]

I'd support the rewrite approach and would also participate in the coding part. I am just not very sure to what extent this is really planned and meant to be done/implemented as this issue is from May.

By less error-prone way I mean some kind of Quickstart (okay I found the sample now :P) to setup a project using this library. And also check in the library for the required components (or through an extension) and throw errors designed for this. But this issue seems like already being addressed in the issue mentioned by you.

Also adding the dotnet-codegen tooling automatically on installation of a NuGet package would be such a thing that would increase usability.

[amis92]

Also adding the dotnet-codegen tooling automatically on installation of a NuGet package would be such a thing that would increase usability.

Unfortunately, no matter how we approach things, there will always have to be at least a single manual step/addition.

• currently: add DotNetCliToolReference to project file
• with (future) SDK: the Sdk can add the reference to the dotnet-codegen, but referencing the Sdk is manual
• with (future) dotnet-codegen as a Global/Local Tool - adding the tool to the tool manifest (dotnet-tools.json)

Nothing from the above can be done when installing just the NuGet package. However, there may be no need to install any package at all when using the Sdk.

[hypervtechnics]

Those were only examples of making the usage of the library in a new/existing project as easy as possible. So what do you think? Let's do the rewrite/enhancing of the library?

[amis92]

Sure. I can spare some time. How do you propose to work on this? Maybe on my fork?

[hypervtechnics]

First I'd make i priotized roadmap and then create a suitable architecture. Your fork is good.

[amis92]

Okay. I think this issue is a good place to start designing the roadmap. Please don't hesitate to fundamentally disagree with me, I'm not really sure I'm doing the right thing now. :)

1. Change assembly loading to use McMaster plugins (Use McMaster.NETCore.Plugins for assembly loading #156)
2. Provide a consumer Sdk (Create SDK package #113) - for using generators in projects
3. Provide a plugin Sdk (Create SDK package #113) - for creating generator projects
4. Rewrite the framework for maximum extensibility (this issue)

So essentially, I'd say that the 1-3 are pre-requisites for this issue to proceed. How do you see it?

[hypervtechnics]

I'd go the other way and first design like it would be nice to integrate/use the framework and design some sketches on how different kind of code generators can be defined. Then we can evaluate how much of the desired stuff in the sketch is really necessary/doable.

As we are rewriting huge parts we could also do some breaking changes.

The Sdk approach mentioned by you is a very good start. This would the possible error of mismatching versions.

[hypervtechnics]

@amis92 Any plans? I'd suggest changing channel to a more realtime one to exchange some plans/information.

[amis92]

Yep. What channel? Gitter'd be okay?

[amis92]

Oh. gitter isn't available, something weird happened. https://gitter.im/CodeGeneration-Roslyn

[hypervtechnics]

I tried to put some points regarding the project here: https://github.com/hypervtechnics/CodeGeneration.Roslyn/projects/1

Any feedback? Or more ideas? Working on some drafts in the next day for the usage.

EDIT: Currently I am trying to understand the BuildTime package/project and the issue #113

[amis92]

Hmm. Yeah, we need a more chatty/realtime place to discuss this. Ideas?

High priority sounds ok. Not sure what "Trigger:: Implementation of interface" means.

[hypervtechnics]

Okay maybe this: https://matrix.to/#/+codegeneration.roslyn:matrix.org

This is actually not a trigger (but same function). This is discovery method to find code generators. Tbd of course ;D

[amis92]

Cool, I've registered and joined the community, but I can't join the room:

edit: username is amis92 (or @amis92:matrix.org)

[hypervtechnics]

Sorry, I am new to Matrix. Now I invited you to join the room.

[amis92]

Closed in favor of #162 (new draft proposal)