Language configuration (comments) not adjusting properly

[christosnc]

Hello,

In monaco-editor, each language has a configuration (type of comments, foldind, etc.). So if you are in html mode, the "toggle comment" key wraps the code in <!-- -->, if you are in css, the comments become /* */, etc.

This behavior stops working correctly if we have languages with embedded modes (ex. html with <style> or <script>). In that case, all the comments become <!-- -->, no matter if you are in one of the tags mentioned.

Is there a call you can make on the fly (during tokenization) to change the configuration, based on the language type of the current line that we are tokenizing?

[christosnc]

I tried several things to resolve the issue, with no success. I will list some ideas:

1. loadGrammarWithEmbeddedLanguages instead of loadGrammar could help maybe?
2. Implement the tokenize2 - tokenizeEncoded instead of tokenize? (tokenize2 provides metadata such as languageId per token)

The only workaround I found, is to use the monarch tokenizer for HTML, as it provides the nextEmbedded and works perfectly with the CSS/JS TM Grammars, but unfortunately, this is not a great solution for obvious reasons..

[zikaari]

Can you please setup a minimal project for us to look at (and use as a base reference for patches that follow)?

Could be as little as an index.html and main.js file if you want. (TypeScript source files preferred but not required) bundled with webpack that replicates your issue.

Please create a repo and link it here.

(If it's done well, we can look into linking it into readme of this project as an example for other folks to look at)

[DylanPiercey]

@NeekSandhu I think you can reproduce this issue just by loading up an html tmlLanguage like https://github.com/Microsoft/vscode/blob/master/extensions/html/syntaxes/html.tmLanguage.json alone side the css and javascript embeds (https://github.com/microsoft/vscode/blob/master/extensions/javascript/syntaxes/JavaScript.tmLanguage.json & https://github.com/microsoft/vscode/blob/master/extensions/css/syntaxes/css.tmLanguage.json). Then simply write something like:

<div>
  Hello
</div>

<script>
  console.log("Hello")
</script>

<style>
  .hello { color: green }
</style>

Then try commenting out (cmd + /) the content inside each tag. It will always use <!-- , eg:

<div>
  <!-- Hello -->
</div>

<script>
  <!-- console.log("Hello") -->
</script>

<style>
  <!-- .hello { color: green } -->
</style>