Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add extension points for defining new syntax #3

Open
thomaswilburn opened this issue May 3, 2021 · 0 comments
Open

Add extension points for defining new syntax #3

thomaswilburn opened this issue May 3, 2021 · 0 comments

Comments

@thomaswilburn
Copy link
Collaborator

thomaswilburn commented May 3, 2021

  • Document the tokens that are currently parsed (https://github.com/nprapps/betty/blob/master/parser.js#L59), and add a way to pass in new token types (e.g., @key(filename.json), used to load content from an external file, would require us to add @, (, and ) as token types). The tokenizer will probably need to be a class instance and not a pure function for this to work.
  • Add the ability to pass in additional parsing function and type tuples (https://github.com/nprapps/betty/blob/master/parser.js#L59) that will be called in the context of the parser class and have access to its stack manipulation methods. (e.g., [loadFileRef, "AT", "TEXT", "LEFT_PAREN", "TEXT", "RIGHT_PAREN"]).

Generally speaking, we shouldn't need to extend the syntax much as that's what the field name and value hooks are for. But it would be nice to have the option to be able to define new syntax without completely forking the code, and I think it's written in a clean enough way that we can do that solely through the tokenizer and parser.

The example above is not a great use case, but it does point out a real deficiency in the existing hooks, which is that they're processed without any real context of each other. There's no good way to define a new value type that mutates the field name on use (as the @key(file) syntax would do--otherwise you could just write @key: filename, but then you'd have a key with a sobachka in it instead of the key directly stored). New syntax offers us a way around that, as it provides ways for us to process the text before it reaches the hooks (which are called in the assembler stage).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant