How hard would it be to creating a custom parser based on an existing shell language?

[hwittenborn]

I was looking into making some sort-of custom syntax for a project I'm working on:

It would be sort of like the Bats @test syntax, but would ommit the @test part and change the string part after it to have no quotes like so:

package {
    # Anything in here is normal shell syntax again, until the closing `}` is found.
    a_function() { true; }
    echo "Run commands, and do any normal shell stuff in here"
}

It's also like just defining a package() { ... } function, I'm just wanting to omit the () for stylistic purposes in my program.

Does the public API expose anything to do the things I'm looking for? If not, would there be anywhere in the codebase you'd recommend I'd look at if a fork would be necessary?

[mvdan]

There is no easy API for you to do this, no - you would have to modify the recursive descent parser to allow for it. Looking at how the Bats syntax is implemented would be a good start. I don't think we want to have API for supporting arbitrary syntax like this, because I'm not even sure what the API would look like. A soft fork seems like a good way to experiment.

[mvdan]

Moved to a discussion btw, since this seems more like a question than an actionable feature request.

[hwittenborn]

Yeah, definitely more of a question. The stuff I'm gonna be making the changes for is gonna be project-specific so I wouldn't expect it to land into this repository itself anytime soon.

[hwittenborn]

Cool, I'll go ahead and start experimenting with a new repository. I'm not sure what a public API to modify the parser would look like either, I feel like there'd be a lot of ways about doing it and it's not something I could see happening too easily for sure.

A fork is more than fine on my end, I was just looking for some starting points and available options honestly. I'll go ahead and give that a spin, thanks a ton! :)

[hwittenborn]

You can do whatever you want with this discussion, I was gonna leave it open until I figured out what worked well, but I was just gonna send a message or two if I had any questions diving into the internals of the parser. Whatever works best for you is fine with me.

[hwittenborn]

I'm just leaving this here for commentary, but I think this is going to be the approach I'm going to use in my program:

Without going too much into the specifics, I'm going to be restricting the stuff that can happen at a global scope in a file (the file format is going to be for something similar to PKGBUILDs), so I'm probably going to do some quick string-based pre-processing and then just convert things like package { to package() {, and then pass that result into this library. I'm going to have to do some AST parsing for some static analysis anyway, so I'll be able to detect all those use cases just fine.

I think I'm at a pretty confident place on what I need to do, but I'll send a message if anything comes up that I need help with. Thanks a ton for everything you've done so far!