[BigQuery] Support for arbitrary REST API configurations

[akerone]

Describe the feature

The BigQuery REST API is very well documented, and new features are being added at a rapid pace. Especially important to this project would be the ExternalDataConfiguration section. In my case I absolutely need to be able to set the connectionId field to create a BigLake table, but it feels weird to ask for something that specific when we're configuring the desired table in YML that could be effortlessly converted to a JSON for the REST API.

Making the external section of the YML identical to the ExternalDataConfiguration of the REST API for BigQuery tables would immediately make this project way more usable and better documented for any BigQuery users. It would also mean virtually zero maintenance work in this project for BigQuery from this point forward.

Describe alternatives you've considered

As I mentioned, the alternative is to implement each and every new feature in SQL, but that's extra work for this project's contributors every time a new feature is added.

Additional context

This request affects BigQuery only.

Who will this benefit?

This would benefit all BigQuery users, and offload a lot of maintenance and documentation work from this project.

[github-actions]

This issue has been marked as Stale because it has been open for 180 days with no activity. If you would like the issue to remain open, please comment on the issue or else it will be closed in 7 days.

[github-actions]

Although we are closing this issue as stale, it's not gone forever. Issues can be reopened if there is renewed community interest. Just add a comment to notify the maintainers.

[thomas-vl]

@jeremyyeo I do like this suggestion but it is a backwards incompatible change. What are your thoughts?