Move Reflections to Rest API Calls and Extend Implementation

[bcmeireles]

Summary

Reflections were being created through the SQL Runner and that was the cause for some issues we were facing.

Description

The current reflections implementation requires sys.reflections access, which is only available to superusers (admins). When creating the reflection with a regular user, it will error. To fix this issue, the creation of the reflections has been moved from queries being ran inside the SQL Runner to Dremio's Rest API calls.
The current logic of writing reflections through a custom materialization remains to make sure reflections created under the previous approach still work.

On top of this, the behaviour where all reflections would be dropped and then recreated when running a dbt project has been changed to instead first check if a reflection with the same name is already present in the same dataset and, if that's the case, it will update the already existing reflection instead.

Test Results

A total of 11 tests have been added, approaching different situations that may occur when dealing with reflections

Changelog

• Reflections are now handled through the Rest API, allowing non-admin users to also create them
• It is now possible to set a custom name for reflections
• If a reflection already exists in the dataset with the same name defined in the model, it will be updated instead of creating a new one
• New date_dimensions parameter was added to the reflection materialization, to set fields that have a DATE granularity
• Added distribution fields under distribute_by
• Added partition transformations under partition_transform
  • Defaults to Original/Identity if not defined
  • year/month/day/hour/bucket(n), truncate(n)
• Computations default to SUM, COUNT if mapped measure is numeric, COUNT if not
• reflections_enabled adapter option has been renamed to reflections_metadata_enabled (requires user privileges to run in dremio)
• CI now creates and formats a Samples source as it is needed for the reflection tests

[howareyouman]

Great PR!
I have some not huge blocking changes requests.

The main part is documentation. We should update changelog, we should create a ticket for docs team for sure about this feature in our dbt connector.

I remember that we were discussing the waiting part of reflection to be created. Will we provide it in the next PR? Should we create a ticket for it?

[howareyouman]

@bcmeireles we should transfer the PR description somewhere in the docs and changelog as well!

[bcmeireles]

Great PR! I have some not huge blocking changes requests.

The main part is documentation. We should update changelog, we should create a ticket for docs team for sure about this feature in our dbt connector.

I remember that we were discussing the waiting part of reflection to be created. Will we provide it in the next PR? Should we create a ticket for it?

@bcmeireles we should transfer the PR description somewhere in the docs and changelog as well!

I'll be copying the PR description into the Changelog. Once merged I will also be updating this repo's wiki with the new functionalities and changes.
Regarding the waiting for a reflection, that has been requested in #184 and is not addressed in this PR.

[howareyouman]

Could you please add somewhere links about REST API of reflections in Dremio?
The rest is great, thank you!

[bcmeireles]

A link to the Reflections API docs has also been added in the entity