Enhancement of ViewMetadata

[nk1506]

Feature Request / Improvement

Motivation: #8907 (comment)

Currently there is no relation between TableMetadata and ViewMetadata. Which causes many duplicate codes just to use commonTable/View definitions like Schema/MetadataLocation/Properties.

At many places we have similar methods with overloaded parameters like TableMetadata and ViewMetadata.

To support purge on view it is difficult to traverse to the historical metadata files.

I would like to start a discussion on potential enhancement of ViewMetadata.

1. An abstract class such as IcebergMetadata with common definition from TableMetadata and ViewMetadata.
2. Add something similar of MetadataLogEntry for ViewMetadata too.

Query engine

None

[nastra]

@nk1506 which particular methods did you have in mind that would make sense to extract into IcebergMetadata? I'm not sure this is something we'd want to do.
ViewMetadata is an interface where the impl is generated by Immutables, whereas TableMetadata is a normal class.
Even if we'd have IcebergMetadata defined as an interface with a few common methods, would this make certain things easier?

[nk1506]

@nk1506 which particular methods did you have in mind that would make sense to extract into IcebergMetadata? I'm not sure this is something we'd want to do. ViewMetadata is an interface where the impl is generated by Immutables, whereas TableMetadata is a normal class. Even if we'd have IcebergMetadata defined as an interface with a few common methods, would this make certain things easier?

To simplify TableOperations and ViewOperations I think extracting at least following methods will help:

• List<Schema> schemas();
• Map<String, String> properties();
• String metadataFileLocation();
• String location();
• If we are going to keep previous metadata files for view then MetadataLogEntry

I did a small poc with Hive catalog but mostly with generic class. It makes code more cleaner.

[nk1506]

cc: @pvary

[pvary]

@nk1506 which particular methods did you have in mind that would make sense to extract into IcebergMetadata?

@nastra: When managing tables and views in the catalogs we often have to handle them similarly. Especially when we drop, alter a table but even when creating them we need to manage the metadata similarly. Also, IMHO it is important to have a lock around the commit operations for the views as well, and it is a quite complicated logic which is easier to abstract out when the view and table metadata has a common interface.

Also this commonality is already highlighted by having the same names for the methods. It is just not formalized.

[nk1506]

@szehon-ho , regarding comment . that discussion is already happening here. But we haven't concluded yet.

[szehon-ho]

Hi @nk1506 , I think we can have a lazy consensus, if nobody objects then we can go. Did you have any concern about adding the common interface ?

[github-actions]

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.