-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Core: Track metadata file location in ViewMetadata #8608
Conversation
@@ -90,12 +90,20 @@ static void toJson(ViewMetadata metadata, JsonGenerator gen) throws IOException | |||
gen.writeEndObject(); | |||
} | |||
|
|||
public static ViewMetadata fromJson(String metadataLocation, String json) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
these new methods will be used later on by BaseViewOperations
// when associated with a metadata file, metadata must have no changes so that the metadata | ||
// matches exactly what is in the metadata file, which does not store changes. metadata | ||
// location with changes is inconsistent. | ||
Preconditions.checkArgument( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we don't want to fail in the check()
method but rather in build()
. Error messaging is aligned with TableMetadata
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree with placing this here, although I would note that there's no way to read changes using ViewMetadataParser
, so that should always produce metadata with a location but no changes. That means even if we did the check in ViewMetadata
, we wouldn't expect a problem.
@@ -176,6 +181,7 @@ private Builder(ViewMetadata base) { | |||
this.currentVersionId = base.currentVersionId(); | |||
this.location = base.location(); | |||
this.uuid = base.uuid(); | |||
this.metadataLocation = null; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just want to note that we're deviating from TableMetadata
behavior a little in this builder, since it relates to the metadataLocation
. In TableMetadata.Builder
, changes are accumulated across builder instances. Basically the changes get reset to the empty set when the new table metadata is written out to an object store and reloaded from that copy. In this builder, we accumulate a new set of changes in every builder and don't carry them through. The difference is that the table metadata needs to accumulate changes across multiple operations in a transaction, while but there are no transactions for view metadata because every changes affects only the view metadata object (not other data or metadata files).
No description provided.