Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[da-vinci][dvc] Dropping unassigned partitions #1332

Draft
wants to merge 20 commits into
base: main
Choose a base branch
from

Conversation

kristyelee
Copy link
Contributor

Summary, imperative, start upper case, don't end with a period

The objective is to remove unassigned partitions.
Applying this issue to Da Vinci Client which has information over partitions in storage.

This PR is an extension of the PR #1196 [https://github.com//pull/1196] that checks which partitions should be kept [in StorageService] and applies the check in VeniceServer.

Resolves #650

How was this PR tested?

A corresponding test will be written.

Does this PR introduce any user-facing changes?

  • No. You can skip the rest of this section.
  • Yes. Make sure to explain your proposed changes and call out the behavior change.

@kristyelee kristyelee marked this pull request as draft November 20, 2024 23:33
@kristyelee kristyelee force-pushed the kristy_lee/650_davinci branch from 2055b04 to 52a557c Compare December 5, 2024 01:08
@kristyelee kristyelee force-pushed the kristy_lee/650_davinci branch from b94b49a to 262bbc6 Compare December 9, 2024 18:32
…iption. Enabled config flag over bootstrap subscription
…iption. Enabled config flag over bootstrap subscription
@kristyelee kristyelee force-pushed the kristy_lee/650_davinci branch from a163e56 to 9dbffc9 Compare December 10, 2024 23:45
@kristyelee kristyelee force-pushed the kristy_lee/650_davinci branch from 9dbffc9 to fd9fa02 Compare December 11, 2024 00:59
@@ -1283,6 +1283,94 @@ public void testBootstrap(DaVinciConfig daVinciConfig) throws Exception {
}
}

@Test(timeOut = TEST_TIMEOUT, dataProvider = "dv-client-config-provider", dataProviderClass = DataProviderUtils.class)
public void testBootstrapSubscription(DaVinciConfig clientConfig) throws Exception {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since DA_VINCI_SUBSCRIBE_ON_DISK_PARTITIONS_AUTOMATICALLY is already true by default, we should instead write a test where it's set to false.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Reconcile on disk state with assigned partitions for given resource and delete unused partitions
2 participants