Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spark: Handle concurrently dropped view during CREATE OR REPLACE #9623

Merged

Conversation

nastra
Copy link
Contributor

@nastra nastra commented Feb 2, 2024

This is related to #9596 (comment). If the view is concurrently dropped during CREATE OR REPLACE we create the view in CreateV2ViewExec

Copy link
Contributor

@amogh-jahagirdar amogh-jahagirdar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just had a comment on another possible concurrent case, but I don't consider it a blocker since it's not practical for views, and in the end the query would fail and a user could retry (so there's no bad outcome of the concurrent case).

Comment on lines 75 to 77
// view might have been concurrently dropped during replace
case _: NoSuchViewException =>
createView(currentCatalog, currentNamespace, newProperties)
Copy link
Contributor

@amogh-jahagirdar amogh-jahagirdar Feb 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it really matters in practice because it seems unlikely to happen but if we want to be considerate of a concurrent drop during the create or replace, wouldn't we also want to be considerate if someone also concurrently creates the view right after dropping it?

So the timing would look something like:

1.) Thread 1 starts a CREATE OR REPLACE
2.) Thread 2 performs DROP
3.) Thread 2 has executed a createView
4.) Thread 1 catches the NoSuchViewException and calls createView
5.) Thread 1 fails because it was a normal create and the view already exists.

It'd be equally odd if CREATE OR REPLACE fails due to "ViewAlreadyExists" exception.
In that case I think this would need to be a re-execution of the entire createOrReplace instead of just a create.

Again, I don't think it's a really practical case but it seems feasible to handle if we want to handle the "NoSuchViewException" case as well

Copy link
Contributor Author

@nastra nastra Feb 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case I think this would need to be a re-execution of the entire createOrReplace instead of just a create.

I agree, that's probably the best to just issue the replace again here

@nastra nastra force-pushed the spark-view-handle-concurrent-drop-when-replacing branch from fc2fa8b to 9e3a25f Compare February 6, 2024 12:13
@nastra
Copy link
Contributor Author

nastra commented Feb 7, 2024

Thanks for reviewing @amogh-jahagirdar

@nastra nastra merged commit 2a39af8 into apache:main Feb 7, 2024
31 checks passed
@nastra nastra deleted the spark-view-handle-concurrent-drop-when-replacing branch February 7, 2024 07:32
zachdisc pushed a commit to zachdisc/iceberg that referenced this pull request Dec 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants