-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Spark: Handle concurrently dropped view during CREATE OR REPLACE #9623
Spark: Handle concurrently dropped view during CREATE OR REPLACE #9623
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just had a comment on another possible concurrent case, but I don't consider it a blocker since it's not practical for views, and in the end the query would fail and a user could retry (so there's no bad outcome of the concurrent case).
// view might have been concurrently dropped during replace | ||
case _: NoSuchViewException => | ||
createView(currentCatalog, currentNamespace, newProperties) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think it really matters in practice because it seems unlikely to happen but if we want to be considerate of a concurrent drop during the create or replace, wouldn't we also want to be considerate if someone also concurrently creates the view right after dropping it?
So the timing would look something like:
1.) Thread 1 starts a CREATE OR REPLACE
2.) Thread 2 performs DROP
3.) Thread 2 has executed a createView
4.) Thread 1 catches the NoSuchViewException and calls createView
5.) Thread 1 fails because it was a normal create and the view already exists.
It'd be equally odd if CREATE OR REPLACE fails due to "ViewAlreadyExists" exception.
In that case I think this would need to be a re-execution of the entire createOrReplace instead of just a create.
Again, I don't think it's a really practical case but it seems feasible to handle if we want to handle the "NoSuchViewException" case as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In that case I think this would need to be a re-execution of the entire createOrReplace instead of just a create.
I agree, that's probably the best to just issue the replace again here
fc2fa8b
to
9e3a25f
Compare
Thanks for reviewing @amogh-jahagirdar |
This is related to #9596 (comment). If the view is concurrently dropped during
CREATE OR REPLACE
we create the view inCreateV2ViewExec