Skip to content
This repository has been archived by the owner on Sep 18, 2023. It is now read-only.

Core Dump(Code:134) with WSCG SetDependencies function fail when running TPC-DS q84 #1156

Open
weiting-chen opened this issue Nov 2, 2022 · 0 comments
Labels
bug Something isn't working

Comments

@weiting-chen
Copy link
Collaborator

weiting-chen commented Nov 2, 2022

Describe the bug
Core Dump(Code:134) with a TypedWholeStageCodeGenImpl::WholeStageCodeGenResultIterator::SetDependencies function fail when running TPC-DS q84

To Reproduce
This issue can be reproduced in below steps:
Under SF500 testing in a single server,

  1. There is a four continue BHJ in q84 if set autoBroadcastJoinThreshold to "60M"
  2. We need to set spark.sql.autoBroadcastJoinThreshold to a certain value in my case is "1000k", let 4 BHJs become SMJ -> BHJ -> BHJ -> SMJ and trigger this issue.
  3. If set the value too small, all BHJ will be replaced to SMJ and no issue in this case(4 SMJs).

Expected behavior
If we don't support BHJ -> SMJ scenario, then we can set up a rule to eliminate this case.
20221102_SelfTest_DAG

Additional context
Add any other context about the problem here.

image

@weiting-chen weiting-chen added the bug Something isn't working label Nov 2, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant