Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Partial success results in a 0 exit code, denoting success #11114

Closed
2 tasks done
QMalcolm opened this issue Dec 9, 2024 · 0 comments · Fixed by #11115
Closed
2 tasks done

[Bug] Partial success results in a 0 exit code, denoting success #11114

QMalcolm opened this issue Dec 9, 2024 · 0 comments · Fixed by #11115
Assignees
Labels
bug Something isn't working microbatch Issues related to the microbatch incremental strategy

Comments

@QMalcolm
Copy link
Contributor

QMalcolm commented Dec 9, 2024

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

Currently, if:

  • you have a dbt project with a microbatch model
  • that microbatch model is a leaf node (nothing down stream depends on it)
  • one of the batches for that microbatch model fails (other than the first batch)
    Then the exit code is 0 which indicates success

Note
If the microbatch model isn't a leaf node (i.e. there is a downstream dependent), then it's downstream dependents will be appropriately skipped making the exit code 1

Expected Behavior

Any batch failure should mean result in a non-zero exit code

Steps To Reproduce

  1. Have a project with a microbatch model
  2. Alter the microbatch model so a batch (not the first batch) will fail
  3. dbt run
  4. Observe the non-zero exit code for the run via ``

Relevant log output

(venv) quigleymalcolm@Quigley-Malcolm microbatch_demo % dbt run --select +microbatch_model
20:44:27  Running with dbt=1.10.0-a1
20:44:27  Registered adapter: postgres=1.9.0-rc1
20:44:28  Found 3 models, 429 macros
20:44:28  
20:44:28  Concurrency: 1 threads (target='dev')
20:44:28  
20:44:28  1 of 2 START sql table model microbatch_demo.input_model ....................... [RUN]
20:44:28  1 of 2 OK created sql table model microbatch_demo.input_model .................. [SELECT 4 in 0.08s]
20:44:28  2 of 2 START sql microbatch model microbatch_demo.microbatch_model ............. [RUN]
20:44:28  Batch 1 of 4 START batch 2024-12-06 of microbatch_demo.microbatch_model .............. [RUN]
20:44:28  Batch 1 of 4 OK created batch 2024-12-06 of microbatch_demo.microbatch_model ......... [MERGE 0 in 0.09s]
20:44:28  Batch 2 of 4 START batch 2024-12-07 of microbatch_demo.microbatch_model .............. [RUN]
20:44:28  Batch 2 of 4 OK created batch 2024-12-07 of microbatch_demo.microbatch_model ......... [MERGE 0 in 0.04s]
20:44:28  Batch 3 of 4 START batch 2024-12-08 of microbatch_demo.microbatch_model .............. [RUN]
20:44:28  Batch 3 of 4 OK created batch 2024-12-08 of microbatch_demo.microbatch_model ......... [MERGE 0 in 0.04s]
20:44:28  Batch 4 of 4 START batch 2024-12-09 of microbatch_demo.microbatch_model .............. [RUN]
20:44:28  Unhandled error while executing 
Exception on worker thread. Database Error
  syntax error at or near "SQL"
  LINE 31:   INVALID SQL;
                     ^
20:44:28  Batch 4 of 4 ERROR creating batch 2024-12-09 of microbatch_demo.microbatch_model ..... [ERROR in 0.02s]
20:44:28  2 of 2 PARTIALLY created sql microbatch model microbatch_demo.microbatch_model . [PARTIAL SUCCESS (3/4) in 0.21s]
20:44:28  
20:44:28  Finished running 1 incremental model, 1 table model in 0 hours 0 minutes and 0.42 seconds (0.42s).
20:44:28  
20:44:28  Completed with 1 partial success and 0 warnings
20:44:28  
20:44:28  Done. PASS=1 WARN=0 ERROR=1 SKIP=0 TOTAL=2
(venv) quigleymalcolm@Quigley-Malcolm microbatch_demo % echo $?                           
0

Environment

- OS: macOS 15.1
- Python: 3.10.15
- dbt: 1.9.0

Which database adapter are you using with dbt?

postgres

Additional Context

Screenshot 2024-12-09 at 14 48 22
@QMalcolm QMalcolm added bug Something isn't working triage microbatch Issues related to the microbatch incremental strategy and removed triage labels Dec 9, 2024
@QMalcolm QMalcolm self-assigned this Dec 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working microbatch Issues related to the microbatch incremental strategy
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant