Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fix] Ensure inferred primary_key is a List[str] #10984

Merged
merged 1 commit into from
Nov 6, 2024

Conversation

MichelleArk
Copy link
Contributor

@MichelleArk MichelleArk commented Nov 6, 2024

Resolves #10983

Problem

Even though our schemas dictate that node.primary_key is a List[str], it is possible for None values to creep into the primary_key if unique or unique_combination_of_columns are not specified correctly and contain null values. This is a violation of our published manifest schema and can lead to parsing issues downstream.

Solution

Do a better job validating the types of generic test kwargs primary key inference is depending on, especially given they may come from other packages like dbt_utils and not be validated at all (unlike our own constraint structures).

This is not a breaking behaviour change -- no project should have been relying on null values in the primary_key as they are semantically useless. If anything it was probably breaking parsing / validation on the consumption end if the unique or unique_combination_of_columns was invalid in niche cases.

Checklist

  • I have read the contributing guide and understand what's expected of me.
  • I have run this code in development, and it appears to resolve the stated issue.
  • This PR includes tests, or tests are not required or relevant for this PR.
  • This PR has no interface changes (e.g., macros, CLI, logs, JSON artifacts, config files, adapter interface, etc.) or this PR has already received feedback and approval from Product or DX.
  • This PR includes type annotations for new and modified functions.

@MichelleArk MichelleArk requested a review from a team as a code owner November 6, 2024 19:53
@cla-bot cla-bot bot added the cla:yes label Nov 6, 2024
@MichelleArk MichelleArk requested a review from gshank November 6, 2024 19:54
Copy link

codecov bot commented Nov 6, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 89.13%. Comparing base (0c51985) to head (29ae8e9).
Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #10984      +/-   ##
==========================================
+ Coverage   89.08%   89.13%   +0.04%     
==========================================
  Files         183      183              
  Lines       23580    23580              
==========================================
+ Hits        21006    21017      +11     
+ Misses       2574     2563      -11     
Flag Coverage Δ
integration 86.49% <100.00%> (+0.11%) ⬆️
unit 62.79% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
Unit Tests 62.79% <100.00%> (ø)
Integration Tests 86.49% <100.00%> (+0.11%) ⬆️

@MichelleArk MichelleArk merged commit e451a37 into main Nov 6, 2024
67 checks passed
@MichelleArk MichelleArk deleted the more-defensive-infer-primary-key branch November 6, 2024 20:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
2 participants