Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: harmonize python enum naming conventions #26948

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

villebro
Copy link
Member

@villebro villebro commented Feb 1, 2024

SUMMARY

This is a continuation of #26875, and refactors all Python enums to comply with the commonly used naming conventions of PascalCase for the class name, and UPPER_CASE for the names: https://docs.python.org/3/library/enum.html This exercise was quite a bit less painful than on the frontend. 😸

Unfortunately I'm not aware of anything that would come close to ESLint for Python, so for now we'll just have to try to make sure everyone adheres to these conventions during code review. I believe something similar might be possible with pylint, but I wasn't able to scope it to only apply to enums, so I'll just leave it be. But if someone knows how this can be done, or can point me in the right direction, please do let me know.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@villebro villebro requested a review from a team as a code owner February 1, 2024 05:10
@github-actions github-actions bot added the risk:db-migration PRs that require a DB migration label Feb 1, 2024
@villebro villebro removed the risk:db-migration PRs that require a DB migration label Feb 1, 2024
Copy link
Member

@john-bodley john-bodley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@villebro it would be great if pylint could prevent any future regressions. Per pylint-dev/pylint#3896 it seems like this functionality exists, so I'm a little perplexed as to why it's not working.

@villebro
Copy link
Member Author

villebro commented Feb 1, 2024

Thanks for the pointing me in the right direction @john-bodley! I agree, this seems to be covered by vanilla pylint already. Let me try to figure out why this isn't being checked correctly..

@pull-request-size pull-request-size bot added size/L and removed size/M labels Feb 1, 2024
@github-actions github-actions bot added the risk:db-migration PRs that require a DB migration label Feb 1, 2024
Copy link

codecov bot commented Feb 1, 2024

Codecov Report

Attention: Patch coverage is 63.63636% with 16 lines in your changes missing coverage. Please review.

Project coverage is 65.35%. Comparing base (76d897e) to head (e8d0809).
Report is 1173 commits behind head on master.

Files with missing lines Patch % Lines
superset/commands/chart/update.py 20.00% 4 Missing ⚠️
superset/tags/models.py 75.00% 4 Missing ⚠️
superset/common/tags.py 0.00% 3 Missing ⚠️
superset/commands/dashboard/update.py 0.00% 2 Missing ⚠️
superset/tags/filters.py 0.00% 2 Missing ⚠️
superset/daos/tag.py 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #26948      +/-   ##
==========================================
+ Coverage   60.48%   65.35%   +4.86%     
==========================================
  Files        1931      537    -1394     
  Lines       76236    39066   -37170     
  Branches     8568        0    -8568     
==========================================
- Hits        46114    25532   -20582     
+ Misses      28017    13534   -14483     
+ Partials     2105        0    -2105     
Flag Coverage Δ
hive 48.75% <50.00%> (-0.42%) ⬇️
javascript ?
presto 53.26% <50.00%> (-0.54%) ⬇️
python 65.35% <63.63%> (+1.86%) ⬆️
unit 60.87% <63.63%> (+3.25%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Comment on lines -255 to +256
object_id=1, object_type=ObjectType.dashboard.name, tag_id=tag.id
object_id=1, object_type=ObjectType.DASHBOARD, tag_id=tag.id
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@john-bodley the fact that my IDE was able to identify these type errors indicates that our /tests/ directory is in need of some serious linting love. I think the time has come to enable full linting on our Python test suites. WDYT? I can't remember how many bycatch unused imports I've removed over the last few years. I can probably start by cleaning up the unit test suite, as that seems to be the simplest place to start..

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@villebro I’ve been making some rather XXL refactors recently and wished that we had mypy and pylint checks for the entirety of the Python code base including tests and migrations.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've found mypy can be very temperamental, but mostly that's in terms of checking far beyond the scope of what it's being asked to in the first place. It does do a good job of catching problems when it's not a false positive though. pylint is much less temperamental so at least having that seems a no-brainer.

@villebro villebro removed the risk:db-migration PRs that require a DB migration label Feb 1, 2024
@github-actions github-actions bot added the risk:db-migration PRs that require a DB migration label Feb 1, 2024
@villebro villebro removed the risk:db-migration PRs that require a DB migration label Feb 1, 2024
@villebro
Copy link
Member Author

villebro commented Feb 1, 2024

image

image

@michael-s-molina michael-s-molina added the v4.0 Label added by the release manager to track PRs to be included in the 4.0 branch label Feb 1, 2024
@villebro
Copy link
Member Author

villebro commented Feb 1, 2024

This turned ugly, just as I suspected 🤦 Link to a Slack thread that I opened to get some feedback: https://apache-superset.slack.com/archives/CCKHMGRRB/p1706829726367459?thread_ts=1706828839.332729&cid=CCKHMGRRB

@michael-s-molina michael-s-molina added v4.0 Label added by the release manager to track PRs to be included in the 4.0 branch and removed v4.0 Label added by the release manager to track PRs to be included in the 4.0 branch labels Feb 23, 2024
@michael-s-molina michael-s-molina added v5.0 Label added by the release manager to track PRs to be included in the 5.0 branch and removed v4.0 Label added by the release manager to track PRs to be included in the 4.0 branch labels Mar 4, 2024
@rusackas
Copy link
Member

rusackas commented Dec 9, 2024

Converting this to a draft while it's still in need of a rebase. Hopefully it gets more attention, but if not, it might be closed in time.

@rusackas rusackas marked this pull request as draft December 9, 2024 18:35
@villebro
Copy link
Member Author

villebro commented Dec 9, 2024

I'll work on 5.0 tasks this week, and I'll see if this is still relevant.

@giftig
Copy link
Contributor

giftig commented Dec 10, 2024

Incidentally not sure if others have used it but I've heard good things about ruff as an alternative to black+isort+mypy. I haven't had chance to use it myself but maybe it's worth playing with to see if it'll resolve some of the problems we see from mypy, and possibly pylint issues as well.

@villebro
Copy link
Member Author

@giftig we indeed have ruff nowadays! Check #28158 for the initial PR enabling it!

@giftig
Copy link
Contributor

giftig commented Dec 11, 2024

Oops, guess I'm behind the times, I should have checked the project :D

@giftig
Copy link
Contributor

giftig commented Dec 11, 2024

(I see the PR also mentions uv, which I was also going to suggest next because pip-compile-multi was a headache)

@michael-s-molina
Copy link
Member

michael-s-molina commented Dec 11, 2024

@villebro If you intend to work on this, please check this comment. We found some problems when deploying 4.1 regarding the permission names. I'm mentioning this here as it's important to check if an enum values exists in the database and add migrations in case the values change.

@github-actions github-actions bot added the risk:db-migration PRs that require a DB migration label Dec 11, 2024
Comment on lines -114 to +115
if datasource_id is not None:
if datasource_id is not None and datasource_type is not None:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bycatch, a more recent linter would have likely started complaining about this sooner or later.

@github-actions github-actions bot added the api Related to the REST API label Dec 12, 2024
Comment on lines -214 to +224
summary: Get all objects associated with a tag
parameters:
- in: path
schema:
type: integer
name: tag_id
summary: Bulk create tags and tagged objects
requestBody:
description: Tag schema
required: true
content:
application/json:
schema:
type: object
properties:
tags:
description: list of tag names to add to object
type: array
items:
type: string
objects_to_tag:
description: list of object names to add to object
type: array
items:
type: array
$ref: '#/components/schemas/TagPostBulkSchema'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tags API and related schemas are in pretty rough shape. I would almost recommend doing a full refactor of these in some future version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api Related to the REST API review:draft risk:db-migration PRs that require a DB migration size/L v5.0 Label added by the release manager to track PRs to be included in the 5.0 branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants