Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scoring tests and fixes #131

Merged
merged 8 commits into from
May 26, 2021
Merged

Scoring tests and fixes #131

merged 8 commits into from
May 26, 2021

Conversation

clusterfudge
Copy link
Collaborator

Addresses some components of #130, including

  • Removing custom code for an itertools replacement

  • Updating some docs typos

  • Updating some docstrings to better reflect reality

  • Fixing scoring in Intent.validate_with_tags, adding associated tests.

  • Description of how to validate or test this PR
    ./run_tests.sh should capture new tests and linter.

  • Whether you have signed a CLA (Contributor Licensing Agreement)
    Signed

"""
intent, tags = self.validate_with_tags(tags, confidence)
return intent

def validate_with_tags(self, tags, parse_weight):
def validate_with_tags(self, tags, confidence):
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's unfortunate, but this actually constitutes a breaking change (specifically named arguments; caught it when merging some new tests). I have kept all the previous fixes where we were overriding confidence or using the wrong one, and we have scoring tests to validate that.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth keeping the compatibility with something like

    def validate_with_tags(self, tags, confidence=None, parse_weight=None):
        if parse_weight:
            print("parse_weight is deprecated, use confidence instead")
        confidence = confidence or parse_weight
        if confidence is None:
            raise TypeError('Missing required argument confidence')

Probably not...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so; the parser is so lightweight that I'm actually concerned about the perf of something like that. I suppose we could just emit on first use? But I don't think the changes from the PR that introduced parse_weight have made it too broadly into the world yet, so its uses should be minimal.

Copy link
Collaborator

@forslund forslund left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the logic changes and the cleanups all look good, as far as I understand the code here atleast :)

I added some nitpicky-comments.

Also could you drop the two commits renaming the EventEmitter class back and forth?

"""
for tag in tags:
for entity in tag.get('entities'):
for v, t in entity.get('data'):
if t.lower() == entity_type.lower() and \
(tag.get('start_token', 0) > after_index or tag.get('from_context', False)):
(tag.get('start_token', 0) > after_index or \
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the \ is technically not needed when in parenthesis like this

"""
intent, tags = self.validate_with_tags(tags, confidence)
return intent

def validate_with_tags(self, tags, parse_weight):
def validate_with_tags(self, tags, confidence):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth keeping the compatibility with something like

    def validate_with_tags(self, tags, confidence=None, parse_weight=None):
        if parse_weight:
            print("parse_weight is deprecated, use confidence instead")
        confidence = confidence or parse_weight
        if confidence is None:
            raise TypeError('Missing required argument confidence')

Probably not...

adapt/intent.py Outdated
@@ -165,17 +171,22 @@ def validate_with_tags(self, tags, confidence):
intent_confidence += confidence

if len(self.at_least_one) > 0:
best_resolution = resolve_one_of(tags, self.at_least_one)
# we want to use a copy of the original tags here, as opposed to
# local_tags. This allows us to have overlaps between `required`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be "as opposed to tags" instead of ...local_tags

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this whole comment block was supposed to go. I'll update.

@clusterfudge
Copy link
Collaborator Author

As for commit history, I'm actually planning to squash, so the EventEmitter commits should just fall out. I'll need to add the version bump as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants