Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: DIA-1489: Multiskill prompt autorefinement fixes #237

Merged
merged 5 commits into from
Oct 29, 2024

Conversation

robot-ci-heartex
Copy link
Collaborator

@robot-ci-heartex robot-ci-heartex commented Oct 28, 2024

Hi @niklub!

This PR was automaticaly generated via Follow Merge.
Please ensure that all linked upstream Pull Requests are merged before proceeding with this one.

@codecov-commenter
Copy link

codecov-commenter commented Oct 28, 2024

Codecov Report

Attention: Patch coverage is 40.00000% with 18 lines in your changes missing coverage. Please review.

Project coverage is 65.67%. Comparing base (0c667eb) to head (f5da333).

Files with missing lines Patch % Lines
adala/skills/collection/prompt_improvement.py 34.78% 15 Missing ⚠️
adala/agents/base.py 0.00% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #237      +/-   ##
==========================================
- Coverage   65.89%   65.67%   -0.22%     
==========================================
  Files          47       47              
  Lines        2372     2389      +17     
==========================================
+ Hits         1563     1569       +6     
- Misses        809      820      +11     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Comment on lines +434 to +437
if reapply:
predictions = await self.skills.aapply(inputs, runtime=runtime)
else:
predictions = inputs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the future, it should be possible to automatically detect if predictions are missing and aapply in that case, and get rid of this reapply arg, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually it doesn't seem clear to me whether predictions must be applied or not. However, the performance/cost implications are clear. So I'd leave this option for now and put off to decide on the best setup later when we will have some benchmarks to understand the quality tradeoff.

adala/skills/_base.py Show resolved Hide resolved
.str.cat(sep=self.input_separator)
)

chunk.to_csv('chunk.csv', index=False)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this for? debug left in?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops, debuging stuff

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@matt-bernstein matt-bernstein marked this pull request as ready for review October 29, 2024 16:33
@robot-ci-heartex robot-ci-heartex enabled auto-merge (squash) October 29, 2024 20:57
@robot-ci-heartex robot-ci-heartex merged commit c34fd15 into master Oct 29, 2024
10 of 11 checks passed
@robot-ci-heartex robot-ci-heartex deleted the fb-dia-1489 branch October 29, 2024 21:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants