Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[OPIK-641] create guardrails integration #954

Merged
merged 20 commits into from
Dec 24, 2024

Conversation

alexkuzmik
Copy link
Collaborator

@alexkuzmik alexkuzmik commented Dec 23, 2024

Details

Added the integration with guardrails.
After tracking the Guard instance, every validator starts producing traces with results to Opik. "pass" and "fail" tags are added to the corresponding traces based on the validation result.
Example:

from guardrails import Guard, OnFailAction
from guardrails.hub import CompetitorCheck, ToxicLanguage

import opik
from opik.integrations.guardrails.guardrails_tracker import track_guardrails

competitor_check = CompetitorCheck(
    ["Apple", "Microsoft", "Google"], on_fail=OnFailAction.NOOP
)
toxic_check = ToxicLanguage(
    threshold=0.5, validation_method="sentence", on_fail=OnFailAction.NOOP
)
guard: Guard = Guard().use_many(competitor_check, toxic_check)
guard = track_guardrails(guard, project_name="guardrails-example")

guard.validate(
    "An apple a day keeps a doctor away. This is good advice for keeping your health."
)  # Both the guardrails pass
    

Issues

Resolves #

Testing

integration tests are added

Documentation

docstrings are added

@alexkuzmik alexkuzmik requested review from a team as code owners December 23, 2024 15:15
@alexkuzmik alexkuzmik self-assigned this Dec 23, 2024
@alexkuzmik alexkuzmik marked this pull request as draft December 23, 2024 15:15
@alexkuzmik alexkuzmik marked this pull request as ready for review December 23, 2024 21:26
@alexkuzmik
Copy link
Collaborator Author

@japdubengsub the PR ready for review but the integration test will likely be updated. Current version pulls quite heavy dependencies that take 2.5 minutes to install.

japdubengsub
japdubengsub previously approved these changes Dec 24, 2024
@alexkuzmik alexkuzmik merged commit af3dbcf into main Dec 24, 2024
34 checks passed
@alexkuzmik alexkuzmik deleted the OPIK-641-create-guardrails-integration branch December 24, 2024 13:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants