feat: Add prompt injection protection mechanism #28
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: nextedoff <[email protected]>
…g injection protection Signed-off-by: nextedoff <[email protected]>
Signed-off-by: nextedoff <[email protected]>
Signed-off-by: nextedoff <[email protected]>
nextedoff
force-pushed
the
feature-injection-model
branch
from
36d880f to
7d2d98b
Compare
phoevos
requested changes
Aug 19, 2024
…nisms Signed-off-by: nextedoff <[email protected]>
Signed-off-by: nextedoff <[email protected]>
phoevos
changed the title
Fixed docstrings in promptprotection.py, slightly rephrasing some for clarity and making sure that they're in proper markdown so that they are rendered correctly in the documentation. Renamed the 'config' dictionary of the 'PromptProtection' class to 'protections'. Signed-off-by: Phoevos Kalemkeris <[email protected]>
Signed-off-by: Phoevos Kalemkeris <[email protected]>
phoevos
approved these changes
Aug 21, 2024
Signed-off-by: Phoevos Kalemkeris <[email protected]>
Signed-off-by: Phoevos Kalemkeris <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
This feature is adding a way to implement new protection methods for the query, focusing on prompt injection protection in this PR. For injection protection model, model named
protectai/deberta-v3-base-prompt-injectionwas used.New models and guard mechanisms can be added to
promptprotection.py.Currently, the injection protection can either be turned on via API and user interface as seen in the screenshot, or it can be set via environment variables:
USE_INJECTION_PROTECTION="true"Does this introduce a breaking change?
When developers merge from main and run the server, azd up, or azd deploy, will this produce an error?
If you're not sure, try it out on an old environment.
Does this require changes to learn.microsoft.com docs?
Type of change
Code quality checklist
python -m pytest).python -m pytest --covto verify 100% coverage of added linespython -m mypyto check for type errorsruffandblackmanually on my code.