Add documentation for default use cases #6767
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
amitgalitz
requested review from
hdhalter,
kolchfa-aws,
Naarcha-AWS,
vagimeli,
AMoo-Miki,
natebower,
dlvenable and
stephen-crawford
as code owners
hdhalter
reviewed
Mar 21, 2024
| nav_order: 10 | ||
| --- | ||
|
|
||
| # Default Use Cases |
There was a problem hiding this comment.
I would suggest a more meaningful name for the topic. Workflow templates? Sample workflows?
There was a problem hiding this comment.
changed name to Default Workflows
hdhalter
added
v2.13.0
release-notes
ohltyler
reviewed
Mar 21, 2024
|
Thank you for providing the PR, @amitgalitz! After you are done with the tech review, please let me know and I will do a doc review on the PR. Thanks! |
|
Linking opensearch-project/flow-framework#541 to this PR. |
amitgalitz
force-pushed
the
default-params
branch
6 times, most recently
from
188cd5b to
4b8958f
Compare
|
For the table headers I made the fix required by style check but style check is persisting there. Also for the openai comment, this is already set to openai in code for use_case so can't be changed to OpenAi. Also all use cases should be all lower case. |
Signed-off-by: Amit Galitzky <[email protected]>
Signed-off-by: Amit Galitzky <[email protected]>
amitgalitz
force-pushed
the
default-params
branch
from
4b8958f to
7a5a28b
Compare
hdhalter
added
4 - Doc review
Signed-off-by: Fanit Kolchina <[email protected]>
kolchfa-aws
reviewed
Mar 28, 2024
Signed-off-by: kolchfa-aws <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>
natebower
reviewed
Mar 29, 2024
There was a problem hiding this comment.
@amitgalitz @kolchfa-aws Please see my comments and changes and let me know if you have any questions. Thanks!
|
|
||
| - Deploys the default pretrained sparse encoding model (`amazon/neural-sparse/opensearch-neural-sparse-encoding-v1`) | ||
| - Creates an ingest pipeline that contains a `sparse_encoding` processor, which converts the text in a document field to vector embeddings using the deployed model | ||
| - Creates a sample index for sparse search, specifying the default pipeline as the newly created ingest pipeline |
There was a problem hiding this comment.
I know I initially said no periods here, but once the fragments become complex/lengthy, as the second two are, then periods are preferred.
| | `openai_chat_model_deploy` | Creates and deploys an OpenAI chat model (by default, `gpt-3.5-turbo`). | `create_connector.credential.key` |[Defaults](https://github.com/opensearch-project/flow-framework/blob/2.13/src/main/resources/defaults/openai-chat-defaults.json) | | ||
| | `local_neural_sparse_search_bi_encoder` | Configures [neural sparse search]({{site.url}}{{site.baseurl}}/search-plugins/neural-sparse-search/): <br> - Deploys a pretrained sparse encoding model<br> - Creates an ingest pipeline with a sparse encoding processor <br> - Creates a sample index to use for sparse search, specifying the newly created pipeline as default pipeline | None |[Defaults](https://github.com/opensearch-project/flow-framework/blob/2.13/src/main/resources/defaults/local-sparse-search-biencoder-defaults.json) | | ||
| | `semantic_search` | Configures [semantic search]({{site.url}}{{site.baseurl}}/search-plugins/semantic-search/): <br> - Creates an ingest pipeline with a `text_embedding` processor and a k-NN index <br> You must provide a model ID of the text embedding model to use. | `create_ingest_pipeline.model_id` |[Defaults](https://github.com/opensearch-project/flow-framework/blob/2.13/src/main/resources/defaults/semantic-search-defaults.json) | | ||
| | `semantic_search_with_query_enricher` | Configures [semantic search]({{site.url}}{{site.baseurl}}/search-plugins/semantic-search/) similarly to the `semantic_search` template. Adds a [`query_enricher`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/neural-query-enricher/) search processor that sets a default model ID is defaulted for neural queries. You must provide a model ID of the text embedding model to use. | `create_ingest_pipeline.model_id` |[Defaults](https://github.com/opensearch-project/flow-framework/blob/2.13/src/main/resources/defaults/semantic-search-query-enricher-defaults.json) | |
There was a problem hiding this comment.
I'm not following "search processor that sets a default model ID is defaulted for neural queries".
| | `semantic_search` | Configures [semantic search]({{site.url}}{{site.baseurl}}/search-plugins/semantic-search/): <br> - Creates an ingest pipeline with a `text_embedding` processor and a k-NN index <br> You must provide a model ID of the text embedding model to use. | `create_ingest_pipeline.model_id` |[Defaults](https://github.com/opensearch-project/flow-framework/blob/2.13/src/main/resources/defaults/semantic-search-defaults.json) | | ||
| | `semantic_search_with_query_enricher` | Configures [semantic search]({{site.url}}{{site.baseurl}}/search-plugins/semantic-search/) similarly to the `semantic_search` template. Adds a [`query_enricher`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/neural-query-enricher/) search processor that sets a default model ID is defaulted for neural queries. You must provide a model ID of the text embedding model to use. | `create_ingest_pipeline.model_id` |[Defaults](https://github.com/opensearch-project/flow-framework/blob/2.13/src/main/resources/defaults/semantic-search-query-enricher-defaults.json) | | ||
| | `semantic_search_with_cohere_embedding` | Configures [semantic search]({{site.url}}{{site.baseurl}}/search-plugins/semantic-search/) and deploys a Cohere embedding model. You must provide the API key for the Cohere model. | `create_connector.credential.key` |[Defaults](https://github.com/opensearch-project/flow-framework/blob/2.13/src/main/resources/defaults/cohere-embedding-semantic-search-defaults.json) | | ||
| | `semantic_search_with_cohere_embedding_query_enricher` | Configures [semantic search]({{site.url}}{{site.baseurl}}/search-plugins/semantic-search/) and deploys a Cohere embedding model. Adds a [`query_enricher`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/neural-query-enricher/) search processor that sets a default model ID is defaulted for neural queries. You must provide the API key for the Cohere model. | `create_connector.credential.key` |[Defaults](https://github.com/opensearch-project/flow-framework/blob/2.13/src/main/resources/defaults/cohere-embedding-semantic-search-with-query-enricher-defaults.json) | |
hdhalter
added
5 - Editorial review
kolchfa-aws
reviewed
Mar 29, 2024
Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]>
kolchfa-aws
reviewed
Mar 29, 2024
kolchfa-aws
reviewed
Mar 29, 2024
Signed-off-by: kolchfa-aws <[email protected]>
Signed-off-by: Fanit Kolchina <[email protected]>
kolchfa-aws
reviewed
Mar 29, 2024
Signed-off-by: kolchfa-aws <[email protected]>
kolchfa-aws
approved these changes
Mar 29, 2024
hdhalter
added
3 - Tech review
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
opensearch-project/flow-framework#496 added default use cases to the flow framework plugin through substitution, this is a follow up for our substitution param PR we had.
nit: currently links in default link are all set to same link, will update to correct links but rest is ready for review
Checklist
For more information on following Developer Certificate of Origin and signing off your commits, please check here.