[META] Improvements for default use cases

[amitgalitz]

Is your feature request related to a problem?

In 2.13 we released a new feature that enables customers to utilize predefined templates to configure their workflows by selecting a given use case. Currently the feature is documented here: https://opensearch.org/docs/latest/automating-configurations/workflow-templates/

For the first release of the feature in order not to cause any later breaking changes we made a few decisions that can be revisited.

1. We currently host our substitution ready templates in a folder here, and the different defaults for each use case here. The reason these aren’t in the sample is because we didn’t want to make a breaking change to our template and because we felt that in the future their can be more optimal ways to utilize one substitution template with multiple different default files so it wont always be 1:1 match between substitution ready templates and defaults.

2. The defaults and predefined templates are currently in our repo as part of the resources folder to allow for us to have full ownership and to couple them with the official release of the feature in 2.13. However as we expand the number of use cases we support we might need a better path forward on where to host these templates, whether it be in a different structure within the repo, uploaded to a system index, or in an entirely new location. (related: [FEATURE] Enhance ease of adding additional defaults to code #584)

Additional Action Items and suggest improvements

• We need to define criteria on what meets the bar for a new predefined template
• id field need to be removed from some of the create index steps in various templates
• Template that use pretrained model can have their dimensions automatically fetched from get model so user doesn’t have to over-write that parameter if they change model
• Increased variation in defined template [FEATURE] More variation for default use cases  #667
• [FEATURE] Enhance ease of adding additional defaults to code #584
• Change substiution file names to align with use case names, with _ instead -
• Remove the default weights field from hybrid search on template and defaults file
• Allow capability to change default text and image type for multimodal templates
• Add more pretrained model templates
• Add template with reindex step

[owaiskazi19]

Thanks for creating this issue. We should also change the substitution templates files names with _ instead of - since that's what we support in the API calls

POST /_plugins/_flow_framework/workflow?use_case=semantic_search_with_cohere_embedding_query_enricher&provision=true
{
    "create_connector.credential.key" : "<YOUR API KEY>"
}

[amitgalitz]

Thanks for creating this issue. We should also change the substitution templates files names with _ instead of - since that's what we support in the API calls

POST /_plugins/_flow_framework/workflow?use_case=semantic_search_with_cohere_embedding_query_enricher&provision=true
{
    "create_connector.credential.key" : "<YOUR API KEY>"
}

good call out, added this task