Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Complete API spec implementation for flow_framework #565

Merged
merged 11 commits into from
Sep 11, 2024
3 changes: 2 additions & 1 deletion .cspell
Original file line number Diff line number Diff line change
Expand Up @@ -192,4 +192,5 @@ vectory
whoamiprotected
wordnet
Yrtsd
reprovision
reprovision
deprovision
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
- Added request and response schemas for `/_cluster/decommission/awareness` ([#524](https://github.com/opensearch-project/opensearch-api-specification/pull/524))
- Added `Bytes` component of type number ([#552](https://github.com/opensearch-project/opensearch-api-specification/pull/552))
- Added `xy_shape` query ([#531](https://github.com/opensearch-project/opensearch-api-specification/pull/531))
- Added `/_plugins/_flow_framework/` ,`_search`,`state/_search`,`_provision`,`_deprovision`,`_steps`,`_status`([#508](https://github.com/opensearch-project/opensearch-api-specification/issues/508))

### Changed

Expand Down
374 changes: 353 additions & 21 deletions spec/namespaces/flow_framework.yaml

Large diffs are not rendered by default.

246 changes: 245 additions & 1 deletion spec/schemas/flow_framework.common.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,34 @@ info:
paths: { }
components:
schemas:
AllowDelete:
description: Required when deleting resources involves a high potential for data loss.
type: string
WorkflowID:
type: string
WorkflowStepName:
type: string
Provision:
description: Provision the workflow as part of the request.
type: boolean
default: false
Reprovision:
type: boolean
default: false
UseCase:
type: string
Validation:
description: Validate the workflow. Valid values are all (validate the template) and none (do not validate the template). Default is all.
type: string
default: all
UpdateFields:
description: Update only the fields included in the request body.
type: boolean
default: false
All:
description: The all parameter specifies whether the response should return all fields.
type: boolean
default: false
FlowFrameworkCreate:
type: object
properties:
Expand Down Expand Up @@ -105,6 +133,82 @@ components:
failed:
type: integer
description: The number of shards where the deletion failed.
query:
type: object
description: The search query to match workflows. Use `match_all` to retrieve all workflows, or `match` to search by specific fields like `use_case`.
properties:
match:
type: object
match_all:
type: object
total:
type: object
properties:
value:
type: integer
description: Total number of matching documents.
relation:
type: string
description: The relation of the total hits.
itemsObject:
type: object
properties:
_index:
type: string
description: The index name where the document resides.
_id:
type: string
description: The document ID.
_version:
type: integer
description: The version of the document.
_seq_no:
type: integer
description: The sequence number of the document.
_primary_term:
type: integer
description: The primary term of the document.
_score:
type: number
format: float
description: The score of the search hit.
_source:
$ref: '#/components/schemas/FlowFrameworkGetResponse'
StateItems:
type: object
properties:
_index:
type: string
description: The index name where the document resides.
_id:
type: string
description: The document ID.
_version:
type: integer
description: The version of the document.
_seq_no:
type: integer
description: The sequence number of the document.
_primary_term:
type: integer
description: The primary term of the document.
_score:
type: number
format: float
description: The score of the search hit.
_source:
$ref: '#/components/schemas/SearchStateResponse'
SearchStateResponse:
type: object
properties:
workflow_id:
type: string
provisioning_progress:
type: string
state:
type: string
user:
$ref: '#/components/schemas/user'
user:
type: object
properties:
Expand All @@ -124,4 +228,144 @@ components:
type: string
user_requested_tenant:
type: string
nullable: true
nullable: true
hits:
type: object
properties:
total:
$ref: '#/components/schemas/total'
max_score:
type: number
format: float
description: The maximum score of the search hits.
hits:
type: array
items:
$ref: '#/components/schemas/itemsObject'
StateHits:
type: object
properties:
total:
$ref: '#/components/schemas/total'
max_score:
type: number
format: float
description: The maximum score of the search hits.
hits:
type: array
items:
$ref: '#/components/schemas/StateItems'
WorkflowStep:
type: object
properties:
inputs:
type: array
items:
type: string
description: The list of required inputs for the workflow step
outputs:
type: array
items:
type: string
description: The list of outputs generated by the workflow step
required_plugins:
type: array
items:
type: string
description: The list of plugins required for the workflow step
WorkFlowStatusDefaultResponse:
type: object
properties:
workflow_id:
type: string
description: The ID of the workflow
error:
type: string
description: Any error state associated with the workflow
nullable: true
state:
type: string
description: The current state of the workflow
resources_created:
type: array
items:
type: string
description: A list of resources created by the workflow
WorkFlowStatusFullResponse:
type: object
properties:
workflow_id:
type: string
description: The ID of the workflow
error:
type: string
description: Any error state associated with the workflow
nullable: true
state:
type: string
description: The current state of the workflow
enum:
- COMPLETED
- FAILED
- NOT_STARTED
- PROVISIONING
resources_created:
type: array
items:
type: string
description: A list of resources created by the workflow
provisioning_progress:
type: string
description: The progress of the provisioning process
nullable: true
provision_start_time:
type: string
format: date-time
description: The start time of the provisioning process
provision_end_time:
type: string
format: date-time
description: The end time of the provisioning process
user:
$ref: '#/components/schemas/user'
user_outputs:
type: array
items:
type: string
description: Outputs generated by the user
UserProvidedSubstitutionExpressions:
type: object
additionalProperties:
type: string
description: The value for the user-provided key. Keys represent substitution expressions.
SearchWorkflowRequest:
type: object
properties:
query:
$ref: '#/components/schemas/query'
WorkflowSearchResponse:
type: object
properties:
took:
type: integer
description: Time in milliseconds that the request took to complete.
timed_out:
type: boolean
description: Indicates if the request timed out.
_shards:
$ref: '#/components/schemas/shards'
hits:
$ref: '#/components/schemas/hits'
WorkflowSearchStateResponse:
type: object
properties:
took:
type: integer
description: Time in milliseconds that the request took to complete.
timed_out:
type: boolean
description: Indicates if the request timed out.
_shards:
$ref: '#/components/schemas/shards'
hits:
$ref: '#/components/schemas/StateHits'
82 changes: 80 additions & 2 deletions spec/schemas/flow_framework.errors.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ components:
example: This API is disabled. To enable it, set [flow_framework.enabled] to true.
status:
type: integer
BadRequestError:
InvalidParameterError:
content:
application/json:
type: object
Expand Down Expand Up @@ -66,6 +66,35 @@ components:
example: Workflow ID can not be null
status:
type: integer
DuplicateKeyError:
type: object
properties:
error:
type: string
description: Error message when a duplicate key is found in the request.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure "duplicate key" is self-descriptive here. Is this a duplicate param, or a duplicate field in the body JSON?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/opensearch-project/flow-framework/blob/dc20feb9d25b7c60bd76c8f5cc66a5a1eed860e2/src/main/java/org/opensearch/flowframework/rest/RestProvisionWorkflowAction.java#L125C2-L128C22

Because we can pass it as a query parameter or as a string value of a request body field

The error message is this. I naming DuplicateKeyError by the error message, should i change it ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks correct to me, should match the code.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now as @dblock says it's ok to match the code. The error message should probably be "duplicate param" but not a big deal to leave it as is. If you get the message you'll know what you did wrong. Whoever wrote that code should have thought better. ;)

status:
type: integer
description: HTTP status code for the error.
example: 400
InvalidRequestBodyFieldError:
type: object
properties:
error:
type: string
description: Error message when a request body field is not a string.
status:
type: integer
description: HTTP status code for the error.
RequestBodyParsingFailedError:
type: object
properties:
error:
type: string
description: Error message when request body parsing fails.
status:
type: integer
description: HTTP status code for the error.
example: 400
WorkflowSaveError:
content:
application/json:
Expand Down Expand Up @@ -95,4 +124,53 @@ components:
type: string
example: Failed to retrieve template (12345) from global context.
code:
type: integer
type: integer
WorkflowStepsRetrieveError:
content:
application/json:
type: object
properties:
message:
type: string
example: Failed to retrieve workflow step json.
code:
type: integer
DeprovisioningError:
type: object
properties:
error:
type: string
description: Describes the deprovisioning error and identifies resources that were not deprovisioned
example: Failed to deprovision some resources [connector_id Lw7PX4wBfVtHp98y06wV].
required:
- error
DeprovisioningForbiddenError:
type: object
properties:
error:
type: string
description: Describes the resources that require the allow_delete parameter for deprovisioning
example: These resources require the allow_delete parameter to deprovision [index_name my-index].
required:
- error
RequestTimeoutError:
type: object
properties:
error:
type: string
description: Error message when the request times out.
status:
type: integer
description: HTTP status code for the error.
example: 408
BadRequestError:
type: object
properties:
error:
type: string
description: Error message when the request body or parameters are invalid.
example: Invalid request body or query parameters.
status:
type: integer
description: HTTP status code for the error.
example: 400
Loading
Loading