[FEATURE] Unify the async query request metadata model

[ykmr1224]

(moved from opensearch-spark#602 since this issue is for SQL repository)

Is your feature request related to a problem?
Currently in async-query-core, we have three models which store the async query request and status: AsyncQueryJobMetadata, StatementModel, and IndexDMLResult. And those are stored in the same request index (.query_request_index_{datasourceName}) with different documentId (qid{requestId}, {requestId}, index{requestId}).
These three models overwraps each other and redundant. It also causes confusion.

What solution would you like?
Unify them to single model, and store single record in request index.

I propose following unified model to be used:

AsyncQueryMetadata

• queryId (PK)
• query // from StatementModel
• langType // from StatementModel
• datasourceName
• accountId
• jobType
• submitTime
• sessionId
• indexName
• state // from StatementModel/IndexDMLResult
• error // from StatementModel/IndexDMLResult
• queryRunTime
• applicationId
• jobId
• version (= "2.0")
• resultIndex
• statementId (= queryId) // from StatementModel

How to make it backward compatible?

In new version, we will use single interface to query/save the query metadata.
The documentId will be queryId (without prefix).
Since current StatementModel uses queryId as a key, use case for StatementModel would be fine (whether it hits older record or newer record, it contains required fields for statement use case).
For the use case for AsyncQueryMetadata (It is only when GetAsyncQueryResults API is called), we can first query with queryId (without prefix), and if the version field is not "2.0", fallback to query with "qid" prefix, which would retrieve AsyncQueryJobMetadata(older) record.
IndexDMLResult does not have read use case. (does not have concern on backward compatibility)

[andrross]

[Catch All Triage - 1, 2, 3, 4, 5]