Skip to content

Commit

Permalink
Open-API: Refactor TableRequirements (#7710)
Browse files Browse the repository at this point in the history
* Open-API: Refactor TableRequirements

I believe the constraints are not defined in the right way.
After generating code I got:

```python
class TableRequirement(BaseModel):
    type: Literal[
        'assert-create',
        'assert-table-uuid',
        'assert-ref-snapshot-id',
        'assert-last-assigned-field-id',
        'assert-current-schema-id',
        'assert-last-assigned-partition-id',
        'assert-default-spec-id',
        'assert-default-sort-order-id',
    ]
    ref: Optional[str] = None
    uuid: Optional[str] = None
    snapshot_id: Optional[int] = Field(None, alias='snapshot-id')
    last_assigned_field_id: Optional[int] = Field(None, alias='last-assigned-field-id')
    current_schema_id: Optional[int] = Field(None, alias='current-schema-id')
    last_assigned_partition_id: Optional[int] = Field(
        None, alias='last-assigned-partition-id'
    )
    default_spec_id: Optional[int] = Field(None, alias='default-spec-id')
    default_sort_order_id: Optional[int] = Field(None, alias='default-sort-order-id')
```

Which encapulates all the requirements. After the refactor in this
PR, we'll end up with:

```python
class AssertCreate(BaseModel):
    type: Literal['assert-create']

class AssertTableUUID(BaseModel):
    type: Literal['assert-table-uuid']
    uuid: str

class AssertRefSnapshotId(BaseModel):
    type: Literal['assert-ref-snapshot-id']
    ref: str
    snapshot_id: int = Field(..., alias='snapshot-id')

class AssertLastAssignedFieldId(BaseModel):
    type: Literal['assert-last-assigned-field-id']
    last_assigned_partition_id: int = Field(..., alias='last-assigned-partition-id')

class AssertCurrentSchemaId(BaseModel):
    type: Literal['assert-current-schema-id']
    current_schema_id: int = Field(..., alias='current-schema-id')

class AssertLastAssignedPartitionId(BaseModel):
    type: Literal['assert-last-assigned-partition-id']
    last_assigned_partition_id: int = Field(..., alias='last-assigned-partition-id')

class AssertDefaultSpecId(BaseModel):
    type: Literal['assert-default-spec-id']
    default_spec_id: int = Field(..., alias='default-spec-id')

class AssertDefaultSortOrderId(BaseModel):
    type: Literal['assert-default-sort-order-id']
    default_sort_order_id: int = Field(..., alias='default-sort-order-id')

class TableRequirement(BaseModel):
    __root__: Union[
        AssertCreate,
        AssertTableUUID,
        AssertRefSnapshotId,
        AssertLastAssignedFieldId,
        AssertCurrentSchemaId,
        AssertLastAssignedPartitionId,
        AssertDefaultSpecId,
        AssertDefaultSortOrderId,
    ] = Field(..., discriminator='type')
```

Which makes sense to me.

* Apply suggestions from code review

Co-authored-by: Eduard Tudenhoefner <[email protected]>

* WIP

* Fixed the inheritance

* Generate the code

* Fix order

---------

Co-authored-by: Eduard Tudenhoefner <[email protected]>
  • Loading branch information
Fokko and nastra authored Oct 30, 2023
1 parent 0f44262 commit a661663
Show file tree
Hide file tree
Showing 2 changed files with 187 additions and 58 deletions.
103 changes: 72 additions & 31 deletions open-api/rest-catalog-open-api.py
Original file line number Diff line number Diff line change
Expand Up @@ -302,38 +302,79 @@ class RemovePropertiesUpdate(BaseUpdate):


class TableRequirement(BaseModel):
type: str


class AssertCreate(TableRequirement):
"""
Assertions from the client that must be valid for the commit to succeed. Assertions are identified by `type` -
- `assert-create` - the table must not already exist; used for create transactions
- `assert-table-uuid` - the table UUID must match the requirement's `uuid`
- `assert-ref-snapshot-id` - the table branch or tag identified by the requirement's `ref` must reference the requirement's `snapshot-id`; if `snapshot-id` is `null` or missing, the ref must not already exist
- `assert-last-assigned-field-id` - the table's last assigned column id must match the requirement's `last-assigned-field-id`
- `assert-current-schema-id` - the table's current schema id must match the requirement's `current-schema-id`
- `assert-last-assigned-partition-id` - the table's last assigned partition id must match the requirement's `last-assigned-partition-id`
- `assert-default-spec-id` - the table's default spec id must match the requirement's `default-spec-id`
- `assert-default-sort-order-id` - the table's default sort order id must match the requirement's `default-sort-order-id`
"""

type: Literal[
'assert-create',
'assert-table-uuid',
'assert-ref-snapshot-id',
'assert-last-assigned-field-id',
'assert-current-schema-id',
'assert-last-assigned-partition-id',
'assert-default-spec-id',
'assert-default-sort-order-id',
]
ref: Optional[str] = None
uuid: Optional[str] = None
snapshot_id: Optional[int] = Field(None, alias='snapshot-id')
last_assigned_field_id: Optional[int] = Field(None, alias='last-assigned-field-id')
current_schema_id: Optional[int] = Field(None, alias='current-schema-id')
last_assigned_partition_id: Optional[int] = Field(
None, alias='last-assigned-partition-id'
)
default_spec_id: Optional[int] = Field(None, alias='default-spec-id')
default_sort_order_id: Optional[int] = Field(None, alias='default-sort-order-id')
The table must not already exist; used for create transactions
"""

type: Literal['assert-create']


class AssertTableUUID(TableRequirement):
"""
The table UUID must match the requirement's `uuid`
"""

type: Literal['assert-table-uuid']
uuid: str


class AssertRefSnapshotId(TableRequirement):
"""
The table branch or tag identified by the requirement's `ref` must reference the requirement's `snapshot-id`; if `snapshot-id` is `null` or missing, the ref must not already exist
"""

type: Literal['assert-ref-snapshot-id']
ref: str
snapshot_id: int = Field(..., alias='snapshot-id')


class AssertLastAssignedFieldId(TableRequirement):
"""
The table's last assigned column id must match the requirement's `last-assigned-field-id`
"""

type: Literal['assert-last-assigned-field-id']
last_assigned_field_id: int = Field(..., alias='last-assigned-field-id')


class AssertCurrentSchemaId(TableRequirement):
"""
The table's current schema id must match the requirement's `current-schema-id`
"""

type: Literal['assert-current-schema-id']
current_schema_id: int = Field(..., alias='current-schema-id')


class AssertLastAssignedPartitionId(TableRequirement):
"""
The table's last assigned partition id must match the requirement's `last-assigned-partition-id`
"""

type: Literal['assert-last-assigned-partition-id']
last_assigned_partition_id: int = Field(..., alias='last-assigned-partition-id')


class AssertDefaultSpecId(TableRequirement):
"""
The table's default spec id must match the requirement's `default-spec-id`
"""

type: Literal['assert-default-spec-id']
default_spec_id: int = Field(..., alias='default-spec-id')


class AssertDefaultSortOrderId(TableRequirement):
"""
The table's default sort order id must match the requirement's `default-sort-order-id`
"""

type: Literal['assert-default-sort-order-id']
default_sort_order_id: int = Field(..., alias='default-sort-order-id')


class RegisterTableRequest(BaseModel):
Expand Down
142 changes: 115 additions & 27 deletions open-api/rest-catalog-open-api.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -1879,54 +1879,142 @@ components:
- $ref: '#/components/schemas/RemovePropertiesUpdate'

TableRequirement:
description:
Assertions from the client that must be valid for the commit to succeed. Assertions are identified by `type` -

- `assert-create` - the table must not already exist; used for create transactions

- `assert-table-uuid` - the table UUID must match the requirement's `uuid`

- `assert-ref-snapshot-id` - the table branch or tag identified by the requirement's `ref` must reference the requirement's `snapshot-id`; if `snapshot-id` is `null` or missing, the ref must not already exist

- `assert-last-assigned-field-id` - the table's last assigned column id must match the requirement's `last-assigned-field-id`

- `assert-current-schema-id` - the table's current schema id must match the requirement's `current-schema-id`

- `assert-last-assigned-partition-id` - the table's last assigned partition id must match the requirement's `last-assigned-partition-id`

- `assert-default-spec-id` - the table's default spec id must match the requirement's `default-spec-id`
discriminator:
propertyName: type
mapping:
assert-create: '#/components/schemas/AssertCreate'
assert-table-uuid: '#/components/schemas/AssertTableUUID'
assert-ref-snapshot-id: '#/components/schemas/AssertRefSnapshotId'
assert-last-assigned-field-id: '#/components/schemas/AssertLastAssignedFieldId'
assert-current-schema-id: '#/components/schemas/AssertCurrentSchemaId'
assert-last-assigned-partition-id: '#/components/schemas/AssertLastAssignedPartitionId'
assert-default-spec-id: '#/components/schemas/AssertDefaultSpecId'
assert-default-sort-order-id: '#/components/schemas/AssertDefaultSortOrderId'
type: object
required:
- type
properties:
type:
type: "string"

- `assert-default-sort-order-id` - the table's default sort order id must match the requirement's `default-sort-order-id`
AssertCreate:
allOf:
- $ref: "#/components/schemas/TableRequirement"
type: object
description: The table must not already exist; used for create transactions
required:
- type
properties:
type:
type: string
enum:
- assert-create
- assert-table-uuid
- assert-ref-snapshot-id
- assert-last-assigned-field-id
- assert-current-schema-id
- assert-last-assigned-partition-id
- assert-default-spec-id
- assert-default-sort-order-id
ref:
enum: ["assert-create"]

AssertTableUUID:
allOf:
- $ref: "#/components/schemas/TableRequirement"
description: The table UUID must match the requirement's `uuid`
required:
- type
- uuid
properties:
type:
type: string
enum: ["assert-table-uuid"]
uuid:
type: string

AssertRefSnapshotId:
allOf:
- $ref: "#/components/schemas/TableRequirement"
description:
The table branch or tag identified by the requirement's `ref` must reference the requirement's `snapshot-id`;
if `snapshot-id` is `null` or missing, the ref must not already exist
required:
- type
- ref
- snapshot-id
properties:
type:
type: string
enum: [ "assert-ref-snapshot-id" ]
ref:
type: string
snapshot-id:
type: integer
format: int64

AssertLastAssignedFieldId:
allOf:
- $ref: "#/components/schemas/TableRequirement"
description:
The table's last assigned column id must match the requirement's `last-assigned-field-id`
required:
- type
- last-assigned-field-id
properties:
type:
type: string
enum: [ "assert-last-assigned-field-id" ]
last-assigned-field-id:
type: integer

AssertCurrentSchemaId:
allOf:
- $ref: "#/components/schemas/TableRequirement"
description:
The table's current schema id must match the requirement's `current-schema-id`
required:
- type
- current-schema-id
properties:
type:
type: string
enum: [ "assert-current-schema-id" ]
current-schema-id:
type: integer

AssertLastAssignedPartitionId:
allOf:
- $ref: "#/components/schemas/TableRequirement"
description:
The table's last assigned partition id must match the requirement's `last-assigned-partition-id`
required:
- type
- last-assigned-partition-id
properties:
type:
type: string
enum: [ "assert-last-assigned-partition-id" ]
last-assigned-partition-id:
type: integer

AssertDefaultSpecId:
allOf:
- $ref: "#/components/schemas/TableRequirement"
description:
The table's default spec id must match the requirement's `default-spec-id`
required:
- type
- default-spec-id
properties:
type:
type: string
enum: [ "assert-default-spec-id" ]
default-spec-id:
type: integer

AssertDefaultSortOrderId:
allOf:
- $ref: "#/components/schemas/TableRequirement"
description:
The table's default sort order id must match the requirement's `default-sort-order-id`
required:
- type
- default-sort-order-id
properties:
type:
type: string
enum: [ "assert-default-sort-order-id" ]
default-sort-order-id:
type: integer

Expand Down

0 comments on commit a661663

Please sign in to comment.