feat: Add support for v2 model_dump and v1 dict to pydantic plugin

[kedod]

Description

• Add support for additional V1 model_dump and V2 dict parameters in PydanticPlugin

Closes #3147

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.26%. Comparing base (45d2523) to head (1e1a98c).

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #3171   +/-   ##
========================================
  Coverage    98.26%   98.26%           
========================================
  Files          322      322           
  Lines        14728    14738   +10     
  Branches      2347     2347           
========================================
+ Hits         14472    14482   +10     
  Misses         117      117           
  Partials       139      139           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[guacs]

What if instead of taking in the arguments explicitly, we take a dictionary of the keyword arguments that we'll pass to each function? My reasoning is that, this way if pydantic adds a new kwarg or changes a default value or deprecates a kwarg, then we don't have to worry about it. The user can make the change accordingly in their code and there would be no issues. Otherwise, we would have to keep in sync with pydantic which can become difficult if the kwargs differ based on the pydantic version.

A drawback to this approach is that users lose the type checking, but I feel the flexibility it provides is nice.

Also, I was thinking that along with this we could take arguments for model_validate/pase_obj as well though I don't know if parse_obj does take any arguments other than the value as a dictionary.

[provinzkraut]

@guacs I agree that it would be easier, but I'd say we should side with the more type safe option here, since that's what we're doing everywhere else.

There is a way to type something like this btw, but it's not pretty:

from typing import Callable, Any, Generic, ParamSpec

P = ParamSpec("P")

class _Foo(Generic[P]):
    def __init__(self, **kwargs: P.kwargs) -> None:
        pass


def _make_foo(f: Callable[P, Any]) -> type[_Foo[P]]:
    return _Foo[P]


def func(param: str) -> None:
    pass


Foo = _make_foo(func)

Foo(param=1)

This actually type-checks correctly, but it's quite ugly. Not sure if there's a better way to do something like this; If there is, I'm not aware. @sobolevn maybe you have an idea here? :)

[kedod]

@guacs @provinzkraut
I've met a few problems with extending PydanticSchemaPlugin.

Let's for example talk about exclude_none=True parameter.
Then we have two possible response outputs:

• our response will have param if it's value is not None.
• our response will not have param key if value == None

How would you see this in OpenAPI spec?
I was thinking about using oneOf. Something similar to the following example:

responses:
        '200':
          content:
            application/json:
              schema:
                oneOf:
                  - $ref: '#/components/schemas/ResponseOne'
                  - $ref: '#/components/schemas/ResponseTwo'

So there will be two possible schemas. I guess we can do the same for the examples (but IMO it's better is to keep single example with all possible fields IMO).

But there is a downsides too:

• As I can see only SwaggerUI shows response schemas
  
• **There can be many response combinations of fields (exclude_none, exclude_default...) and implementing display all of them may be time consuming. Also looking at those schemas from the user POV can be overwhelming. **

Now the worst part.
We have four others params to support not only exclude_none.
How to proceed with this? Maybe I missed something but it looks like that implementing could be not so easy.

[provinzkraut]

@kedod Have you checked what schema Pydantic generates in those cases?

[kedod]

@kedod Have you checked what schema Pydantic generates in those cases?

I've made MVCE:

from pydantic import BaseModel, Field

from litestar import Litestar, get
from litestar.contrib.pydantic import PydanticPlugin


class Model(BaseModel):
    name: str = Field(serialization_alias="alias_name")
    none: None = None
    default: str = "default"
    exclude: str


@get()
async def handler() -> Model:
    return Model(
        name="name",
        exclude="i should not be here"
    )


app = Litestar(
    [handler],
    plugins=[
        PydanticPlugin(
            prefer_alias=True, exclude_none=True, exclude_defaults=True, exclude_unset=True, exclude={"exclude"}
        )
    ],
)

This is how our response looks like:

{"alias_name":"name"}

And the SwaggerUI:

I'm wondering if can move fixing docs to the separate issue.
There is already opened problem for the alias -> #2870

[provinzkraut]

No I meant what Pydantic itself generates, not what we generate: https://docs.pydantic.dev/latest/api/base_model/#pydantic.BaseModel.model_json_schema

[kedod]

No I meant what Pydantic itself generates, not what we generate: https://docs.pydantic.dev/latest/api/base_model/#pydantic.BaseModel.model_json_schema

For the following handler:

@get()
async def handler() -> dict:
    return Model(
        name="name",
        exclude="i should not be here"
    ).model_json_schema()

We get this ->

{
  "properties": {
    "name": {
      "title": "Name",
      "type": "string"
    },
    "none": {
      "default": null,
      "title": "None",
      "type": "null"
    },
    "default": {
      "type": "string",
      "default": "default",
      "title": "Default"
    },
    "exclude": {
      "title": "Exclude",
      "type": "string"
    }
  },
  "required": [
    "name",
    "exclude"
  ],
  "title": "Model",
  "type": "object"
}

[provinzkraut]

Interesting. It seems like Pydantic itself simply doesn't take these particularities into account. There is the by_alias option for model_json_schema, but that's about it.

[provinzkraut]

How about we simply make them not required if any of the exclude options is set that could apply to a certain field? E.g. a field that can be None would be optional if exclude_none=True?

[kedod]

How about we simply make them not required if any of the exclude options is set that could apply to a certain field? E.g. a field that can be None would be optional if exclude_none=True?

Great. I will try to implement this approach.

[guacs]

@guacs I agree that it would be easier, but I'd say we should side with the more type safe option here, since that's what we're doing everywhere else.

There is a way to type something like this btw, but it's not pretty:

from typing import Callable, Any, Generic, ParamSpec

P = ParamSpec("P")

class _Foo(Generic[P]):
    def __init__(self, **kwargs: P.kwargs) -> None:
        pass


def _make_foo(f: Callable[P, Any]) -> type[_Foo[P]]:
    return _Foo[P]


def func(param: str) -> None:
    pass


Foo = _make_foo(func)

Foo(param=1)

This actually type-checks correctly, but it's quite ugly. Not sure if there's a better way to do something like this; If there is, I'm not aware. @sobolevn maybe you have an idea here? :)

I didn't understand how this black magic will allow us to type hint the arguments automatically.

[github-actions]

Documentation preview will be available shortly at https://litestar-org.github.io/litestar-docs-preview/3171

[JacobCoffee]

Sorry about this, but this should be reopened and pointed to main.