Opaque Object Layout

[mbway]

This PR introduces the capability to inherit from classes where the exact layout/size is not known using the mechanism described in PEP 697. This allows inheriting from type in order to create metaclasses. These metaclasses cannot yet be used with pyo3 classes (i.e. #[pyclass(metaclass=Foo)]) but this should be a possibility in future. It may also be possible in future for pyo3 classes to extend builtin classes using the limited API and to potentially extend imported classes (eg enum.EnumType).

This PR supersedes my first attempt at creating metaclasses: #4621. Previously I was using the normal class layout which relies on the exact layout of every base class to be known. For type the base structure is ffi::PyHeapTypeObject. The contents of this structure are intended to be private (even for the 'unlimited' API) and so an alternative mechanism is required (PEP 697).

Instead of nesting each base class at the beginning of the derived class in memory, objects created with PEP 697 only require knowledge of how much additional memory the derived class requires and the address of this section of memory is accessed via PyObject_GetTypeData. The PEP 697 mechanism could potentially be used to inherit from other builtins when only the 'limited' API is available however many of these builtins also store items (eg list, set) which introduces more complications, so this PR focuses on inheriting from PyType only.

This PR is a proof of concept but I am sure that some of the decisions I made aren't the best so feedback is welcome. The current design is as follows:

Before this PR the only possible layout was the 'static' layout where the size of the base class is known. This layout was described by PyClassObject<T> (now renamed to PyStaticClassObject). This PR introduces hides the details of the static layout behind a trait InternalPyClassObjectLayout so that other layouts are possible.

Before this PR: PyClassObject<T> was used to obtain the exact layout of T: PyClassImpl assuming that layout is the 'static' layout. This PR introduces PyClassImpl::Layout so that a pyclass declares its layout. The pyclass uses PyTypeInfo::Layout to set T::Layout. Each base type declares either PyTypeInfo::Layout = PyStaticClassObject or PyTypeInfo::Layout = PyVariableClassObject.

Potential Improvements

Some questions still remain:

• how best to handle initialisation (see below)
  • using the __init__ approach, how to initialise super classes?
  • using the __init__ approach, how to restrict return value to () or PyResult<()>?
• how to handle subclassing variable sized pyclasses
  • I tried writing some unit tests that inherited from type -> MyMetaclass -> MySubMetaclass but I think there are some outstanding problems. I also am not sure how to prevent users from doing this so I have left it for now.
    • problems encountered: setattr works without using #[pyclass(dict)] but the set values aren't accessible in the subclass
    • values are not as expected after initialisation, perhaps only the subclass data is getting initialised?
• for some reason using setattr on a class extending type does not fail, unlike on a class extending object. I'm not sure if this means type has __dict__ already?

Initialisation

I'm not sure what the best way is to handle initialisation. tp_new cannot be used with metaclasses (source) however the best semantics of __init__ aren't clear. For now I am requiring that metaclasses implement Default which allows the struct to be initialised before __init__ is called (but this is a hack and may have issues properly initialising superclasses). There are runtime checks when a class is constructed that metaclasses do not define #[new] and do define __init__. This should be sufficient to ensure that users cannot create a struct with uninitialized memory.

An alternative to using __init__ could be to repurpose #[new] to wrap it and assign it to tp_init instead in the case of a metaclass but that would be more complex to implement.

[mbway]

I didn't realise I was going to hit the issue that the minimum supported rust version of pyo3 (1.63) doesn't support generic associated types which I'm using for type Layout<T: PyClassImpl>: InternalPyClassObjectLayout<T>;.

Rust 1.65 (the first with GAT support) was released on 2022-11-03 and python 3.12 (the first version to allow extending variable sized base classes) was released on 2023-10-02.

I'm going to bump the MSRV to 1.65 but I'm open to other suggestions that avoid requiring GATs. I checked the MSRV requirements according to Contributing.md and from what I can see Debian stable ships 1.63 so this is going to be a problem.

Debian bookworm is supported until June 2026 which seems like a long time to wait...

[mejrs]

I'm going to bump the MSRV to 1.65

You can't do that unfortunately. Our msrv policy is at https://github.com/PyO3/pyo3/blob/main/Contributing.md#python-and-rust-version-support-policy

What you can do is to feature gate this and only enable the functionality on Rust 1.65 and up, or figure out a way to do it without gats.

[mbway]

What you can do is to feature gate this and only enable the functionality on Rust 1.65 and up

That sounds good to me. Do you mean introduce a feature like variable_base_types that has a different MSRV? How would I go about doing that?

Would it be sufficient to introduce the feature and have it disabled by default?

[mejrs]

A feature is one way to do it, a cfg that is automatically enabled when the build script detects it's on rust 1.65+ is another method. Both have pros and cons as far as user experience goes, so it's best to have a solution that doesn't need them. The layout code is also reasonably complex, so I'd rather not mix conditional compilation into it if possible.

[mbway]

The design currently revolves around the #[pyclass] macro being able to fill out the Layout entry like so

type Layout = <#cls as #pyo3_path::PyTypeInfo>::Layout<Self>;

This way the layout is set generically in the base types like PyAny and PyType and inherited from there.

The ways I can currently see around this are:

1. continue using GAT and feature gate the functionality
   • when disabled, hard code to the static layout and catch any discrepancy at construction? Better designs are probably possible but this would be simple to implement.
2. use a mechanism like PyClass::Frozen and instead of T::Layout have PyClassLayout<T> be a single struct containing the functionality that selects between implementations at runtime by observing T::VariableSized::VALUE. This of course has some runtime overhead.
3. use some trait magic to have a marker trait like IsStaticType that is inherited from the base type and again there is a single PyClassLayout<T> that implements the functionality but the functionality is selected at compile time using the marker traits. Unfortunately I'm not sure how to achieve this. Below is my attempt
   • related to this approach, have a single marker IsVariableType and an implementation with and without this marker. Unfortunately this requires specialisation which isn't stable yet

I'm going to implement (1) for now because the steps seem clearest, so this PR can be in a state where it could hypothetically be merged, then I'm open to suggestions if someone finds a way around the limitations I'm running into.

Attempt 1

The problem here is that the static and variable markers aren't known to be disjoint, so they conflict.

trait IsStaticType {}
trait IsVariableType {}

struct PyAny;
impl IsStaticType for PyAny {}
struct PyType;
impl IsVariableType for PyType {}

struct Layout<T> { _data: PhantomData<T> }

trait LayoutMethods {
    fn access_data();
}

impl<T: IsStaticType> LayoutMethods for Layout<T> {
    fn access_data() {
        println!("hi from static");
    }
}

impl<T: IsVariableType> LayoutMethods for Layout<T> {  // ERROR: conflicting implementations
    fn access_data() {
        println!("hi from variable");
    }
}

Attempt 2

The problem here is that <MyClass as LayoutMethods>::access_data() panics, and while <MyClass as StaticLayout>::access_data() compiles and <MyClass as VariableLayout>::access_data() does not, it doesn't solve the problem of being able to use the layout without specifying which it is.

trait LayoutMethods {
    fn access_data() {
        panic!("unknown layout")
    }
}

impl<T: PyClassImpl> LayoutMethods for T {}

trait StaticLayout: Sized + 'static + LayoutMethods {
    fn access_data() {
        println!("hi from static");
    }
}
impl<T, B> StaticLayout for T
where
    T: PyClassImpl<BaseType = B> + 'static,
    B: StaticLayout,
{ }

trait VariableLayout: Sized + 'static + LayoutMethods {
    fn access_data() {
        println!("hi from variable");
    }
}
impl<T, B> VariableLayout for T
where
    T: PyClassImpl<BaseType = B> + 'static,
    B: VariableLayout,
{ }

trait PyClassImpl { type BaseType; }

struct PyAny;
impl LayoutMethods for PyAny {}
impl StaticLayout for PyAny {}

struct PyType;
impl LayoutMethods for PyType {}
impl VariableLayout for PyType {}

struct MyClass {}
impl PyClassImpl for MyClass {
    type BaseType = PyAny;
}

[mbway]

I have finished my proof of concept 🙂 It supports using the opaque layout and extending PyType as an example of a variable-sized native base class.

GATs are no longer required, replaced with PyTypeInfo::OPAQUE: bool that is inherited and can be manually forced to true with #[pyclass(opaque)] (but not to false).

This is a large change so it might be best to split this over several PRs. I think It would still be a good idea to align on whether the general approach here is good then the specifics can be refined in the individual PRs? I'm also happy to keep everything together if you prefer.

There is a lot more code now because in my original post I forgot to remove the temporary use of ffi::PyObject_Type() to obtain the *mut ffi:PyTypeObject necessary for traversing the opaque layout. I now correctly use the PyTypeObject of the pyclass being accessed (so if B extends A, previously accessing the data for A would actually return the data for B). Because access to the data is sometimes required without the GIL being held I needed to make a large refactor where the PyTypeObject can be obtained without the GIL being held (PyTypeInfo::try_get_type_object_raw()). Later I realised that the type object must be constructed in order to create an instance, so PyObjectLayout may be able to always assume the type object is available? I'm not sure so I left the two options for TypeObjectStrategy (one where the GIL is available and one where not)

There is a limitation with extending PyType that tp_new cannot be set so only tp_init is available. My workaroud for now is to initialize to Default::default() before calling the user's __init__ function but this doesn't work with multi-level inheritance (currently I catch this case with an assert). I decided not to invest more into this because it might be entirely scrapped in favour of re-purposing #[new] in the case of a metaclass but that's open to discussion. I thought the __init__ approach would at least be simpler to implement for my proof of concept.

[davidhewitt]

Awesome, thank you for driving this forward! If others don't get to this sooner, I will try to make time for this on a Friday in the next few weeks most likely. I have quite a lot of family demands at the moment so it may take a bit of time to find cognitive space to really dig in to this 🙏