chore(weave): Create a system of defining schema-validated objects

[tssweeney]

This PR allows developers to define a single pydantic type which is then used in 3 key ways:

1. These objects can be constructed in Python, published to weave, and "gotten" from weave - resulting in the correct type - not a WeaveObject wrapper thing.
2. Construction of these objects can be done over the network by setting the set_base_object_class field on the POST request which will enforce server-side validation of these object payloads.
3. The frontend now has fully typed & validated read/write hooks that allows users to build UI experiences that safely read, create, and edit such objects with type safety.

A top level make command make synchronize-base-object-schemas ensures the frontend is up to date!

With all this infrustructure, you simply define your types once, like:

weave/trace_server/interface/base_object_classes/test_only_example.py:

from pydantic import BaseModel

from weave.trace_server.interface.base_object_classes import base_object_def


class TestOnlyNestedBaseModel(BaseModel):
    a: int


class TestOnlyNestedBaseObject(base_object_def.BaseObject):
    b: int


class TestOnlyExample(base_object_def.BaseObject):
    primitive: int
    nested_base_model: TestOnlyNestedBaseModel
    nested_base_object: base_object_def.RefStr

__all__ = ["TestOnlyExample", "TestOnlyNestedBaseObject"]

And you now have:

1. The ability to publish

weave.publish(TestOnlyExample(...))

2. The ability to fetch:

ref = weave.publish(TestOnlyExample(...))
assert isinstance(ref.get(), TestOnlyExample)

3. The ability to create via HTTP

curl -X 'POST' \
  'https://trace.wandb.ai/obj/create' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "obj": {
    "project_id": "user/project",
    "object_id": "my_thing",
    "val": {...},
    "set_base_object_class": "TestOnlyExample" # forces `val` to conform to the schema
  }

4. The ability to query via HTTP

curl -X 'POST' \
  'https://trace.wandb.ai/objs/query' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "project_id": "user/project",
  "filter": {
    "latest_only": true,
    "base_object_classes": ["TestOnlyExample"]
  },
}'

5. Typesafe React Hooks (Read-side)

const res = useBaseObjectInstances("TestOnlyExample", ...)
// res is typed strictly according to the pydantic schema

6. Typesafe React Hooks (Write-side)

const createFn = useCreateBaseObjectInstance("TestOnlyExample")
createFn({...})
// createFn is strictly typed and validates client-side before sending to the server!

See dev_docs/BaseObjectClasses.md for a more complete discussion

[circle-job-mirror]

Preview this PR with FeatureBee: https://beta.wandb.ai/?betaVersion=df0dffa53ec33d5bbe8e0ba1181fa242de13d9a0

[socket-security]

New and removed dependencies detected. Learn more about Socket for GitHub ↗︎

Package	New capabilities	Transitives	Size	Publisher
npm/@glideapps/[email protected]	None	0	38.3 kB	mark.probst
npm/@mark.probst/[email protected]	filesystem, unsafe	+2	68.9 MB	mark.probst
npm/@tsd/[email protected]	None	0	46.5 MB	samverschueren
npm/@types/[email protected]	None	0	165 kB	types
npm/@types/[email protected]	None	0	6.27 kB	types
npm/@types/[email protected]	None	0	5.81 kB	types
npm/[email protected]	None	0	8.95 kB	75lb
npm/[email protected]	None	0	2.34 kB	sindresorhus
npm/[email protected]	None	0	8.35 kB	flexdinesh
npm/[email protected]	None	0	8.86 kB	sindresorhus
npm/[email protected]	None	0	9.74 kB	qix
npm/[email protected]	None	0	48 kB	mark.probst
npm/[email protected]	None	0	117 kB	75lb
npm/[email protected]	None	+2	161 kB	75lb
npm/[email protected]	None	+1	6.55 kB	sindresorhus
npm/[email protected]	environment	0	8.82 kB	sindresorhus
npm/[email protected]	None	0	200 kB	stefanbuck
npm/[email protected]	None	0	12.1 kB	75lb
npm/[email protected]	None	0	5.14 kB	sindresorhus
npm/[email protected]	None	0	24.4 kB	gar
npm/[email protected]	None	0	6.92 kB	sindresorhus
npm/[email protected]	None	0	60.8 kB	leebyron
npm/[email protected]	None	0	38.6 kB	dankogai
npm/[email protected]	None	0	15.2 MB	stefan-terdell
npm/[email protected]	None	0	21.9 kB	jdalton
npm/[email protected]	None	0	9.49 kB	sindresorhus
npm/[email protected]	Transitive: environment, filesystem	+2	251 kB	sindresorhus
npm/[email protected]	None	0	8.53 kB	vdemedes
npm/[email protected]	None	0	27.9 kB	gar
npm/[email protected]	None	0	9.96 kB	unional
npm/[email protected]	None	0	5 kB	sindresorhus
npm/[email protected]	None	0	7.47 kB	sindresorhus
npm/[email protected]	environment, filesystem, network	+2	3.01 MB	dvdsgl
npm/[email protected]	None	0	49.3 kB	dvdsgl
npm/[email protected]	None	0	20.9 kB	dvdsgl
npm/[email protected]	filesystem Transitive: network	+3	5.54 MB	dvdsgl
npm/[email protected]	None	+1	64.7 kB	sindresorhus
npm/[email protected]	filesystem	+3	88.1 kB	sindresorhus
npm/[email protected]	None	0	30.7 kB	bridgear
npm/[email protected]	None	0	23.4 kB	kemitchell
npm/[email protected]	None	0	3.47 kB	kemitchell
npm/[email protected]	None	0	11.8 kB	kemitchell
npm/[email protected]	None	0	13 kB	kemitchell, shinnn
npm/[email protected]	None	0	39 kB	elazutkin
npm/[email protected]	None	0	90.8 kB	elazutkin
npm/[email protected]	None	0	3.63 kB	feross
npm/[email protected]	None	0	6.93 kB	sindresorhus
npm/[email protected]	None	0	62 kB	75lb
npm/[email protected]	None	0	43 kB	devongovett
npm/[email protected]	None	0	3.85 kB	sindresorhus
npm/[email protected]	None	0	94.4 kB	sindresorhus
npm/[email protected]	None	0	66.8 MB	typescript-bot
npm/[email protected]	None	0	22.5 kB	75lb
npm/[email protected]	None	0	76.3 kB	devongovett
npm/[email protected]	None	+1	790 kB	devongovett
npm/[email protected]	None	0	191 kB	rodneyrehm
npm/[email protected]	None	0	16.6 kB	kemitchell
npm/[email protected]	None	0	26.7 kB	75lb

🚮 Removed packages: npm/[email protected]

View full report↗︎

[gtarpenning]

Sweet, excited to see this in action.

[gtarpenning]

these are convenient for testing but if we ever display a registry to the user might be annoying to have to filter these out. I wonder if there is a better way, or have multiple registries, one for tests. punt for now.

[tssweeney]

👍