RFC: Add official Jupyter widget with AnyWidget

[jonmmease]

Background

I had a great chat with @manzt at SciPy 2023 about his anywidget project. This approach to building Jupyter widgets is a dramatic simplification compared to what I've done previously (For the plotly FigureWidget and the VegaFusionWidget).

In the announcement blog post, there is actually a proof-of-concept example of creating a Vega-Lite widget that syncs selections back to Python. It's defined by a single Python class, with inline JavaScript:

import anywidget
import traitlets


class ChartWidget(anywidget.AnyWidget):
    _esm = """
    import embed from "https://cdn.jsdelivr.net/npm/vega-embed@6/+esm";

    export async function render({ model, el }) {
        let spec = JSON.parse(model.get("spec"));
        let api = await embed(el, spec);
        api.view.addSignalListener(spec.params[0].name, (_, update) => {
            console.log(update);
            model.set("selection", update);
            model.save_changes();
        });
    }
    """
    spec = traitlets.Unicode().tag(sync=True)
    selection = traitlets.Dict().tag(sync=True)

When loading from a CDN, there's no bundling step at all. For offline use, there are relatively straightforward instructions on bundling.

anywidget is pretty widely compatible with (just about?) every environment that supports Jupyter widgets including the classic notebook, JupyterLab, Binder, Visual Studio Code, and Colab.

Proposal

The anywidget approach looks so straightforward and concise that I'd like to propose we use it to build an official AltairWidget (name TBD, could also be VegaWidget) with the following features:

1. Sync selections back to Python, where a "selection" is a combination of the selection's signal and it's corresponding _store dataset.
2. Support initializing a chart with datasets as binary serialized arrow tables.
3. Support updating arbitrary signals and datasets from Python
4. Support registering callbacks that listen for changes to arbitrary signals and datasets in the live chart

(1) is an end user API to make it possible to use Altair charts in larger dashboards. And by including the *_store dataset in the selection info, I think there's a path to using the selection as an optional filter in chart.transformed_data (cc @joelostblom, as this is something we talked about at some point).

(2) Is an optimization that should significantly improve render time for large, unaggregated charts (or any large chart when the new vegafusion data transformer is not enabled) by avoiding the usual row-wise JSON serialization / deserialization

(3) and (4) are low-level APIs that a future version of VegaFusion could plug into. This would allow me to deprecate VegaFusionWidget (and the entire vegafusion-jupyter package for that matter). We could support something like AltairWidget(chart, vegafusion=True) to optionally enable the current VegaFusionWidget functionality. And there are probably other uses for these capabilities as well.

It works differently, but I think this approach would also largely remove the need for altair_data_server within widget-compatible environments. I think it would also replace the need for the ipyvega VegaWidget

Questions

1. Is this a good idea?
2. If so, would we want to include this in the altair package itself, or in a separate altair-widget package (name TBD)? I'm on the fence here, and my preference would likely depend on how much code it ends up being, and on whether it could be added to the main repo without introducing any additional required dependencies.
3. As an alternative to manually wrapping an Altair Chart in an AltairWidget, could we also have a renderer that would automatically do this? This way you would get the performance benefit without API changes (for the common case where you don't need access to the selection info from Python).

cc @domoritz, as this is basically the proposal we talked about a few months back. It just feels like a much smaller lift with anywidget.

[domoritz]

I’m super supportive. One thing that’s missing from the proposal to me is support for sending data separately from the spec (e.g., pandas dataframes). Ideally we go through some good serialization.

I’d also be open to have this happen in the ipyvega repo (the Vega pip package). That would keep Altair focused on being the API only and the widget could work without Altair as well.

[jonmmease]

One thing that’s missing from the proposal to me is support for sending data separately from the spec (e.g., pandas dataframes). Ideally we go through some good serialization.

Yeah, I was picturing sending the tables as Arrow and then deserializing with the Arrow JS library, then splicing the Arrow tables back into the spec before calling vega-embed.

I’d also be open to have this happen in the ipyvega repo (the Vega pip package). That would keep Altair focused on being the API only and the widget could work without Altair as well.

Thanks, that does sounds like a good idea

[binste]

1. Is this a good idea?

Absolutely, I think it's a great idea for the reasons you outlined in your proposal 🚀 Thank you for putting it together!

2. If so, would we want to include this in the altair package itself, or in a separate altair-widget package (name TBD)? I'm on the fence here, and my preference would likely depend on how much code it ends up being, and on whether it could be added to the main repo without introducing any additional required dependencies.

Including it in the ipyvega repo sounds good to me as well for the outlined reasons. Altair can focus on being the API and the widget can be used independently of Altair.

3. As an alternative to manually wrapping an Altair Chart in an AltairWidget, could we also have a renderer that would automatically do this? This way you would get the performance benefit without API changes (for the common case where you don't need access to the selection info from Python).

Sounds like a very useful addition in a second step so +1 for this as well.

[joelostblom]

This sounds interesting and powerful although I'll admit that I don't fully grasp all the ins and outs of it.

(1) is an end user API to make it possible to use Altair charts in larger dashboards

Does this mean that if I create a dashboard using Panel or Dash, I would be able to use AltairWidget/ipyvega to create widgets that play nicely with the rest of those dashboard components? Or would I only be able to use this new widget package in a JupyterLab-compatible environment like those that you listed above (+maybe dashboarding solutions building on JupyterLab, like Voila etc?).

This way you would get the performance benefit without API changes

This sounds attractive and could maybe follow a similar approach as we are taking with vl-convert and vegafusion that if the AltairWidget/ipyvega package is installed Altair will use it by default but Altair still works perfectly fine without it.

I’d also be open to have this happen in the ipyvega repo (the Vega pip package). That would keep Altair focused on being the API only and the widget could work without Altair as well.

Does ipyvega work well with Altair 5 or would there be additional updates needed there since the package is not required for JupyterLab and we added the Altair 5 compatibility directly into the JupyterLab repo? I haven't really used ipyvega at all so I don't know if it would introduced any additional useful features (e.g. the performance imporovements here vega/ipyvega#346), or if there is a lot of functionality that it is not needed for Altair when working in more modern environment than the classic notebooks and it would maybe be more agile to make what is proposed here into a new package? FWIW, there was an attempt at a previous altair-widget package here https://github.com/altair-viz/altair_widgets

[mattijn]

Thanks @jonmmease for starting this discussion! Thanks @manzt for developing anywidget!

I saw the poc-example in this jupyter blog post, where in the same blog post is also nicely discussed why this Python <> JavaScript interchange is (was?) very difficult.

I'm not yet completely convinced why this widget is better served from within ipyvega.
Altair contains (1) a python API to Vega-Lite, it has (2) data serialization logic (both into the chart and since recently also out of the chart) and it has (3) visualization logic building on top of JavaScript.

In my understanding this proposal would also mean an almost complete redo of ipyvega (?) including both (2) and (3) of above.

I've to admit, I cannot oversee to what extend the intended code-changes can explode, but if the code changes are a fraction of the current implementation of the ipyvega and vegafusion-jupyter package and the result is still pip-installable (no compiled code), then I'm not exactly sure why altair is not a good candidate for this.

[jonmmease]

Thanks for the feedback everyone! Sounds like the next step is for me to actually play with this a bit more and demonstrate how much code is actually involved.

I've started extending the poc-example from the blog post to support multiple selections, and support updating arbitrary signals/datasets from Python, and it seems to be going smoothly. I'll keep you all posted on my progress.

Another nice benefit of this approach is that if we use bundling then we'll have a way to display charts in widget-compatible environments that works offline (unlike the "default" renderer) and where we control the versions of Vega and Vega-Lite (unlike the "mimetype") renderer.

[jonmmease]

I made a quick PR in #3108 that demonstrates how we could add a simple ChartWidget to the Altair repo/package. The widget implementation is copied from the AnyWidget blog post for now, but if we like the general approach, I'll work on adding the functionality discussed above.

[jonmmease]

Does this mean that if I create a dashboard using Panel or Dash, I would be able to use AltairWidget/ipyvega to create widgets that play nicely with the rest of those dashboard components? Or would I only be able to use this new widget package in a JupyterLab-compatible environment like those that you listed above (+maybe dashboarding solutions building on JupyterLab, like Voila etc?).

This would only work in environments that explicitly support Jupyter widgets. So not in Dash. Panel has some support through ipywidgets_bokeh, but there are a fair number of open issues there (e.g. VegaFusionWidget doesn't work yet), so I'm not sure if this will work. Solara seems to support widgets well, so I expect that would be an option for dashboards.

[mattijn]

Done by #3119