diff --git a/docs/assets/mnist_pm_draw.png b/docs/assets/mnist_pm_draw.png new file mode 100644 index 000000000000..1d3e8be6e5e0 Binary files /dev/null and b/docs/assets/mnist_pm_draw.png differ diff --git a/docs/assets/mnist_pm_draw_hover.png b/docs/assets/mnist_pm_draw_hover.png new file mode 100644 index 000000000000..3098f7233801 Binary files /dev/null and b/docs/assets/mnist_pm_draw_hover.png differ diff --git a/docs/assets/small_stream_table.png b/docs/assets/small_stream_table.png new file mode 100644 index 000000000000..da23ae72ac0c Binary files /dev/null and b/docs/assets/small_stream_table.png differ diff --git a/docs/assets/stream_table_from_notebook.png b/docs/assets/stream_table_from_notebook.png new file mode 100644 index 000000000000..c507c63bbf51 Binary files /dev/null and b/docs/assets/stream_table_from_notebook.png differ diff --git a/examples/experimental/ProductionMonitoring/StreamTable.md b/examples/experimental/ProductionMonitoring/StreamTable.md new file mode 100644 index 000000000000..3c3a1b9f58e1 --- /dev/null +++ b/examples/experimental/ProductionMonitoring/StreamTable.md @@ -0,0 +1,88 @@ +# Weave StreamTable + +Log and explore some basic StreamTables now in the [interactive notebook version](../ProductionMontoring/stream_table_api.ipynb) of these docs. + +A Weave StreamTable object enables continuous streaming of data from an application or service to W&B. You can append data repeatedly to the same StreamTable object with `.log([your data rows])` and build dynamic visualizations from the streaming data, like example to recognize MNIST digits hand-drawn in an [interactive Jupyter notebook](../ProductionMonitoring/ProductionMonitoringConceptualOverview.ipynb). + +![small_prodmon_board](../../../docs/assets/mnist_pm_draw_hover.png) + +## Create a StreamTable + +The only required argument to create a StreamTable is the name of the StreamTable object. + +```python +from weave.monitoring import StreamTable +st = StreamTable("my_entity_name/my_project_name/my_table_name") +``` +If an entity (W&B username or shared team name) is not provided, this will attempt to default to the current logged-in entity. + +## Log data to a StreamTable + +Call `.log()` to add rows to a StreamTable: + +```python +st.log({"one_column_name" : "value_a", "another_column_name" : 7}) +st.log([ + {"one_column_name" : "value_b", "another_column_name" : 19}, + {"one_column_name" : "value_c", "another_column_name" : 28}, + {"one_column_name" : "value_d", "another_column_name" : 36}]) +``` +`.log()` accepts a single dictionary or a list of dictionaries, where each dictionary entry corresponds to one row of the table. In each dictionary, the keys are column names and the values are the corresponding cell values. + +## Visualize the StreamTable + +The first call to `.log()` will return a Weave Panel URL, where you can view, edit, and save the resulting StreamTable as a Weave Board, of the form: + +View data at : https://weave.wandb.ai/?exp=get%28%0A++++%22wandb-artifact%3A%2F%2F%2Fstacey%2Fmesa%2Fmy_stream_table%3Alatest%2Fobj%22%29%0A++.rows + +![prodmon_tiny_table](../../../docs/assets/small_stream_table.png) + + +Subsequent log calls will silently append these rows to the StreamTable instance. + +In a notebook, the StreamTable variable on a line by itself will return a Weave Panel view of the StreamTable. The StreamTable will contain all the logged columns and their values, as well as a `timestamp` column indicating when the row was logged. By default, rows will be ordered by oldest first. You can modify a StreamTable Panel from the UI to sort by columns, group by column values, filter for specific ranges or values, etc. + +**Note:** If you would like to customize and save a specific view of a StreamTable Panel, open the StreamTable Panel in a new window as a Board and edit/save a Board from this seed panel. There are two options to achieve this: +* via the weave.wandb.ai/?exp=... URL +* via "Open in new tab" arrow button, revealed in the menu when you hover on the right side of a StreamTable panel displayed in the notebok) + +![stream_table_from_notebook](../../../docs/assets/stream_table_from_notebook.png) + +Continue logging as much data as you like. If you save the StreamTable Panel as a Board, the Board will continue to update as you send more data to the same StreamTable instance. + +## StreamTable API Reference + +### StreamTable() + +Create a StreamTable by providing a table name, with W&B entity (username or team name) and W&B project as prefixes (in the form `entity_name/project_name/table_name`) or separate arguments. + +```python +StreamTable( + table_name: str, + project_name: typing.Optional[str] = None, + entity_name: typing.Optional[str] = None +) +``` + +### .log() + +Append rows to the SteamTable. Each row is a dictionary, and `.log()` accepts a single dictionary or a list of dictionaries. + +```python +st = StreamTable("stream_table") +st.log({"col_A" : 10, "col_B" : "x"}) +st.log([{"col_A" : 20, "col_B" : "y"}, {"col_A" : 30, "col_B" : "z"}]) +``` + +### .rows() + +Add this Weave op to the expression at the top of a Weave Panel to show the contents/actual rows of a StreamTable. Without this op, a StreamTable Panel in the UI will only display the entity, project, and table names (and not the row contents of the StreamTable object). + +### .finish() + +Call `.finish()` to block the user process until all rows and data have been uploaded successfully. This will also wait to display a StreamTable Panel in the notebook UI until all the `.log()` calls have completed (including any downstream processes, e.g. to compute the values of the rows) and have finished writing to the StreamTable instance. + + +## Usage notes + +* **optionally use `.finish()` before viewing the StreamTable**: helpful in cases where you'd like all the rows/logging to complete before viewing the StreamTable. Note that the weave.wandb.ai URL will still show a snapshot of your data at the time it finishes loading — you may need to refresh the page to get all the rows. diff --git a/examples/experimental/ProductionMonitoring/stream_table_api.ipynb b/examples/experimental/ProductionMonitoring/stream_table_api.ipynb new file mode 100644 index 000000000000..2d61a34e2ca3 --- /dev/null +++ b/examples/experimental/ProductionMonitoring/stream_table_api.ipynb @@ -0,0 +1,164 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "eccb44d3", + "metadata": {}, + "source": [ + "# Weave StreamTable Usage\n", + "\n", + "This notebook demonstrates basic Weave StreamTable usage with interactive examples.\n", + "\n", + "## Step 0: Setup\n", + "\n", + "All the StreamTables created in this notebook will be saved to the WB_PROJECT under the WB_ENTITY account on the public W&B cloud. \n", + "\n", + "**Please login to W&B and set your WB_ENTITY** before running this demo." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "51c30e7a", + "metadata": {}, + "outputs": [], + "source": [ + "import weave\n", + "from weave.monitoring import StreamTable" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5684f0b8", + "metadata": {}, + "outputs": [], + "source": [ + "STREAM_TABLE_NAME = \"my_stream_table\"\n", + "WB_PROJECT = \"mesa\"\n", + "WB_ENTITY = " + ] + }, + { + "cell_type": "markdown", + "id": "6f676d96", + "metadata": {}, + "source": [ + "## Step 1: Define a StreamTable\n", + "\n", + "StreamTable has a single required argument: the name of the StreamTable object.\n", + "\n", + "```python\n", + "st = StreamTable(\"stacey/mesa/my_stream_table\")\n", + "```\n", + "\n", + "This takes the form `my_wb_entity/my_wb_project_name/my_stream_table_name` where you can modify the component names to the relevant strings (e.g. your W&B username or shared W&B team name, a new or existing W&B project name).\n", + "\n", + "**Note**: if entity is not provided explicitly, this will attempt to default to the current logged-in entity if one is available." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "25bfa9dc", + "metadata": {}, + "outputs": [], + "source": [ + "st = StreamTable(f\"{WB_ENTITY}/{WB_PROJECT}/{STREAM_TABLE_NAME}\")" + ] + }, + { + "cell_type": "markdown", + "id": "073d25c9", + "metadata": {}, + "source": [ + "## Step 2: Log some data\n", + "\n", + "To add rows to the StreamTable, call `.log()` on the StreamTable object. \n", + "`.log()` accepts a single dictionary or a list of dictionaries, where each dictionary entry corresponds to one row of the table. In each dictionary, the keys are column names and the values are the corresponding cell values.\n", + "\n", + "```python\n", + "st.log({\"one_column_name\" : \"value_a\", \"another_column_name\" : 7})\n", + "st.log([{\"one_column_name\" : \"value_b\", \"another_column_name\" : 19},\n", + " {\"one_column_name\" : \"value_c\", \"another_column_name\" : 28},\n", + " {\"one_column_name\" : \"value_d\", \"another_column_name\" : 36}]\n", + "```\n", + "\n", + "The first call to `.log()` will return a Weave Panel URL, where you can view, edit, and save the resulting StreamTable as a Weave Board, of the form:\n", + "\n", + "View data at: https://weave.wandb.ai/?exp=get%28%0A++++%22wandb-artifact%3A%2F%2F%2Fstacey%2Fmesa%2Fmy_stream_table%3Alatest%2Fobj%22%29%0A++.rows\n", + "\n", + "All log calls on a given StreamTable instance will append the given rows to that instance.\n", + "\n", + "In a notebook, the StreamTable variable on a line by itself will return a Weave Panel view of the StreamTable. The StreamTable will contain all the logged columns and their values, as well as a `timestamp` column indicating when the row was logged. By default, rows will be ordered by oldest first. You can modify a StreamTable Panel from the UI to sort by columns, group by column values, filter for specific ranges or values, etc.\n", + "\n", + "**Note:** If you would like to customize and save a specific view of a StreamTable Panel, open the StreamTable Panel in a new window as a Board and edit/save a Board from this seed panel. There are two options to achieve this:\n", + "* via the weave.wandb.ai/?exp=... URL\n", + "* via \"Open in new tab\" arrow button, revealed in the menu when you hover on the right side of a StreamTable panel displayed in the notebok)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b928c2e0", + "metadata": {}, + "outputs": [], + "source": [ + "# log data to the StreamTable as a dictionary or list of dictionaries\n", + "st.log({\"col_a\" : \"1\", \"col_b\" : \"17\", \"col_c\" : \"42\"})\n", + "\n", + "# show the StreamTable\n", + "st" + ] + }, + { + "cell_type": "markdown", + "id": "83cec916", + "metadata": {}, + "source": [ + "## Step 3: Log more data & explore the results!\n", + "\n", + "Continue logging as much data as you like to any StreamTable instance. You can keep a reference to a given Python StreamTable object in your notebook session or script, and you can reconnect to the same StreamTable instance across multiple sessions/runs of your script via the StreamTable's unique name (e.g. `st = StreamTable(\"stacey/mesa/my_stream_table\")` ) and keep adding rows. Multiple/parallel processes writing to the same StreamTable are also supported—the server will use a queue to order any concurrent messages.\n", + "\n", + "If you save the StreamTable Panel as a Board, the Board will continue to update as you send more data to the same StreamTable instance." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "294d68c0", + "metadata": {}, + "outputs": [], + "source": [ + "st.log({\"col_a\" : 5, \"col_b\" : -24, \"col_c\" : \"hello\"})\n", + "st.log([{\"col_a\" : 255, \"col_b\" : 3.1415926, \"col_c\" : \"hi!\"}])\n", + "\n", + "# optional: wait for all the rows to finish logging before loading\n", + "st.finish()\n", + "\n", + "st" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.7" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}