Skip to content

Commit

Permalink
[DOCS] Changing docs for UDF (#2880)
Browse files Browse the repository at this point in the history
1. Improves UDF documentation by adding API pages for `StatefulUDF` and
`StatelessUDF`, with lots of docstrings and examples
~2. Moves our `daft.udf` module to `daft.udfs` instead, which avoids a
naming conflict with `daft.udf` which is our decorator. This might be a
breaking change...~ (Reverted)

---------

Co-authored-by: Jay Chia <[email protected]@users.noreply.github.com>
  • Loading branch information
jaychia and Jay Chia authored Nov 8, 2024
1 parent 95168fd commit 990149a
Show file tree
Hide file tree
Showing 6 changed files with 272 additions and 71 deletions.
3 changes: 3 additions & 0 deletions daft/dataframe/dataframe.py
Original file line number Diff line number Diff line change
Expand Up @@ -3083,10 +3083,13 @@ def map_groups(self, udf: Expression) -> "DataFrame":
Example:
>>> import daft, statistics
>>>
>>> df = daft.from_pydict({"group": ["a", "a", "a", "b", "b", "b"], "data": [1, 20, 30, 4, 50, 600]})
>>>
>>> @daft.udf(return_dtype=daft.DataType.float64())
... def std_dev(data):
... return [statistics.stdev(data.to_pylist())]
>>>
>>> df = df.groupby("group").map_groups(std_dev(df["data"]))
>>> df.show()
╭───────┬────────────────────╮
Expand Down
8 changes: 5 additions & 3 deletions daft/expressions/expressions.py
Original file line number Diff line number Diff line change
Expand Up @@ -1001,7 +1001,7 @@ def apply(self, func: Callable, return_dtype: DataType) -> Expression:
Returns:
Expression: New expression after having run the function on the expression
"""
from daft.udf import StatelessUDF
from daft.udf import CommonUDFArgs, StatelessUDF

def batch_func(self_series):
return [func(x) for x in self_series.to_pylist()]
Expand All @@ -1015,8 +1015,10 @@ def batch_func(self_series):
name=name,
func=batch_func,
return_dtype=return_dtype,
resource_request=None,
batch_size=None,
common_args=CommonUDFArgs(
resource_request=None,
batch_size=None,
),
)(self)

def is_null(self) -> Expression:
Expand Down
Loading

0 comments on commit 990149a

Please sign in to comment.