Skip to content

Commit

Permalink
Update documentation (#696)
Browse files Browse the repository at this point in the history 
Co-authored-by: quentin-quix <[email protected]>
  • Loading branch information
@github-actions @quentin-quix
github-actions[bot] and quentin-quix authored Dec 20, 2024
1 parent 6509238 commit 7d67658
Show file tree
Hide file tree
Showing 3 changed files with 56 additions and 52 deletions.
2 changes: 1 addition & 1 deletion docs/api-reference/context.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ sdf = sdf.update(lambda value: alter_context(value))
#### message\_context

```python
def message_context() -> Optional[MessageContext]
def message_context() -> MessageContext
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/context.py#L53)
Expand Down
36 changes: 19 additions & 17 deletions docs/api-reference/dataframe.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
class StreamingDataFrame(BaseStreaming)
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L68)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L71)

`StreamingDataFrame` is the main object you will use for ETL work.

Expand Down Expand Up @@ -71,8 +71,10 @@ sdf = sdf.to_topic(topic_obj)
```python
def apply(func: Union[
ApplyCallback,
ApplyExpandedCallback,
ApplyCallbackStateful,
ApplyWithMetadataCallback,
ApplyWithMetadataExpandedCallback,
ApplyWithMetadataCallbackStateful,
],
*,
Expand All @@ -81,7 +83,7 @@ def apply(func: Union[
metadata: bool = False) -> Self
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L174)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L186)

Apply a function to transform the value and return a new value.

Expand Down Expand Up @@ -139,7 +141,7 @@ def update(func: Union[
metadata: bool = False) -> Self
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L263)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L293)

Apply a function to mutate value in-place or to perform a side effect

Expand Down Expand Up @@ -207,7 +209,7 @@ def filter(func: Union[
metadata: bool = False) -> Self
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L355)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L399)

Filter value using provided function.

Expand Down Expand Up @@ -259,7 +261,7 @@ def group_by(key: Union[str, Callable[[Any], Any]],
key_serializer: Optional[SerializerType] = "json") -> Self
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L441)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L487)

"Groups" messages by re-keying them via the provided group_by operation

Expand Down Expand Up @@ -323,7 +325,7 @@ a clone with this operation added (assign to keep its effect).
def contains(key: str) -> StreamingSeries
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L514)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L565)

Check if the key is present in the Row value.

Expand Down Expand Up @@ -362,7 +364,7 @@ or False otherwise.
def to_topic(topic: Topic, key: Optional[Callable[[Any], Any]] = None) -> Self
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L539)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L590)

Produce current value to a topic. You can optionally specify a new key.

Expand Down Expand Up @@ -415,7 +417,7 @@ the updated StreamingDataFrame instance (reassignment NOT required).
def set_timestamp(func: Callable[[Any, Any, int, Any], int]) -> Self
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L584)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L635)

Set a new timestamp based on the current message value and its metadata.

Expand Down Expand Up @@ -467,7 +469,7 @@ def set_headers(
]) -> Self
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L625)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L676)

Set new message headers based on the current message value and metadata.

Expand Down Expand Up @@ -516,7 +518,7 @@ a new StreamingDataFrame instance
def print(pretty: bool = True, metadata: bool = False) -> Self
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L676)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L727)

Print out the current message value (and optionally, the message metadata) to

Expand Down Expand Up @@ -572,7 +574,7 @@ def compose(
) -> Dict[str, VoidExecutor]
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L718)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L771)

Compose all functions of this StreamingDataFrame into one big closure.

Expand Down Expand Up @@ -626,7 +628,7 @@ def test(value: Any,
topic: Optional[Topic] = None) -> List[Any]
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L752)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L805)

A shorthand to test `StreamingDataFrame` with provided value

Expand Down Expand Up @@ -663,7 +665,7 @@ def tumbling_window(duration_ms: Union[int, timedelta],
name: Optional[str] = None) -> TumblingWindowDefinition
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L789)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L842)

Create a tumbling window transformation on this StreamingDataFrame.

Expand Down Expand Up @@ -749,7 +751,7 @@ def hopping_window(duration_ms: Union[int, timedelta],
name: Optional[str] = None) -> HoppingWindowDefinition
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L865)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L918)

Create a hopping window transformation on this StreamingDataFrame.

Expand Down Expand Up @@ -843,7 +845,7 @@ def sliding_window(duration_ms: Union[int, timedelta],
name: Optional[str] = None) -> SlidingWindowDefinition
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L957)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L1010)

Create a sliding window transformation on this StreamingDataFrame.

Expand Down Expand Up @@ -932,7 +934,7 @@ def drop(columns: Union[str, List[str]],
errors: Literal["ignore", "raise"] = "raise") -> Self
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L1038)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L1091)

Drop column(s) from the message value (value must support `del`, like a dict).

Expand Down Expand Up @@ -976,7 +978,7 @@ a new StreamingDataFrame instance
def sink(sink: BaseSink)
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L1082)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L1135)

Sink the processed data to the specified destination.

Expand Down
Loading

0 comments on commit 7d67658

Please sign in to comment.