Skip to content

Commit

Permalink
Update documentation (#668)
Browse files Browse the repository at this point in the history 
Co-authored-by: quentin-quix <[email protected]>
  • Loading branch information
@github-actions @quentin-quix
github-actions[bot] and quentin-quix authored Dec 4, 2024
1 parent d1bc156 commit bfab5d1
Show file tree
Hide file tree
Showing 3 changed files with 171 additions and 200 deletions.
43 changes: 21 additions & 22 deletions docs/api-reference/dataframe.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
class StreamingDataFrame(BaseStreaming)
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L69)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L68)

`StreamingDataFrame` is the main object you will use for ETL work.

Expand Down Expand Up @@ -81,7 +81,7 @@ def apply(func: Union[
metadata: bool = False) -> Self
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L175)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L174)

Apply a function to transform the value and return a new value.

Expand Down Expand Up @@ -139,7 +139,7 @@ def update(func: Union[
metadata: bool = False) -> Self
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L264)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L263)

Apply a function to mutate value in-place or to perform a side effect

Expand Down Expand Up @@ -207,7 +207,7 @@ def filter(func: Union[
metadata: bool = False) -> Self
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L356)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L355)

Filter value using provided function.

Expand Down Expand Up @@ -259,7 +259,7 @@ def group_by(key: Union[str, Callable[[Any], Any]],
key_serializer: Optional[SerializerType] = "json") -> Self
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L442)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L441)

"Groups" messages by re-keying them via the provided group_by operation

Expand Down Expand Up @@ -323,7 +323,7 @@ a clone with this operation added (assign to keep its effect).
def contains(key: str) -> StreamingSeries
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L515)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L514)

Check if the key is present in the Row value.

Expand Down Expand Up @@ -362,7 +362,7 @@ or False otherwise.
def to_topic(topic: Topic, key: Optional[Callable[[Any], Any]] = None) -> Self
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L540)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L539)

Produce current value to a topic. You can optionally specify a new key.

Expand Down Expand Up @@ -415,7 +415,7 @@ the updated StreamingDataFrame instance (reassignment NOT required).
def set_timestamp(func: Callable[[Any, Any, int, Any], int]) -> Self
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L585)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L584)

Set a new timestamp based on the current message value and its metadata.

Expand Down Expand Up @@ -461,14 +461,13 @@ a new StreamingDataFrame instance

```python
def set_headers(
func: Callable[
[Any, Any, int, List[Tuple[str, HeaderValue]]],
Collection[Tuple[str, HeaderValue]],
]
) -> Self
func: Callable[
[Any, Any, int, HeadersTuples],
HeadersTuples,
]) -> Self
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L626)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L625)

Set new message headers based on the current message value and metadata.

Expand Down Expand Up @@ -517,7 +516,7 @@ a new StreamingDataFrame instance
def print(pretty: bool = True, metadata: bool = False) -> Self
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L677)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L676)

Print out the current message value (and optionally, the message metadata) to

Expand Down Expand Up @@ -573,7 +572,7 @@ def compose(
) -> Dict[str, VoidExecutor]
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L719)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L718)

Compose all functions of this StreamingDataFrame into one big closure.

Expand Down Expand Up @@ -627,7 +626,7 @@ def test(value: Any,
topic: Optional[Topic] = None) -> List[Any]
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L753)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L752)

A shorthand to test `StreamingDataFrame` with provided value

Expand Down Expand Up @@ -664,7 +663,7 @@ def tumbling_window(duration_ms: Union[int, timedelta],
name: Optional[str] = None) -> TumblingWindowDefinition
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L790)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L789)

Create a tumbling window transformation on this StreamingDataFrame.

Expand Down Expand Up @@ -750,7 +749,7 @@ def hopping_window(duration_ms: Union[int, timedelta],
name: Optional[str] = None) -> HoppingWindowDefinition
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L866)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L865)

Create a hopping window transformation on this StreamingDataFrame.

Expand Down Expand Up @@ -844,7 +843,7 @@ def sliding_window(duration_ms: Union[int, timedelta],
name: Optional[str] = None) -> SlidingWindowDefinition
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L958)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L957)

Create a sliding window transformation on this StreamingDataFrame.

Expand Down Expand Up @@ -933,7 +932,7 @@ def drop(columns: Union[str, List[str]],
errors: Literal["ignore", "raise"] = "raise") -> Self
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L1039)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L1038)

Drop column(s) from the message value (value must support `del`, like a dict).

Expand Down Expand Up @@ -977,7 +976,7 @@ a new StreamingDataFrame instance
def sink(sink: BaseSink)
```

[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L1083)
[[VIEW SOURCE]](https://github.com/quixio/quix-streams/blob/main/quixstreams/dataframe/dataframe.py#L1082)

Sink the processed data to the specified destination.

Expand Down
Loading

0 comments on commit bfab5d1

Please sign in to comment.