Skip to content

Commit

Permalink
Merge pull request #1484 from fluent/lynettemiles/sc-105696/update-co…
Browse files Browse the repository at this point in the history
…ncepts-data-pipeline-buffer-md-fluent
  • Loading branch information
esmerel authored Oct 24, 2024
2 parents 467344e + fbe454d commit 1de2cbe
Showing 1 changed file with 8 additions and 6 deletions.
14 changes: 8 additions & 6 deletions concepts/data-pipeline/buffer.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,12 @@ description: Data processing with reliability

# Buffer

Previously defined in the [Buffering](../buffering.md) concept section, the `buffer` phase in the pipeline aims to provide a unified and persistent mechanism to store your data, either using the primary in-memory model or using the filesystem based mode.
The [`buffer`](../buffering.md) phase in the pipeline aims to provide a unified and
persistent mechanism to store your data, using the primary in-memory model or the
file system-based mode.

The `buffer` phase already contains the data in an immutable state, meaning that no other filter can be applied.
The `buffer` phase contains the data in an immutable state, meaning that no other
filter can be applied.

```mermaid
graph LR
Expand All @@ -22,8 +25,7 @@ graph LR
style D stroke:darkred,stroke-width:2px;
```

{% hint style="info" %}
Note that buffered data is not raw text, it's in Fluent Bit's internal binary representation.
{% endhint %}
Buffered data uses the Fluent Bit internal binary representation, which isn't raw text.

Fluent Bit offers a buffering mechanism in the file system that acts as a _backup system_ to avoid data loss in case of system failures.
Fluent Bit offers a buffering mechanism in the file system that acts as a backup
system to avoid data loss in case of system failures.

0 comments on commit 1de2cbe

Please sign in to comment.