pipeline: filters: grep: add logical_op property (#1031)

* pipeline: filters: grep: add logical_op property Signed-off-by: Takahiro Yamashita <[email protected]> * pipeline: filters: grep: add legacy mode and default value Signed-off-by: Takahiro Yamashita <[email protected]> * Update pipeline/filters/grep.md Signed-off-by: Pat <[email protected]> --------- Signed-off-by: Takahiro Yamashita <[email protected]> Signed-off-by: Pat <[email protected]> Co-authored-by: Pat <[email protected]>
fluent · Apr 18, 2023 · da943be · da943be
1 parent 8bf0e79
commit da943be
Showing 1 changed file with 44 additions and 0 deletions.
diff --git a/pipeline/filters/grep.md b/pipeline/filters/grep.md
@@ -14,6 +14,7 @@ The plugin supports the following configuration parameters:
 | :--- | :--- | :--- |
 | Regex | KEY  REGEX | Keep records in which the content of KEY matches the regular expression. |
 | Exclude | KEY REGEX | Exclude records in which the content of KEY matches the regular expression. |
+| Logical_Op| Operation | Specify which logical operator to use. `AND` , `OR` and `legacy` are allowed as an Operation. Default is `legacy` for backward compatibility. In `legacy` mode the behaviour is either AND or OR depending whether the `grep` is including (uses AND) or excluding (uses OR). Only available from 2.1+. |
 
 #### Record Accessor Enabled
 
@@ -114,3 +115,46 @@ Here is an example that checks for a specific valid value for the key as well:
 ```
 
 The specified key `iot_timestamp` must match the expected expression - if it does not or is missing/empty then it will be excluded.
+
+### Multiple conditions
+
+If you want to set multiple `Regex` or `Exclude`, you can use `Logical_Op` property to use logical conjuction or disjunction.
+
+Note: If `Logical_Op` is set, setting both 'Regex' and `Exclude` results in an error.
+
+```python
+[INPUT]
+    Name dummy
+    Dummy {"endpoint":"localhost", "value":"something"}
+    Tag dummy
+
+[FILTER]
+    Name grep
+    Match *
+    Logical_Op or
+    Regex value something
+    Regex value error
+
+[OUTPUT]
+    Name stdout
+```
+
+Output will be
+```
+Fluent Bit v2.0.9
+* Copyright (C) 2015-2022 The Fluent Bit Authors
+* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
+* https://fluentbit.io
+
+[2023/01/22 09:46:49] [ info] [fluent bit] version=2.0.9, commit=16eae10786, pid=33268
+[2023/01/22 09:46:49] [ info] [storage] ver=1.2.0, type=memory, sync=normal, checksum=off, max_chunks_up=128
+[2023/01/22 09:46:49] [ info] [cmetrics] version=0.5.8
+[2023/01/22 09:46:49] [ info] [ctraces ] version=0.2.7
+[2023/01/22 09:46:49] [ info] [input:dummy:dummy.0] initializing
+[2023/01/22 09:46:49] [ info] [input:dummy:dummy.0] storage_strategy='memory' (memory only)
+[2023/01/22 09:46:49] [ info] [filter:grep:grep.0] OR mode
+[2023/01/22 09:46:49] [ info] [sp] stream processor started
+[2023/01/22 09:46:49] [ info] [output:stdout:stdout.0] worker #0 started
+[0] dummy: [1674348410.558341857, {"endpoint"=>"localhost", "value"=>"something"}]
+[0] dummy: [1674348411.546425499, {"endpoint"=>"localhost", "value"=>"something"}]
+```