Skip to content

Commit

Permalink
remove redunt comment and improve readmes
Browse files Browse the repository at this point in the history
  • Loading branch information
@8naama
8naama committed Jul 1, 2024
1 parent 36abab6 commit 7b29fd0
Show file tree
Hide file tree
Showing 5 changed files with 19 additions and 18 deletions.
2 changes: 2 additions & 0 deletions src/apis/azure/README.MD
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ Below fields are relevant for **all Azure API types**
| scrape_interval | Time interval to wait between runs (unit: `minutes`) | Optional | 1 (minute) |

## Azure Graph
By default `azure_graph` API type has built in pagination settings and sets the `response_data_path` to `value` field.
The below fields are relevant **in addition** to the required ones listed under Azure General.

| Parameter Name | Description | Required/Optional | Default |
Expand All @@ -30,6 +31,7 @@ The below fields are relevant **in addition** to the required ones listed under
| data_request.additional_fields | Additional custom fields to add to the logs before sending to logzio | Optional | - |

## Azure Mail Reports
By default `azure_mail_reports` API type has built in pagination settings and sets the `response_data_path` to `d.results` field.
The below fields are relevant **in addition** to the required ones listed under Azure General.

| Parameter Name | Description | Required/Optional | Default |
Expand Down
22 changes: 11 additions & 11 deletions src/apis/cloudflare/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,16 @@
By default `cloudflare` API type has built in pagination settings and sets the `response_data_path` to `result` field.

## Configuration
| Parameter Name | Description | Required/Optional | Default |
|-------------------------|--------------------------------------------------------------------------------------------------------------------------------------------|-------------------|-------------------|
| name | Name of the API (custom name) | Optional | the defined `url` |
| cloudflare_account_id | The CloudFlare Account ID | Required | - |
| cloudflare_bearer_token | The Cloudflare Bearer token | Required | - |
| url | The request URL | Required | - |
| next_url | If needed to update the URL in next requests based on the last response. Supports using variables (see [General API](./general/README.md)) | Optional | - |
| additional_fields | Additional custom fields to add to the logs before sending to logzio | Optional | - |
| scrape_interval | Time interval to wait between runs (unit: `minutes`) | Optional | 1 (minute) |
| pagination_off | True if builtin pagination should be off, False otherwise | Optional | `False` |
| Parameter Name | Description | Required/Optional | Default |
|-------------------------|---------------------------------------------------------------------------------------------------------------------------------------------|-------------------|-------------------|
| name | Name of the API (custom name) | Optional | the defined `url` |
| cloudflare_account_id | The CloudFlare Account ID | Required | - |
| cloudflare_bearer_token | The Cloudflare Bearer token | Required | - |
| url | The request URL | Required | - |
| next_url | If needed to update the URL in next requests based on the last response. Supports using variables (see [General API](../general/README.md)) | Optional | - |
| additional_fields | Additional custom fields to add to the logs before sending to logzio | Optional | - |
| scrape_interval | Time interval to wait between runs (unit: `minutes`) | Optional | 1 (minute) |
| pagination_off | True if builtin pagination should be off, False otherwise | Optional | `False` |

## Example
```Yaml
Expand All @@ -20,7 +20,7 @@ apis:
type: cloudflare
cloudflare_account_id: <<CLOUDFLARE_ACCOUNT_ID>>
cloudflare_bearer_token: <<CLOUDFLARE_BEARER_TOKEN>>
url: https://api.cloudflare.com/client/v4/accounts/{account_id}/alerting/v3/history?since=2024-05-21T04%3A06%3A20.522451Z
url: https://api.cloudflare.com/client/v4/accounts/{account_id}/alerting/v3/history?since=2024-05-21T04:06:20.522451Z
next_url: https://api.cloudflare.com/client/v4/accounts/{account_id}/alerting/v3/history?since={res.result.[0].sent}
scrape_interval: 5
additional_fields:
Expand Down
2 changes: 1 addition & 1 deletion src/apis/general/Api.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -231,7 +231,7 @@ def send_request(self):
# Perform pagination
if self.pagination_settings:
for data in self._perform_pagination(r):
responses.extend(data) # bug need to make it go through the extract data path
responses.extend(data)

# Update the url if needed
if self.next_url:
Expand Down
6 changes: 3 additions & 3 deletions src/apis/general/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -76,11 +76,11 @@ Paths to fields values are structured like so:
- `{res.num_arr.[2]}` = `3`
- `{res.obj_arr.[0].field2}` = `345`

Using the fields values in the `next_url` for example like so:
Using the fields values in the `next_url` for example like the below:
```Yaml
next_url: https://logz.io/{res.field}/{res.obj_arr[0].field2}
```
Would update the URL at every call to have the value of the given fields from the response, in our example the url for the next call would be:
Would update the URL at every call to have the value of the given fields from the response. In our example the url for the next call would be:
```
https://logz.io/value/345
```
Expand All @@ -100,7 +100,7 @@ apis:
}
method: POST
additional_fields:
type: my fetcher
type: my_fetcher
another_field: 123
pagination:
type: url
Expand Down
5 changes: 2 additions & 3 deletions src/config/ConfigReader.py
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
import logging
import sys

from pydantic import ValidationError
import yaml
Expand Down Expand Up @@ -49,9 +48,9 @@ def _read_config(conf_file):
logger.debug(f"Reading config file {conf_file}")
with open(conf_file, "r") as conf:
return yaml.safe_load(conf)
except FileNotFoundError as e:
except FileNotFoundError:
logger.error(f"Did not find file {conf_file}.")
except PermissionError as e:
except PermissionError:
logger.error(f"Missing read permission for file {conf_file}.")
except Exception as e:
logger.error(f"Failed to read config from path {conf_file} due to error {e}.")
Expand Down

0 comments on commit 7b29fd0

Please sign in to comment.