diff --git a/docs/_include/api-cookbook/replace-vars.html b/docs/_include/api-cookbook/replace-vars.html index 3737a621..9ad0c748 100644 --- a/docs/_include/api-cookbook/replace-vars.html +++ b/docs/_include/api-cookbook/replace-vars.html @@ -28,4 +28,4 @@ * Replace `<>` with an [API token](https://app.logz.io/#/dashboard/settings/manage-tokens/api) from the account you want to use * Replace `<>` with your region's base API URL. - For more information on finding your account's region, see [Account region]({{site.baseurl}}/user-guide/accounts/account-region.html). \ No newline at end of file + For more information on finding your account's region, see [Account region](/docs/user-guide/admin/hosting-regions/account-region). \ No newline at end of file diff --git a/docs/_include/general-shipping/k8s.md b/docs/_include/general-shipping/k8s.md new file mode 100644 index 00000000..31de34b3 --- /dev/null +++ b/docs/_include/general-shipping/k8s.md @@ -0,0 +1,243 @@ + + + + +The logzio-monitoring Helm Chart ships your Kubernetes telemetry (logs, metrics, traces and security reports) to your Logz.io account. + +## Prerequisites + +1. [Helm](https://helm.sh/) + +Add Logzio-helm repository + +```sh +helm repo add logzio-helm https://logzio.github.io/logzio-helm && helm repo update +``` +{@include: ../../_include/general-shipping/k8s-all-data.md} + +## Send your logs + +```sh +helm install -n monitoring \ +--set logs.enabled=true \ +--set logzio-fluentd.secrets.logzioShippingToken="<>" \ +--set logzio-fluentd.secrets.logzioListener="<>" \ +--set logzio-fluentd.env_id="<>" \ +logzio-monitoring logzio-helm/logzio-monitoring +``` + + +| Parameter | Description | +| --- | --- | +| `<>` | Your [logs shipping token](https://app.logz.io/#/dashboard/settings/general). | +| `<>` | Your account's [listener host](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping?product=logs). | +| `<>` | The cluster's name, to easily identify the telemetry data for each environment. | + + +For log shipping troubleshooting, see our [user guide](https://docs.logz.io/user-guide/kubernetes-troubleshooting/). + +## Send your deploy events logs + +This integration sends data about deployment events in the cluster, and how they affect the cluster's resources. +Currently supported resource kinds are `Deployment`, `Daemonset`, `Statefulset`, `ConfigMap`, `Secret`, `Service Account`, `Cluster Role` and `Cluster Role Binding`. + +```sh +helm install --namespace=monitoring \ +--set logzio-k8s-events.secrets.logzioShippingToken='<>' \ +--set logzio-k8s-events.secrets.logzioListener='<>' \ +--set logzio-k8s-events.secrets.env_id='<>' \ +--set logzio-k8s-events.secrets.customListener='<>' \ +logzio-monitoring logzio-helm/logzio-monitoring + +``` + + +| Parameter | Description | +| --- | --- | +| `<>` | Your [logs shipping token](https://app.logz.io/#/dashboard/settings/general). | +| `<>` | Your account's [listener host](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping?product=logs). | +| `<>` | The cluster's name, to easily identify the telemetry data for each environment. | +| `<>` | (*optional*) HTTP/s listener endpoint that receives JSON input, overrides the Logz.io listener. | + +### Deployment events versioning + +In order to add an indication for the versioning in our K8S 360 and Service Overview UI, the following annotation should be added to the metadata of each resource you'd like to track its versioning. +Commit URL structure: `https://github.com///commit/` + +Example: `https://github.com/logzio/logzio-k8s-events/commit/069c75c95caeca58dd0776405bb8dfb4eed3acb2` + +```yaml +metadata: + annotations: + logzio/commit_url: "" +``` + + +For log shipping troubleshooting, see our [user guide](https://docs.logz.io/user-guide/kubernetes-troubleshooting/). + +## Send your metrics + +```sh +helm install -n monitoring \ +--set metricsOrTraces.enabled=true \ +--set logzio-k8s-telemetry.metrics.enabled=true \ +--set logzio-k8s-telemetry.secrets.MetricsToken="<>" \ +--set logzio-k8s-telemetry.secrets.ListenerHost="https://<>:8053" \ +--set logzio-k8s-telemetry.secrets.p8s_logzio_name="<>" \ +--set logzio-k8s-telemetry.secrets.env_id="<>" \ +logzio-monitoring logzio-helm/logzio-monitoring +``` + +| Parameter | Description | +| --- | --- | +| `<>` | Your [metrics shipping token](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping). | +| `<>` | The cluster's name, to easily identify the telemetry data for each environment. | +| `<>` | Your account's [listener host](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping?product=logs). | + + +For metrics shipping troubleshooting, see our [user guide](https://docs.logz.io/user-guide/infrastructure-monitoring/troubleshooting/k8-helm-opentelemetry-troubleshooting.html). + + + +## Send your traces + +```sh +helm install -n monitoring \ +--set metricsOrTraces.enabled=true \ +--set logzio-k8s-telemetry.traces.enabled=true \ +--set logzio-k8s-telemetry.secrets.TracesToken="<>" \ +--set logzio-k8s-telemetry.secrets.LogzioRegion="<>" \ +--set logzio-k8s-telemetry.secrets.env_id="<>" \ +logzio-monitoring logzio-helm/logzio-monitoring +``` + +| Parameter | Description | +| --- | --- | +| `<>` | Your [traces shipping token](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping?product=tracing). | +| `<>` | The cluster's name, to easily identify the telemetry data for each environment. | +| `<>` | Name of your Logz.io traces region e.g `us`, `eu`... | + + +For traces shipping troubleshooting, see our [user guide]([https://docs.logz.io/user-guide/kubernetes-troubleshooting/](https://docs.logz.io/user-guide/distributed-tracing/tracing-troubleshooting.html)). + + +## Send traces with SPM + +```sh +helm install -n monitoring \ +--set metricsOrTraces.enabled=true \ +--set logzio-k8s-telemetry.traces.enabled=true \ +--set logzio-k8s-telemetry.secrets.TracesToken="<>" \ +--set logzio-k8s-telemetry.secrets.LogzioRegion="<>" \ +--set logzio-k8s-telemetry.secrets.env_id="<>" \ +--set logzio-k8s-telemetry.spm.enabled=true \ +--set logzio-k8s-telemetry.secrets.SpmToken=<> \ +logzio-monitoring logzio-helm/logzio-monitoring +``` + +| Parameter | Description | +| --- | --- | +| `<>` | Your [traces shipping token](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping). | +| `<>` | The cluster's name, to easily identify the telemetry data for each environment. | +| `<>` | Your account's [listener host](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping?product=logs). | +| `<>` | Name of your Logz.io traces region e.g `us`, `eu`... | +| `<>` | Your [span metrics shipping token](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping). | + + +## Scan your cluster for security vulnerabilities + +```sh +helm install -n monitoring \ +--set securityReport.enabled=true \ +--set logzio-trivy.env_id="<>" \ +--set logzio-trivy.secrets.logzioShippingToken="<>" \ +--set logzio-trivy.secrets.logzioListener="<>" \ +``` + +| Parameter | Description | +| --- | --- | +| `<>` | Your [logs shipping token](https://app.logz.io/#/dashboard/settings/general). | +| `<>` | Your account's [listener host](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping?product=logs). | +| `<>` | The cluster's name, to easily identify the telemetry data for each environment. | + + +## Modifying the configuration for logs + +You can see a full list of the possible configuration values in the [logzio-fluentd Chart folder](https://github.com/logzio/logzio-helm/tree/master/charts/fluentd#configuration). + +If you would like to modify any of the values found in the `logzio-fluentd` folder, use the `--set` flag with the `logzio-fluentd` prefix. + +For instance, if there is a parameter called `someField` in the `logzio-telemetry`'s `values.yaml` file, you can set it by adding the following to the `helm install` command: + +```sh +--set logzio-fluentd.someField="my new value" +``` +You can add `log_type` annotation with a custom value, which will be parsed into a `log_type` field with the same value. + + +### Modifying the configuration for metrics and traces + +You can see a full list of the possible configuration values in the [logzio-telemetry Chart folder](https://github.com/logzio/logzio-helm/tree/master/charts/logzio-telemetry). + +If you would like to modify any of the values found in the `logzio-telemetry` folder, use the `--set` flag with the `logzio-k8s-telemetry` prefix. + +For instance, if there is a parameter called `someField` in the `logzio-telemetry`'s `values.yaml` file, you can set it by adding the following to the `helm install` command: + + +```sh +--set logzio-k8s-telemetry.someField="my new value" +``` + +## Sending telemetry data from eks on fargate + +To ship logs from pods running on Fargate, set the `fargateLogRouter.enabled` value to `true`. Doing so will deploy a dedicated `aws-observability` namespace and a `configmap` for the Fargate log router. For more information on EKS Fargate logging, please refer to the [official AWS documentation]((https://docs.aws.amazon.com/eks/latest/userguide/fargate-logging.html). + +```shell +helm install -n monitoring \ +--set logs.enabled=true \ +--set logzio-fluentd.fargateLogRouter.enabled=true \ +--set logzio-fluentd.secrets.logzioShippingToken="<>" \ +--set logzio-fluentd.secrets.logzioListener="<>" \ +--set metricsOrTraces.enabled=true \ +--set logzio-k8s-telemetry.metrics.enabled=true \ +--set logzio-k8s-telemetry.secrets.MetricsToken="<>" \ +--set logzio-k8s-telemetry.secrets.ListenerHost="https://<>:8053" \ +--set logzio-k8s-telemetry.secrets.p8s_logzio_name="<>" \ +--set logzio-k8s-telemetry.traces.enabled=true \ +--set logzio-k8s-telemetry.secrets.TracesToken="<>" \ +--set logzio-k8s-telemetry.secrets.LogzioRegion="<>" \ +logzio-monitoring logzio-helm/logzio-monitoring +``` + +| Parameter | Description | +| --- | --- | +| `<>` | Your [logs shipping token](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping?product=logs). | +| `<>` | Your account's [listener host](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping?product=logs). | +| `<>` | Your [metrics shipping token](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping?product=metrics). | +| `<>` | The name for the environment's metrics, to easily identify the metrics for each environment. | +| `<>` | The name for your environment's identifier, to easily identify the telemetry data for each environment. | +| `<>` | Your custom name for the environment's metrics, to easily identify the metrics for each environment. | +| `<>` | Replace `<>` with the [token](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping?product=tracing) of the account you want to ship to. | +| `<>` | Name of your Logz.io traces region e.g `us` or `eu`. You can find your region code in the [Regions and URLs](https://docs.logz.io/user-guide/accounts/account-region.html#regions-and-urls) table. | + +## Handling image pull rate limit + +In certain situations, such as with spot clusters where pods/nodes are frequently replaced, you may encounter the pull rate limit for images fetched from Docker Hub. This could result in the following error: `You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limits`. + +To address this issue, you can use the `--set` commands provided below in order to access an alternative image repository: + +```shell +--set logzio-k8s-telemetry.image.repository=ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-contrib +--set logzio-k8s-telemetry.prometheus-pushgateway.image.repository=public.ecr.aws/logzio/prom-pushgateway +--set logzio-fluentd.image=public.ecr.aws/logzio/logzio-fluentd +--set logzio-fluentd.daemonset.init.containerImage=public.ecr.aws/docker/library/busybox +--set logzio-trivy.image=public.ecr.aws/logzio/trivy-to-logzio +``` + +## Upgrade logzio-monitoring to v3.0.0 + +Before upgrading your logzio-monitoring Chart to v3.0.0 with `helm upgrade`, note that you may encounter an error for some of the logzio-telemetry sub-charts. + +There are two possible approaches to the upgrade you can choose from: +- Reinstall the chart. +- Before running the `helm upgrade` command, delete the old subcharts resources: `logzio-monitoring-prometheus-pushgateway` deployment and the `logzio-monitoring-prometheus-node-exporter` daemonset. diff --git a/docs/shipping/AWS/1aws-native.md b/docs/shipping/AWS/1aws-native.md index 99483c7d..4780dafa 100644 --- a/docs/shipping/AWS/1aws-native.md +++ b/docs/shipping/AWS/1aws-native.md @@ -84,7 +84,7 @@ If you've used the `services` field, you'll have to **wait 6 minutes** before cr Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ## Metrics diff --git a/docs/shipping/AWS/aws-amplify.md b/docs/shipping/AWS/aws-amplify.md index d62c3e55..a844660e 100644 --- a/docs/shipping/AWS/aws-amplify.md +++ b/docs/shipping/AWS/aws-amplify.md @@ -90,7 +90,7 @@ If you've used the `services` field, you'll have to **wait 6 minutes** before cr Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). @@ -192,4 +192,4 @@ git clone https://github.com/logzio/logzio_aws_serverless.git \ Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting](https://docs.logz.io/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/AWS/aws-api-gateway.md b/docs/shipping/AWS/aws-api-gateway.md index 9efa6884..467d29e1 100644 --- a/docs/shipping/AWS/aws-api-gateway.md +++ b/docs/shipping/AWS/aws-api-gateway.md @@ -89,7 +89,7 @@ If you've used the `services` field, you'll have to **wait 6 minutes** before cr Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ## Metrics diff --git a/docs/shipping/AWS/aws-app-elb.md b/docs/shipping/AWS/aws-app-elb.md index 604f64f7..8cee2ca7 100644 --- a/docs/shipping/AWS/aws-app-elb.md +++ b/docs/shipping/AWS/aws-app-elb.md @@ -23,7 +23,7 @@ When you set Logz.io to fetch Elastic Load Balancing (ELB) logs, Logz.io will pe **Before you begin, you'll need**: -`s3:ListBucket` and `s3:GetObject` [permissions](https://docs.logz.io/user-guide/give-aws-access-with-iam-roles/) for the required S3 bucket (one bucket per region) +`s3:ListBucket` and `s3:GetObject` [permissions](/docs/user-guide/admin/give-aws-access-with-iam-roles) for the required S3 bucket (one bucket per region) @@ -49,7 +49,7 @@ Log into the app to use the dedicated Logz.io [configuration wizard](https://app Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ## Configure AWS to forward metrics to Logz.io diff --git a/docs/shipping/AWS/aws-athena.md b/docs/shipping/AWS/aws-athena.md index 73ac5c77..60d245fc 100644 --- a/docs/shipping/AWS/aws-athena.md +++ b/docs/shipping/AWS/aws-athena.md @@ -84,7 +84,7 @@ If you've used the `services` field, you'll have to **wait 6 minutes** before cr Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ## Metrics diff --git a/docs/shipping/AWS/aws-classic-elb.md b/docs/shipping/AWS/aws-classic-elb.md index 259b4370..68f4138e 100644 --- a/docs/shipping/AWS/aws-classic-elb.md +++ b/docs/shipping/AWS/aws-classic-elb.md @@ -29,7 +29,7 @@ When you set Logz.io to fetch Elastic Load Balancing (ELB) logs, Logz.io will pe **Before you begin, you'll need**: -`s3:ListBucket` and `s3:GetObject` [permissions](https://docs.logz.io/user-guide/give-aws-access-with-iam-roles/) for the required S3 bucket (one bucket per region) +`s3:ListBucket` and `s3:GetObject` [permissions](/docs/user-guide/admin/give-aws-access-with-iam-roles) for the required S3 bucket (one bucket per region) @@ -55,7 +55,7 @@ Log into the app to use the dedicated Logz.io [configuration wizard](https://app Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/AWS/aws-cloudfront.md b/docs/shipping/AWS/aws-cloudfront.md index 989e9ddd..1ad273b0 100644 --- a/docs/shipping/AWS/aws-cloudfront.md +++ b/docs/shipping/AWS/aws-cloudfront.md @@ -32,7 +32,7 @@ CloudFront logs are useful for auditing/security monitoring and business intelli **Before you begin, you'll need**: -* `s3:ListBucket` and `s3:GetObject` [permissions](https://docs.logz.io/user-guide/give-aws-access-with-iam-roles/) for the required S3 bucket +* `s3:ListBucket` and `s3:GetObject` [permissions](/docs/user-guide/admin/give-aws-access-with-iam-roles) for the required S3 bucket * {@include: ../../_include/log-shipping/s3-bucket-file-order.md} @@ -57,7 +57,7 @@ Log into the app to use the dedicated Logz.io [configuration wizard](https://app Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ## Metrics diff --git a/docs/shipping/AWS/aws-cloudtrail.md b/docs/shipping/AWS/aws-cloudtrail.md index e72a4dee..4efa67f6 100644 --- a/docs/shipping/AWS/aws-cloudtrail.md +++ b/docs/shipping/AWS/aws-cloudtrail.md @@ -23,7 +23,7 @@ drop_filter: [] **Before you begin**: -* If you plan on using an access key to authenticate your connection, you'll need to set the `s3:ListBucket` and `s3:GetObject` [permissions](https://docs.logz.io/user-guide/give-aws-access-with-iam-roles/) for the required S3 bucket. +* If you plan on using an access key to authenticate your connection, you'll need to set the `s3:ListBucket` and `s3:GetObject` [permissions](/docs/user-guide/admin/give-aws-access-with-iam-roles) for the required S3 bucket. * If you plan on using an IAM role to authenticate your connection, you can get the role policy by filling out the bucket information and clicking the "Get the role policy" button. @@ -105,7 +105,7 @@ Logz.io cannot fetch past logs retroactively. Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ## Troubleshooting diff --git a/docs/shipping/AWS/aws-control-tower.md b/docs/shipping/AWS/aws-control-tower.md index 9570ee46..c4ae0150 100644 --- a/docs/shipping/AWS/aws-control-tower.md +++ b/docs/shipping/AWS/aws-control-tower.md @@ -153,7 +153,7 @@ If want to delete the S3 Hook Stack - you'll need to detach the policy "LambdaAc Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/AWS/aws-cost-and-usage-report.md b/docs/shipping/AWS/aws-cost-and-usage-report.md index 42f6fdb8..0e9b9080 100644 --- a/docs/shipping/AWS/aws-cost-and-usage-report.md +++ b/docs/shipping/AWS/aws-cost-and-usage-report.md @@ -85,7 +85,7 @@ All logs that were sent from the Lambda function will be under the type **billin To get more out of this functionality, you can enable a dedicated AWS cost and usage dashboard in [ELK Apps](https://app.logz.io/#/dashboard/elk-apps). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/AWS/aws-dynamodb.md b/docs/shipping/AWS/aws-dynamodb.md index e2a5e57d..b7a2fe3e 100644 --- a/docs/shipping/AWS/aws-dynamodb.md +++ b/docs/shipping/AWS/aws-dynamodb.md @@ -20,7 +20,7 @@ drop_filter: [] **Before you begin**: -* If you plan on using an access key to authenticate your connection, you'll need to set the `s3:ListBucket` and `s3:GetObject` [permissions](https://docs.logz.io/user-guide/give-aws-access-with-iam-roles/) for the required S3 bucket. +* If you plan on using an access key to authenticate your connection, you'll need to set the `s3:ListBucket` and `s3:GetObject` [permissions](/docs/user-guide/admin/give-aws-access-with-iam-roles) for the required S3 bucket. * If you plan on using an IAM role to authenticate your connection, you can get the role policy by filling out the bucket information and clicking the "Get the role policy" button. @@ -102,7 +102,7 @@ Logz.io cannot fetch past logs retroactively. Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ## Troubleshooting diff --git a/docs/shipping/AWS/aws-ec2-auto-scaling.md b/docs/shipping/AWS/aws-ec2-auto-scaling.md index 668ddd22..be8a206a 100644 --- a/docs/shipping/AWS/aws-ec2-auto-scaling.md +++ b/docs/shipping/AWS/aws-ec2-auto-scaling.md @@ -21,7 +21,7 @@ drop_filter: [] **Before you begin**: -* If you plan on using an access key to authenticate your connection, you'll need to set the `s3:ListBucket` and `s3:GetObject` [permissions](https://docs.logz.io/user-guide/give-aws-access-with-iam-roles/) for the required S3 bucket. +* If you plan on using an access key to authenticate your connection, you'll need to set the `s3:ListBucket` and `s3:GetObject` [permissions](/docs/user-guide/admin/give-aws-access-with-iam-roles) for the required S3 bucket. * If you plan on using an IAM role to authenticate your connection, you can get the role policy by filling out the bucket information and clicking the "Get the role policy" button. @@ -103,7 +103,7 @@ Logz.io cannot fetch past logs retroactively. Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ## Troubleshooting diff --git a/docs/shipping/AWS/aws-eks.md b/docs/shipping/AWS/aws-eks.md index b4eec4a8..2c1af277 100644 --- a/docs/shipping/AWS/aws-eks.md +++ b/docs/shipping/AWS/aws-eks.md @@ -9,7 +9,7 @@ logo: https://logzbucket.s3.eu-west-1.amazonaws.com/logz-docs/shipper-logos/aws- logs_dashboards: [] logs_alerts: [] logs2metrics: [] -metrics_dashboards: ['1Pm3OYbu1MRGoELc2qhxQ1'] +metrics_dashboards: [] metrics_alerts: [] drop_filter: [] --- diff --git a/docs/shipping/AWS/aws-fsx.md b/docs/shipping/AWS/aws-fsx.md index efeaa1ed..3e74dd7c 100644 --- a/docs/shipping/AWS/aws-fsx.md +++ b/docs/shipping/AWS/aws-fsx.md @@ -84,7 +84,7 @@ If you've used the `services` field, you'll have to **wait 6 minutes** before cr Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ## Metrics diff --git a/docs/shipping/AWS/aws-guardduty.md b/docs/shipping/AWS/aws-guardduty.md index 2f1f234d..48057a31 100644 --- a/docs/shipping/AWS/aws-guardduty.md +++ b/docs/shipping/AWS/aws-guardduty.md @@ -9,203 +9,96 @@ logo: https://logzbucket.s3.eu-west-1.amazonaws.com/logz-docs/shipper-logos/aws- logs_dashboards: [] logs_alerts: [] logs2metrics: [] -metrics_dashboards: ['1Pm3OYbu1MRGoELc2qhxQ1'] +metrics_dashboards: [] metrics_alerts: [] drop_filter: [] --- - -## Manual Lambda configuration - -{@include: ../../_include/log-shipping/note-lambda-test.md} - - - - -### Create a new Kinesis data stream - -If you're not already sending your GuardDuty logs through a Kinesis data stream, create one using the [Kinesis console](https://console.aws.amazon.com/kinesis). - -Save the name of the data stream—you'll need this in the next step. - -### Configure CloudWatch Events - -In the [CloudWatch console](https://console.aws.amazon.com/cloudwatch/) left menu, click **Events > Rules**, and then click **Create rule**. - -In the Event Source panel (on the left), set these options: - -* Choose **Event Pattern**. -* In the **Build event pattern** section, choose **GuardDuty** from the **Service Name** list. - You can choose any **Event Type** that you need. - -In the Targets panel (on the right), click **Add target**, and choose **Kinesis stream**. -Choose the Kinesis data stream from step 1 from the **Stream** list. - -Click **Configure details** (lower right corner). - -### Create a new IAM role - -Create a new IAM role and attach the **AWSLambdaKinesisExecutionRole** policy to the new role. - -### Create a new Lambda function - -This Lambda function will collect CloudWatch logs and sends them to Logz.io in bulk over HTTP. - -Open the AWS Lambda Console, and click **Create function**. -Choose **Author from scratch**, and use this information: - -* **Name**: - We suggest adding the log type to the name, but you can name this function whatever you want. -* **Runtime**: - Choose **Python 3.7** -* **Role**: - Select **Use an existing role**. - Then, select the role that you created in the previous step. It should have the **AWSLambdaKinesisExecutionRole** policy. - -Click **Create Function** (bottom right corner of the page). -After a few moments, you'll see configuration options for your Lambda function. - -You'll need this page later on, so keep it open. - -### Download the Kinesis stream shipper - -Download the latest Kinesis stream shipper zip file from the [Logz.io GitHub page](https://github.com/logzio/logzio_aws_serverless/releases). - -By default, the zip file will be named `logzio-kinesis-0.0.2.zip`. - -### Upload the zip file - -In the _Function_ code section of Lambda, open the **Code entry type** list and select **Upload a .ZIP file**. - -Click **Upload** and select the zip file you created in the previous step (`logzio-kinesis-0.0.2.zip`). - -### Set environment variables - -In the _Environment variables_ section, set your Logz.io account token, URL, and log type, and any other variables that you need to use. - -### Environment variables - -| Parameter | Description | Required/Default | -|---|---|---| -| TOKEN (Required) | Your Logz.io account token. {@include: ../../_include/log-shipping/log-shipping-token.html} | Required | -| REGION | Two-letter region code, or blank for US East (Northern Virginia). This determines your listener URL (where you're shipping the logs to) and API URL. You can find your region code in the [Regions and URLs](https://docs.logz.io/user-guide/accounts/account-region.html#regions-and-urls) table. | Default: *blank* (US East)| -| URL (Deprecated)| Use REGION instead. Protocol, listener host, and port (for example, `https://<>:8071`). {@include: ../../_include/log-shipping/listener-var.html} | Required | -| TYPE | The log type you'll use with this Lambda. This can be a [built-in log type]({{site.baseurl}}/user-guide/log-shipping/built-in-log-types.html), or a custom log type. You should create a new Lambda for each log type you use. | `"guardduty"` | -| FORMAT | `"json"` or `"text"`. If `"json"`, the Lambda function will attempt to parse the message field as JSON and populate the event data with the parsed fields. | `"text"` | -| COMPRESS | Set to `true` to compress logs before sending them. Set to `false` to send uncompressed logs. | `false` | - - -### Configure the function's basic settings - -In Basic settings, we recommend starting with these settings: - -* **Memory**: 512 MB -* **Timeout**: 1 min 0 sec - -:::note -These default settings are just a starting point. Check your Lambda usage regularly, and adjust these values if you need to. +## Logs + +### Create an EventBridge rule + +You'll need to create a new EventBridge rule that will send your GuardDuty findings to a Cloudwatch Log Group. + +1. In your **AWS Console**, go to ** Amazon EventBridge** service. +2. In the left menu of Amazon EventBridge, choose **Rules**, then click on **Create rule**. +3. Enter the name of your new rule, and click **Next**. +4. Scroll down to the **Event pattern** panel. In the **AWS service** field, choose **GuardDuty**. In the **Event type** field choose **All Events**, and click **Next**. +5. For the **Select a target** field, choose **CloudWatch log group**. In the **Log Group** field, choose the first option (`/aws/events`) and enter the name you'd like for your new log group. Click **Next**. +6. Optionally, add tags to your event rule. Click **Next**. +7. Review the details and click **Create rule**. + + +### Auto-deploy the Stack in the relevant region + +This integration will deploy a Firehose connection with your AWS services to forward logs to Logz.io +To deploy this project, click the button that matches the region you wish to deploy your Stack to: + +| Region | Deployment | +|------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `us-east-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-us-east-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `us-east-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=us-east-2#/stacks/create/review?templateURL=https://logzio-aws-integrations-us-east-2.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `us-west-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=us-west-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-us-west-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `us-west-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/create/review?templateURL=https://logzio-aws-integrations-us-west-2.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `eu-central-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-central-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-eu-central-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `eu-north-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-north-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-eu-north-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `eu-west-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-eu-west-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `eu-west-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-2#/stacks/create/review?templateURL=https://logzio-aws-integrations-eu-west-2.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `eu-west-3` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-3#/stacks/create/review?templateURL=https://logzio-aws-integrations-eu-west-3.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `sa-east-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=sa-east-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-sa-east-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `ap-northeast-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-northeast-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `ap-northeast-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-2#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-northeast-2.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `ap-northeast-3` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-3#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-northeast-3.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `ap-south-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-south-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-south-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `ap-southeast-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-southeast-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `ap-southeast-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-2#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-southeast-2.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `ca-central-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ca-central-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-ca-central-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | + +#### Specify stack details + +Specify the stack details as per the table below, check the checkboxes and select **Create stack**. +Add the CloudWatch log group name you created in the first step to field `customLogGroups`. + +| Parameter | Description | Required/Default | +|--------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------| +| `logzioToken` | The [token](https://app.logz.io/#/dashboard/settings/general) of the account you want to ship logs to. | **Required** | +| `logzioListener` | Listener host. | **Required** | +| `logzioType` | The log type you'll use with this Lambda. This can be a [built-in log type](https://docs.logz.io/user-guide/log-shipping/built-in-log-types.html), or a custom log type. | `logzio_firehose` | +| `services` | A comma-seperated list of services you want to collect logs from. Supported options are: `apigateway`, `rds`, `cloudhsm`, `cloudtrail`, `codebuild`, `connect`, `elasticbeanstalk`, `ecs`, `eks`, `aws-glue`, `aws-iot`, `lambda`, `macie`, `amazon-mq`. | - | +| `customLogGroups` | A comma-seperated list of custom log groups you want to collect logs from | - | +| `triggerLambdaTimeout` | The amount of seconds that Lambda allows a function to run before stopping it, for the trigger function. | `60` | +| `triggerLambdaMemory` | Trigger function's allocated CPU proportional to the memory configured, in MB. | `512` | +| `triggerLambdaLogLevel` | Log level for the Lambda function. Can be one of: `debug`, `info`, `warn`, `error`, `fatal`, `panic` | `info` | +| `httpEndpointDestinationIntervalInSeconds` | The length of time, in seconds, that Kinesis Data Firehose buffers incoming data before delivering it to the destination | `60` | +| `httpEndpointDestinationSizeInMBs` | The size of the buffer, in MBs, that Kinesis Data Firehose uses for incoming data before delivering it to the destination | `5` | + + +:::caution Important +AWS limits every log group to have up to 2 subscription filters. If your chosen log group already has 2 subscription filters, the trigger function won't be able to add another one. ::: - - -### Set the Kinesis event trigger - -Find the **Add triggers** list (left side of the Designer panel). -Choose **Kinesis** from this list. - -Below the Designer, you'll see the Configure triggers panel. -Choose the **Kinesis stream** that the Lambda function will watch. - -Click **Add**, and then click **Save** at the top of the page. - -### Check Logz.io for your logs - -Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). - -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). - - - - -## Automated CloudFormation deployment - -{@include: ../../_include/log-shipping/note-lambda-test.md} - - -**Before you begin, you'll need**: -AWS CLI, -an S3 bucket to store the CloudFormation package - - -### Create a new Kinesis data stream - -If you're not already sending your GuardDuty logs through a Kinesis data stream, create one using the [Kinesis console](https://console.aws.amazon.com/kinesis). - -Select the button below according to the region where you need to deploy the stack. - -| REGION | DEPLOYMENT | -| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `us-east-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create/template?templateURL=https://logzio-aws-integrations-us-east-1.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `us-east-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=us-east-2#/stacks/create/template?templateURL=https://logzio-aws-integrations-us-east-2.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `us-west-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-1#/stacks/create/template?templateURL=https://logzio-aws-integrations-us-west-1.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `us-west-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-2#/stacks/create/template?templateURL=https://logzio-aws-integrations-us-west-2.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `eu-central-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-central-1#/stacks/create/template?templateURL=https://logzio-aws-integrations-eu-central-1.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `eu-north-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-north-1#/stacks/create/template?templateURL=https://logzio-aws-integrations-eu-north-1.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `eu-west-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-1#/stacks/create/template?templateURL=https://logzio-aws-integrations-eu-west-1.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `eu-west-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-2#/stacks/create/template?templateURL=https://logzio-aws-integrations-eu-west-2.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `eu-west-3` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-3#/stacks/create/template?templateURL=https://logzio-aws-integrations-eu-west-3.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `sa-east-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=sa-east-1#/stacks/create/template?templateURL=https://logzio-aws-integrations-sa-east-1.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `ca-central-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ca-central-1#/stacks/create/template?templateURL=https://logzio-aws-integrations-ca-central-1.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-shipper¶m_LogzioTOKEN=<>) | -| `ap-northeast-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-1#/stacks/create/template?templateURL=https://logzio-aws-integrations-ap-northeast-1.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `ap-northeast-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-2#/stacks/create/template?templateURL=https://logzio-aws-integrations-ap-northeast-2.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `ap-northeast-3` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-3#/stacks/create/template?templateURL=https://logzio-aws-integrations-ap-northeast-3.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `ap-south-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-south-1#/stacks/create/template?templateURL=https://logzio-aws-integrations-ap-south-1.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `ap-southeast-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-1#/stacks/create/template?templateURL=https://logzio-aws-integrations-ap-southeast-1.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `ap-southeast-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-2#/stacks/create/template?templateURL=https://logzio-aws-integrations-ap-southeast-2.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | - -![Create stack](https://dytvr9ot2sszz.cloudfront.net/logz-docs/guardduty/first.png) - -Keep the default setting in the **Create stack** screen and select **Next**. - -### Specify the stack details - -![Specify stack details](https://dytvr9ot2sszz.cloudfront.net/logz-docs/guardduty/second.png) +#### Send logs +Give the stack a few minutes to be deployed. -Specify the stack details as per the table below and select **Next**. +Once new logs are added to your chosen log group, they will be sent to your Logz.io account. +Your GuardDuty logs will be sent in accordance with your GuardDuty configuration. +GuardDuty publishes its findings to EventBridge every 6 hours. If you want to configure it differently: +1. Go to your GuardDuty settings. +2. Scroll down to **Findings export options**. Click on **Edit** of **Frequency**. +3. Choose your prefered frequency to export GuardDuty findings. -| Parameter | Description | Required/Default | -| ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------- | -| LogzioTOKEN | Your Logz.io account token. {@include: ../../_include/log-shipping/log-shipping-token.html} | Required | -| KinesisStream | The name of the Kinesis stream where this function will listen for updates. | Required | -| LogzioREGION | Two-letter region code, or blank for US East (Northern Virginia). This determines your listener URL (where you're shipping the logs to) and API URL. You can find your region code in the [Regions and URLs](https://docs.logz.io/user-guide/accounts/account-region.html#regions-and-urls) table. | Default: _blank_ (US East) | -| LogzioURL (Deprecated) | Use LogzioREGION instead. Protocol, listener host, and port (for example, `https://<>:8071`). {@include: ../../_include/log-shipping/listener-var.html} | Required | -| LogzioTYPE | The log type you'll use with this Lambda. This can be a [built-in log type]({{site.baseurl}}/user-guide/log-shipping/built-in-log-types.html), or a custom log type. You should create a new Lambda for each log type you use. | `guardduty` | -| LogzioFORMAT | `"json"` or `"text"`. If `"json"`, the Lambda function will attempt to parse the message field as JSON and populate the event data with the parsed fields. | `"text"` | -| LogzioCOMPRESS | Set to `true` to compress logs before sending them. Set to `false` to send uncompressed logs. | `false` | -| KinesisStreamBatchSize | The largest number of records to read from your stream at one time. | `100` | -| KinesisStreamStartingPosition | The position in the stream to start reading from. For more information, see [ShardIteratorType](https://docs.aws.amazon.com/kinesis/latest/APIReference/API_GetShardIterator.html) in the Amazon Kinesis API Reference. | `"LATEST"` | +You can export a sample finding by going to GuardDuty **settings** and clicking the **Generate sample findings**. -### Configure the stack options - -![Configure stack options](https://dytvr9ot2sszz.cloudfront.net/logz-docs/guardduty/third.png) - -Specify the **Key** and **Value** parameters for the **Tags** and select **Next**. - -### Review the deployment - -![Review deployment](https://dytvr9ot2sszz.cloudfront.net/logz-docs/guardduty/fourth.png) - -Confirm that you acknowledge that AWS CloudFormation might create IAM resources and select **Create stack**. +:::caution Important +If you've used the `services` field, you'll have to **wait 6 minutes** before creating new log groups for your chosen services. This is due to cold start and custom resource invocation, that can cause the Lambda to behave unexpectedly. +::: -### Check Logz.io for your logs +#### Check Logz.io for your logs Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). - - - \ No newline at end of file +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/AWS/aws-kinesis-firehose.md b/docs/shipping/AWS/aws-kinesis-firehose.md index 34757761..5d05edab 100644 --- a/docs/shipping/AWS/aws-kinesis-firehose.md +++ b/docs/shipping/AWS/aws-kinesis-firehose.md @@ -90,7 +90,7 @@ Give your logs some time to get from your system to ours, and then open [Open Se -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/AWS/aws-mq.md b/docs/shipping/AWS/aws-mq.md index f03b8095..8ab5a619 100644 --- a/docs/shipping/AWS/aws-mq.md +++ b/docs/shipping/AWS/aws-mq.md @@ -83,7 +83,7 @@ If you've used the `services` field, you'll have to **wait 6 minutes** before cr Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ## Metrics diff --git a/docs/shipping/AWS/aws-msk.md b/docs/shipping/AWS/aws-msk.md index af14a846..8b272377 100644 --- a/docs/shipping/AWS/aws-msk.md +++ b/docs/shipping/AWS/aws-msk.md @@ -83,7 +83,7 @@ If you've used the `services` field, you'll have to **wait 6 minutes** before cr Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ## Metrics diff --git a/docs/shipping/AWS/aws-network-elb.md b/docs/shipping/AWS/aws-network-elb.md index 7c31470c..05636ce6 100644 --- a/docs/shipping/AWS/aws-network-elb.md +++ b/docs/shipping/AWS/aws-network-elb.md @@ -30,7 +30,7 @@ When you set Logz.io to fetch Elastic Load Balancing (ELB) logs, Logz.io will pe **Before you begin, you'll need**: -`s3:ListBucket` and `s3:GetObject` [permissions](https://docs.logz.io/user-guide/give-aws-access-with-iam-roles/) for the required S3 bucket (one bucket per region) +`s3:ListBucket` and `s3:GetObject` [permissions](/docs/user-guide/admin/give-aws-access-with-iam-roles) for the required S3 bucket (one bucket per region) @@ -56,7 +56,7 @@ Log into the app to use the dedicated Logz.io [configuration wizard](https://app Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/AWS/aws-rds.md b/docs/shipping/AWS/aws-rds.md index 6a832161..18deda13 100644 --- a/docs/shipping/AWS/aws-rds.md +++ b/docs/shipping/AWS/aws-rds.md @@ -88,7 +88,7 @@ docker run -d \ Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/AWS/aws-route53.md b/docs/shipping/AWS/aws-route53.md index 7bdd8586..9ec3529a 100644 --- a/docs/shipping/AWS/aws-route53.md +++ b/docs/shipping/AWS/aws-route53.md @@ -20,7 +20,7 @@ drop_filter: [] **Before you begin**: -* If you plan on using an access key to authenticate your connection, you'll need to set the `s3:ListBucket` and `s3:GetObject` [permissions](https://docs.logz.io/user-guide/give-aws-access-with-iam-roles/) for the required S3 bucket. +* If you plan on using an access key to authenticate your connection, you'll need to set the `s3:ListBucket` and `s3:GetObject` [permissions](/docs/user-guide/admin/give-aws-access-with-iam-roles) for the required S3 bucket. * If you plan on using an IAM role to authenticate your connection, you can get the role policy by filling out the bucket information and clicking the "Get the role policy" button. @@ -102,7 +102,7 @@ Logz.io cannot fetch past logs retroactively. Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ## Troubleshooting diff --git a/docs/shipping/AWS/aws-s3-access.md b/docs/shipping/AWS/aws-s3-access.md index 49327545..2310cd05 100644 --- a/docs/shipping/AWS/aws-s3-access.md +++ b/docs/shipping/AWS/aws-s3-access.md @@ -21,7 +21,7 @@ Amazon S3 Access Logs provide detailed records about requests that are made to y **Before you begin, you'll need**: -* `s3:ListBucket` and `s3:GetObject` [permissions](https://docs.logz.io/user-guide/give-aws-access-with-iam-roles/) for the required S3 bucket +* `s3:ListBucket` and `s3:GetObject` [permissions](/docs/user-guide/admin/give-aws-access-with-iam-roles) for the required S3 bucket * {@include: ../../_include/log-shipping/s3-bucket-file-order.md} @@ -50,6 +50,6 @@ For help with this, see [Amazon S3 Server Access Logging](https://docs.aws.amazo Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/AWS/aws-s3-bucket.md b/docs/shipping/AWS/aws-s3-bucket.md index 7d668060..785f1dd4 100644 --- a/docs/shipping/AWS/aws-s3-bucket.md +++ b/docs/shipping/AWS/aws-s3-bucket.md @@ -217,7 +217,7 @@ In Logz.io, paste the ARN in the **Role ARN** field, and then click **Save**. Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). @@ -373,7 +373,7 @@ and then click **Save**. Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ### Troubleshooting diff --git a/docs/shipping/AWS/aws-security-hub.md b/docs/shipping/AWS/aws-security-hub.md index 56ec8778..d859f92c 100644 --- a/docs/shipping/AWS/aws-security-hub.md +++ b/docs/shipping/AWS/aws-security-hub.md @@ -100,7 +100,7 @@ This deployment will automatically create the following resources: Give the stack some time to deploy and the resources to get created. Once this is finished, the stack sends a security event to Logz.io as soon as the event is created on the security hub. You can then see the data in [OpenSearch Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your events, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your events, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/AWS/aws-sqs.md b/docs/shipping/AWS/aws-sqs.md index 2b9a9faf..5bfbf1b9 100644 --- a/docs/shipping/AWS/aws-sqs.md +++ b/docs/shipping/AWS/aws-sqs.md @@ -84,7 +84,7 @@ If you've used the `services` field, you'll have to **wait 6 minutes** before cr Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ## Metrics diff --git a/docs/shipping/AWS/aws-waf.md b/docs/shipping/AWS/aws-waf.md index 07306f27..8e4b91ad 100644 --- a/docs/shipping/AWS/aws-waf.md +++ b/docs/shipping/AWS/aws-waf.md @@ -60,7 +60,7 @@ You'll first need to make sure all your logs are being written to an S3 bucket. 2. In the log type section menu of Logz.io configuration wizard, select `other` and type in `awswaf`. The log type section menu is located beside the hosting region selection menu. :::note -If you run into issues, you can reference the [guide for troubleshooting user permissions](https://docs.logz.io/user-guide/give-aws-access-with-iam-roles/). +If you run into issues, you can reference the [guide for troubleshooting user permissions](/docs/user-guide/admin/give-aws-access-with-iam-roles). ::: @@ -68,5 +68,5 @@ If you run into issues, you can reference the [guide for troubleshooting user pe Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd) and search for `type: awswaf`. -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/Access-Management/active-directory.md b/docs/shipping/Access-Management/active-directory.md index 8f90fe42..37eef2b6 100644 --- a/docs/shipping/Access-Management/active-directory.md +++ b/docs/shipping/Access-Management/active-directory.md @@ -106,4 +106,4 @@ Restart-Service winlogbeat Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/Access-Management/auth0.md b/docs/shipping/Access-Management/auth0.md index 94dd05a4..5d711694 100644 --- a/docs/shipping/Access-Management/auth0.md +++ b/docs/shipping/Access-Management/auth0.md @@ -55,7 +55,7 @@ Select **Save** to save the changes and create the stream. Give your data some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). You can filter for data of type `auth0` to see the incoming Auth0 events. -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). @@ -112,6 +112,6 @@ After you authorize the extension, you are directed to the **Logs Export** page, Give your data some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). You can filter for data of type `auth0` to see the incoming Auth0 events. -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/Access-Management/jumpcloud.md b/docs/shipping/Access-Management/jumpcloud.md index dccf2870..51a87e91 100644 --- a/docs/shipping/Access-Management/jumpcloud.md +++ b/docs/shipping/Access-Management/jumpcloud.md @@ -75,7 +75,7 @@ docker run --name logzio-jumpcloud -v "$(pwd)":/app/src/shared logzio/logzio-jum Give your data some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ### Stop the docker container diff --git a/docs/shipping/Access-Management/okta.md b/docs/shipping/Access-Management/okta.md index 84d350d5..53aae25f 100644 --- a/docs/shipping/Access-Management/okta.md +++ b/docs/shipping/Access-Management/okta.md @@ -134,6 +134,6 @@ For more information about mounting files from root directory click [here](https Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/Access-Management/onelogin.md b/docs/shipping/Access-Management/onelogin.md index 11033ba2..a1667957 100644 --- a/docs/shipping/Access-Management/onelogin.md +++ b/docs/shipping/Access-Management/onelogin.md @@ -53,7 +53,7 @@ When everything is filled out, click **Save**. Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). You can search for `type:onelogin` to filter for your OneLogin logs. -If you still don’t see your logs, see [log shipping troubleshooting](https://docs.logz.io/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don’t see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/Azure/azure-active-directory.md b/docs/shipping/Azure/azure-active-directory.md index 3bd71389..2b58b35b 100644 --- a/docs/shipping/Azure/azure-active-directory.md +++ b/docs/shipping/Azure/azure-active-directory.md @@ -151,6 +151,6 @@ Logs collected by this integration will have the type `Microsoft-Graph` Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/Azure/azure-graph.md b/docs/shipping/Azure/azure-graph.md index 86a5761a..926b8c65 100644 --- a/docs/shipping/Azure/azure-graph.md +++ b/docs/shipping/Azure/azure-graph.md @@ -184,6 +184,6 @@ Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). You can filter for data of your custom field type value or type `api_fetcher` to see the incoming Microsoft Graph logs. If you still don't see your logs, -see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). You can see a full list of the possible configuration values in the [logzio-api-fetcher github repository](https://github.com/logzio/logzio-api-fetcher). \ No newline at end of file diff --git a/docs/shipping/Azure/azure-vm-extension.md b/docs/shipping/Azure/azure-vm-extension.md index c7383036..3315e3ae 100644 --- a/docs/shipping/Azure/azure-vm-extension.md +++ b/docs/shipping/Azure/azure-vm-extension.md @@ -9,7 +9,7 @@ logo: https://logzbucket.s3.eu-west-1.amazonaws.com/logz-docs/shipper-logos/azur logs_dashboards: [] logs_alerts: [] logs2metrics: [] -metrics_dashboards: ['1Pm3OYbu1MRGoELc2qhxQ1'] +metrics_dashboards: [] metrics_alerts: [] drop_filter: [] --- @@ -67,7 +67,7 @@ Run the VM to generate logs. Give your data some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ### Uninstall the extension diff --git a/docs/shipping/CI-CD/github.md b/docs/shipping/CI-CD/github.md index 35b5c51b..dcb4a465 100644 --- a/docs/shipping/CI-CD/github.md +++ b/docs/shipping/CI-CD/github.md @@ -72,7 +72,7 @@ Complete filling in the form: Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). Search for `type:github` in Open Search Dashboards Discover to filter for your GitHub events. Your logs should be already parsed thanks to the Logz.io preconfigured parsing pipeline. -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ## Metrics diff --git a/docs/shipping/Code/go.md b/docs/shipping/Code/go.md index 47b6c5e2..52a1bbf9 100644 --- a/docs/shipping/Code/go.md +++ b/docs/shipping/Code/go.md @@ -9,7 +9,7 @@ logo: https://logzbucket.s3.eu-west-1.amazonaws.com/logz-docs/shipper-logos/go.s logs_dashboards: [] logs_alerts: [] logs2metrics: [] -metrics_dashboards: ['1Pm3OYbu1MRGoELc2qhxQ1'] +metrics_dashboards: [] metrics_alerts: [] drop_filter: [] --- diff --git a/docs/shipping/Code/java.md b/docs/shipping/Code/java.md index 302087fd..0eb8535c 100644 --- a/docs/shipping/Code/java.md +++ b/docs/shipping/Code/java.md @@ -28,7 +28,7 @@ import TabItem from '@theme/TabItem'; :::note -[Project's GitHub repo](https://github.com/logzio/go-metrics-sdk/) +[Project's GitHub repo](https://github.com/logzio/logzio-log4j2-appender/) ::: The Logz.io Log4j 2 appender sends logs using non-blocking threading, bulks, and HTTPS encryption to port 8071. @@ -265,7 +265,7 @@ public class LogzioLog4j2Example { :::note -[Project's GitHub repo](https://github.com/logzio/ogzio-log4j2-appender/) +[Project's GitHub repo](https://github.com/logzio/logzio-logback-appender) ::: Logback sends logs to your Logz.io account using non-blocking threading, bulks, and HTTPS encryption to port 8071. @@ -517,8 +517,10 @@ If the log appender does not ship logs, add `true + ## Metrics + :::note [Project's GitHub repo](https://github.com/logzio/micrometer-registry-logzio/) ::: diff --git a/docs/shipping/Compute/internet-information-services.md b/docs/shipping/Compute/internet-information-services.md index 67521756..50644734 100644 --- a/docs/shipping/Compute/internet-information-services.md +++ b/docs/shipping/Compute/internet-information-services.md @@ -77,6 +77,6 @@ PS C:\Program Files (x86)\nxlog> Restart-Service nxlog Confirm you're shipping logs by opening an IIS-hosted webpage in your browser. Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/Containers/control-plane.md b/docs/shipping/Containers/control-plane.md index 148435c2..4a62a6c0 100644 --- a/docs/shipping/Containers/control-plane.md +++ b/docs/shipping/Containers/control-plane.md @@ -84,4 +84,4 @@ cpln org patch ORG_NAME -f logging-config.yaml Spin up your Docker containers if you haven't done so already. Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/Containers/docker.md b/docs/shipping/Containers/docker.md index d1b6b31b..85d23f11 100644 --- a/docs/shipping/Containers/docker.md +++ b/docs/shipping/Containers/docker.md @@ -107,7 +107,7 @@ By default, logs from docker-collector-logs and docker-collector-metrics contain Spin up your Docker containers if you haven't done so already. Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ### The logzio-logging plugin @@ -221,7 +221,7 @@ For a complete list of options, see the configuration parameters above. 👆 Spin up your Docker containers if you haven't done so already. Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ## Metrics diff --git a/docs/shipping/Containers/kubernetes.md b/docs/shipping/Containers/kubernetes.md index 08d07969..50cf8f31 100644 --- a/docs/shipping/Containers/kubernetes.md +++ b/docs/shipping/Containers/kubernetes.md @@ -6,7 +6,7 @@ product: ['logs', 'metrics', 'tracing'] os: ['windows', 'linux'] filters: ['Containers', 'Most Popular'] logo: https://logzbucket.s3.eu-west-1.amazonaws.com/logz-docs/shipper-logos/kubernetes.svg -logs_dashboards: [] +logs_dashboards: ['3D1grGcEYB5Oe2feUPImak','qryn7oYYoeaBBGMFRvm67'] logs_alerts: ['1AZRkKc64I12yxAMf2Wyny','6H7dfFOPUaHVMIjxdOMASx','1F6zSL5me5XJt9Lrjw3vxU','2dQHLx0WxmKmLk1kc67Ags','3dyFejyivMaZFdudbwKGRG'] logs2metrics: [] metrics_dashboards: ['7nILXHYFZbThgTSMObUxkw','5TGD77ZKuTiZUXtiM51m6V','6pY6DKD0oQJL4sO7bW728','5kkUAuEwA0Ygvlgm9iXTHY','53g5kSILqoj1T10U1jnvKV','5e1xRaDdQnOvs5LCuwKCh5','7Cy6DUN78jlKUtMCsbt6GC','29HGYsE3kgFEdgJbalTqeY','1Hij49FKdnAKVJTjOmpDbH'] @@ -14,248 +14,4 @@ metrics_alerts: ['5Ng398K19vXP9197bRV1If'] drop_filter: [] --- - - - - -The logzio-monitoring Helm Chart ships your Kubernetes telemetry (logs, metrics, traces and security reports) to your Logz.io account. - -## Prerequisites - -1. [Helm](https://helm.sh/) - -Add Logzio-helm repository - -```sh -helm repo add logzio-helm https://logzio.github.io/logzio-helm && helm repo update -``` -{@include: ../../_include/general-shipping/k8s-all-data.md} - -## Send your logs - -```sh -helm install -n monitoring \ ---set logs.enabled=true \ ---set logzio-fluentd.secrets.logzioShippingToken="<>" \ ---set logzio-fluentd.secrets.logzioListener="<>" \ ---set logzio-fluentd.env_id="<>" \ -logzio-monitoring logzio-helm/logzio-monitoring -``` - - -| Parameter | Description | -| --- | --- | -| `<>` | Your [logs shipping token](https://app.logz.io/#/dashboard/settings/general). | -| `<>` | Your account's [listener host](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping?product=logs). | -| `<>` | The cluster's name, to easily identify the telemetry data for each environment. | - - -For log shipping troubleshooting, see our [user guide](https://docs.logz.io/user-guide/kubernetes-troubleshooting/). - -## Send your deploy events logs - -This integration sends data about deployment events in the cluster, and how they affect the cluster's resources. -Currently supported resource kinds are `Deployment`, `Daemonset`, `Statefulset`, `ConfigMap`, `Secret`, `Service Account`, `Cluster Role` and `Cluster Role Binding`. - -```sh -helm install --namespace=monitoring \ ---set logzio-k8s-events.secrets.logzioShippingToken='<>' \ ---set logzio-k8s-events.secrets.logzioListener='<>' \ ---set logzio-k8s-events.secrets.env_id='<>' \ ---set logzio-k8s-events.secrets.customListener='<>' \ -logzio-monitoring logzio-helm/logzio-monitoring - -``` - - -| Parameter | Description | -| --- | --- | -| `<>` | Your [logs shipping token](https://app.logz.io/#/dashboard/settings/general). | -| `<>` | Your account's [listener host](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping?product=logs). | -| `<>` | The cluster's name, to easily identify the telemetry data for each environment. | -| `<>` | (*optional*) HTTP/s listener endpoint that receives JSON input, overrides the Logz.io listener. | - -### Deployment Events Versioning - -In order to add an indication for the versioning in our K8S 360 and Service Overview UI, the following annotation should be added to the metadata of each resource you'd like to track its versioning. -Commit URL structure: `https://github.com///commit/` - -Example: `https://github.com/logzio/logzio-k8s-events/commit/069c75c95caeca58dd0776405bb8dfb4eed3acb2` - -```yaml -metadata: - annotations: - logzio/commit_url: "" -``` - - -For log shipping troubleshooting, see our [user guide](https://docs.logz.io/user-guide/kubernetes-troubleshooting/). - -## Send your Metrics - -```sh -helm install -n monitoring \ ---set metricsOrTraces.enabled=true \ ---set logzio-k8s-telemetry.metrics.enabled=true \ ---set logzio-k8s-telemetry.secrets.MetricsToken="<>" \ ---set logzio-k8s-telemetry.secrets.ListenerHost="https://<>:8053" \ ---set logzio-k8s-telemetry.secrets.p8s_logzio_name="<>" \ ---set logzio-k8s-telemetry.secrets.env_id="<>" \ -logzio-monitoring logzio-helm/logzio-monitoring -``` - -| Parameter | Description | -| --- | --- | -| `<>` | Your [metrics shipping token](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping). | -| `<>` | The cluster's name, to easily identify the telemetry data for each environment. | -| `<>` | Your account's [listener host](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping?product=logs). | - - -For metrics shipping troubleshooting, see our [user guide](https://docs.logz.io/user-guide/infrastructure-monitoring/troubleshooting/k8-helm-opentelemetry-troubleshooting.html). - - - -## Send your traces - -```sh -helm install -n monitoring \ ---set metricsOrTraces.enabled=true \ ---set logzio-k8s-telemetry.traces.enabled=true \ ---set logzio-k8s-telemetry.secrets.TracesToken="<>" \ ---set logzio-k8s-telemetry.secrets.LogzioRegion="<>" \ ---set logzio-k8s-telemetry.secrets.env_id="<>" \ -logzio-monitoring logzio-helm/logzio-monitoring -``` - -| Parameter | Description | -| --- | --- | -| `<>` | Your [traces shipping token](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping?product=tracing). | -| `<>` | The cluster's name, to easily identify the telemetry data for each environment. | -| `<>` | Name of your Logz.io traces region e.g `us`, `eu`... | - - -For traces shipping troubleshooting, see our [user guide]([https://docs.logz.io/user-guide/kubernetes-troubleshooting/](https://docs.logz.io/user-guide/distributed-tracing/tracing-troubleshooting.html)). - - -## Send traces with SPM - -```sh -helm install -n monitoring \ ---set metricsOrTraces.enabled=true \ ---set logzio-k8s-telemetry.traces.enabled=true \ ---set logzio-k8s-telemetry.secrets.TracesToken="<>" \ ---set logzio-k8s-telemetry.secrets.LogzioRegion="<>" \ ---set logzio-k8s-telemetry.secrets.env_id="<>" \ ---set logzio-k8s-telemetry.spm.enabled=true \ ---set logzio-k8s-telemetry.secrets.SpmToken=<> \ -logzio-monitoring logzio-helm/logzio-monitoring -``` - -| Parameter | Description | -| --- | --- | -| `<>` | Your [traces shipping token](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping). | -| `<>` | The cluster's name, to easily identify the telemetry data for each environment. | -| `<>` | Your account's [listener host](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping?product=logs). | -| `<>` | Name of your Logz.io traces region e.g `us`, `eu`... | -| `<>` | Your [span metrics shipping token](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping). | - - -## Scan your cluster for security vulnerabilities - -```sh -helm install -n monitoring \ ---set securityReport.enabled=true \ ---set logzio-trivy.env_id="<>" \ ---set logzio-trivy.secrets.logzioShippingToken="<>" \ ---set logzio-trivy.secrets.logzioListener="<>" \ -``` - -| Parameter | Description | -| --- | --- | -| `<>` | Your [logs shipping token](https://app.logz.io/#/dashboard/settings/general). | -| `<>` | Your account's [listener host](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping?product=logs). | -| `<>` | The cluster's name, to easily identify the telemetry data for each environment. | - - -## Modifying the configuration for logs - -You can see a full list of the possible configuration values in the [logzio-fluentd Chart folder](https://github.com/logzio/logzio-helm/tree/master/charts/fluentd#configuration). - -If you would like to modify any of the values found in the `logzio-fluentd` folder, use the `--set` flag with the `logzio-fluentd` prefix. - -For instance, if there is a parameter called `someField` in the `logzio-telemetry`'s `values.yaml` file, you can set it by adding the following to the `helm install` command: - -```sh ---set logzio-fluentd.someField="my new value" -``` -You can add `log_type` annotation with a custom value, which will be parsed into a `log_type` field with the same value. - - -### Modifying the configuration for metrics and traces - -You can see a full list of the possible configuration values in the [logzio-telemetry Chart folder](https://github.com/logzio/logzio-helm/tree/master/charts/logzio-telemetry). - -If you would like to modify any of the values found in the `logzio-telemetry` folder, use the `--set` flag with the `logzio-k8s-telemetry` prefix. - -For instance, if there is a parameter called `someField` in the `logzio-telemetry`'s `values.yaml` file, you can set it by adding the following to the `helm install` command: - - -```sh ---set logzio-k8s-telemetry.someField="my new value" -``` - -## Sending telemetry data from eks on fargate - -To ship logs from pods running on Fargate, set the `fargateLogRouter.enabled` value to `true`. Doing so will deploy a dedicated `aws-observability` namespace and a `configmap` for the Fargate log router. For more information on EKS Fargate logging, please refer to the [official AWS documentation]((https://docs.aws.amazon.com/eks/latest/userguide/fargate-logging.html). - -```shell -helm install -n monitoring \ ---set logs.enabled=true \ ---set logzio-fluentd.fargateLogRouter.enabled=true \ ---set logzio-fluentd.secrets.logzioShippingToken="<>" \ ---set logzio-fluentd.secrets.logzioListener="<>" \ ---set metricsOrTraces.enabled=true \ ---set logzio-k8s-telemetry.metrics.enabled=true \ ---set logzio-k8s-telemetry.secrets.MetricsToken="<>" \ ---set logzio-k8s-telemetry.secrets.ListenerHost="https://<>:8053" \ ---set logzio-k8s-telemetry.secrets.p8s_logzio_name="<>" \ ---set logzio-k8s-telemetry.traces.enabled=true \ ---set logzio-k8s-telemetry.secrets.TracesToken="<>" \ ---set logzio-k8s-telemetry.secrets.LogzioRegion="<>" \ -logzio-monitoring logzio-helm/logzio-monitoring -``` - -| Parameter | Description | -| --- | --- | -| `<>` | Your [logs shipping token](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping?product=logs). | -| `<>` | Your account's [listener host](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping?product=logs). | -| `<>` | Your [metrics shipping token](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping?product=metrics). | -| `<>` | The name for the environment's metrics, to easily identify the metrics for each environment. | -| `<>` | The name for your environment's identifier, to easily identify the telemetry data for each environment. | -| `<>` | Your custom name for the environment's metrics, to easily identify the metrics for each environment. | -| `<>` | Replace `<>` with the [token](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping?product=tracing) of the account you want to ship to. | -| `<>` | Name of your Logz.io traces region e.g `us` or `eu`. You can find your region code in the [Regions and URLs](https://docs.logz.io/user-guide/accounts/account-region.html#regions-and-urls) table. | - -## Handling image pull rate limit - -In certain situations, such as with spot clusters where pods/nodes are frequently replaced, you may encounter the pull rate limit for images fetched from Docker Hub. This could result in the following error: `You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limits`. - -To address this issue, you can use the `--set` commands provided below in order to access an alternative image repository: - -```shell ---set logzio-k8s-telemetry.image.repository=ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-contrib ---set logzio-k8s-telemetry.prometheus-pushgateway.image.repository=public.ecr.aws/logzio/prom-pushgateway ---set logzio-fluentd.image=public.ecr.aws/logzio/logzio-fluentd ---set logzio-fluentd.daemonset.init.containerImage=public.ecr.aws/docker/library/busybox ---set logzio-trivy.image=public.ecr.aws/logzio/trivy-to-logzio -``` - -## Upgrade logzio-monitoring to v3.0.0 - -The logzio-monitoring chart was upgraded with breaking changes in version 3.0.0. Upon attempting to upgrade the chart using `helm upgrade -n monitoring --version 3.0.0 logzio-monitoring logzio-helm/logzio-monitoring`, an error occurs that correlates with the version upgrade of the subcharts in the logzio-telemetry chart. - -There are two possible approaches to the upgrade you can choose from: - -- Before running the `helm upgrade` command, delete the old subcharts resources: `logzio-monitoring-prometheus-pushgateway` deployment and the `logzio-monitoring-prometheus-node-exporter` daemonset. - -- Reinstall the chart. +{@include: ../../_include/general-shipping/k8s.md} diff --git a/docs/shipping/Containers/openshift.md b/docs/shipping/Containers/openshift.md index 4964ccb6..add245b5 100644 --- a/docs/shipping/Containers/openshift.md +++ b/docs/shipping/Containers/openshift.md @@ -64,7 +64,7 @@ Fluentd will fetch all existing logs, as it is not able to ignore older logs. Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ## Setup using custom configuration diff --git a/docs/shipping/Containers/oracle-cloud-infrastructure-container-engine-for-kubernetes.md b/docs/shipping/Containers/oracle-cloud-infrastructure-container-engine-for-kubernetes.md index 981d3480..1d0e9437 100644 --- a/docs/shipping/Containers/oracle-cloud-infrastructure-container-engine-for-kubernetes.md +++ b/docs/shipping/Containers/oracle-cloud-infrastructure-container-engine-for-kubernetes.md @@ -134,7 +134,7 @@ kubectl apply -f https://raw.githubusercontent.com/logzio/logz-docs/master/shipp Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [Kubernetes log shipping troubleshooting]({{site.baseurl}}/user-guide/kubernetes-troubleshooting/). +If you still don't see your logs, see [Kubernetes log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/troubleshooting-fluentd-for-kubernetes-logs). diff --git a/docs/shipping/Data-Store/mongodb.md b/docs/shipping/Data-Store/mongodb.md index 63553470..cce62d8e 100644 --- a/docs/shipping/Data-Store/mongodb.md +++ b/docs/shipping/Data-Store/mongodb.md @@ -166,7 +166,7 @@ fluentd -c ./fluent.conf --gemfile ./Gemfile Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). You can search for `type:mongodb-fluentd` to filter for your MongoDB logs. Your logs should be already parsed thanks to the Logz.io preconfigured parsing pipeline. -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ## Metrics diff --git a/docs/shipping/GCP/gcp-bigquery-data-transfer-service.md b/docs/shipping/GCP/gcp-bigquery-data-transfer-service.md index 4ca3af77..9437789a 100644 --- a/docs/shipping/GCP/gcp-bigquery-data-transfer-service.md +++ b/docs/shipping/GCP/gcp-bigquery-data-transfer-service.md @@ -9,7 +9,7 @@ logo: https://logzbucket.s3.eu-west-1.amazonaws.com/logz-docs/shipper-logos/bigq logs_dashboards: [] logs_alerts: [] logs2metrics: [] -metrics_dashboards: ['1Pm3OYbu1MRGoELc2qhxQ1'] +metrics_dashboards: [] metrics_alerts: [] drop_filter: [] --- diff --git a/docs/shipping/GCP/gcp-workflows.md b/docs/shipping/GCP/gcp-workflows.md index 2ae5b4a1..93d42118 100644 --- a/docs/shipping/GCP/gcp-workflows.md +++ b/docs/shipping/GCP/gcp-workflows.md @@ -9,7 +9,7 @@ logo: https://logzbucket.s3.eu-west-1.amazonaws.com/logz-docs/shipper-logos/work logs_dashboards: [] logs_alerts: [] logs2metrics: [] -metrics_dashboards: ['1Pm3OYbu1MRGoELc2qhxQ1'] +metrics_dashboards: [] metrics_alerts: [] drop_filter: [] --- diff --git a/docs/shipping/Load-Balancer/haproxy.md b/docs/shipping/Load-Balancer/haproxy.md index dd28764c..1a2bd8b9 100644 --- a/docs/shipping/Load-Balancer/haproxy.md +++ b/docs/shipping/Load-Balancer/haproxy.md @@ -103,7 +103,7 @@ sudo service rsyslog restart Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ## Metrics diff --git a/docs/shipping/Network/cloudflare.md b/docs/shipping/Network/cloudflare.md index fbb7f064..a648a1d0 100644 --- a/docs/shipping/Network/cloudflare.md +++ b/docs/shipping/Network/cloudflare.md @@ -52,7 +52,7 @@ Use [our procedure](https://docs.logz.io/shipping/log-sources/s3-bucket.html#con Give your Cloudflare data some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your data, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your data, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/Network/vpc.md b/docs/shipping/Network/vpc.md index 7cf4082b..03387fe8 100644 --- a/docs/shipping/Network/vpc.md +++ b/docs/shipping/Network/vpc.md @@ -19,7 +19,7 @@ VPC Flow Logs is a feature that enables you to capture information about the IP **Before you begin, you'll need**: -* `s3:ListBucket` and `s3:GetObject` [permissions](https://docs.logz.io/user-guide/give-aws-access-with-iam-roles/) for the required S3 bucket +* `s3:ListBucket` and `s3:GetObject` [permissions](/docs/user-guide/admin/give-aws-access-with-iam-roles) for the required S3 bucket * {@include: ../../_include/log-shipping/s3-bucket-file-order.md} @@ -48,6 +48,6 @@ Log into the app to use the dedicated Logz.io [configuration wizard](https://app Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/Operating-Systems/linux.md b/docs/shipping/Operating-Systems/linux.md index 05fbb0ef..474e91df 100644 --- a/docs/shipping/Operating-Systems/linux.md +++ b/docs/shipping/Operating-Systems/linux.md @@ -202,7 +202,7 @@ The above assumes the following defaults: Give your logs some time to get from your system to ours, and then [open Open Search Dashboards](https://app.logz.io/#/dashboard/osd). You can search for `type:syslog` to filter for your logs. -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/Operating-Systems/windows.md b/docs/shipping/Operating-Systems/windows.md index 5ef73aef..161eedae 100644 --- a/docs/shipping/Operating-Systems/windows.md +++ b/docs/shipping/Operating-Systems/windows.md @@ -9,7 +9,7 @@ logo: https://logzbucket.s3.eu-west-1.amazonaws.com/logz-docs/shipper-logos/wind logs_dashboards: [] logs_alerts: ['72Yry8pK5OfiGdPOV2y9RZ', '4Mkw0OICZz7xnZZjlGWX9x'] logs2metrics: [] -metrics_dashboards: ['1Pm3OYbu1MRGoELc2qhxQ1','7vydxtpnlKLILHIGK4puX5'] +metrics_dashboards: ['7vydxtpnlKLILHIGK4puX5'] metrics_alerts: ['4GVNTAqeH4lSRQBfN7dCXQ'] drop_filter: [] --- diff --git a/docs/shipping/Other/beats.md b/docs/shipping/Other/beats.md index 26ab91fb..db67a8a2 100644 --- a/docs/shipping/Other/beats.md +++ b/docs/shipping/Other/beats.md @@ -77,7 +77,7 @@ Start or restart your Beats shipper for the changes to take effect. Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ## Windows @@ -158,7 +158,7 @@ Start or restart your Beats shipper for the changes to take effect. Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/Other/burrow.md b/docs/shipping/Other/burrow.md index 2c07c8f6..bac2a643 100644 --- a/docs/shipping/Other/burrow.md +++ b/docs/shipping/Other/burrow.md @@ -9,7 +9,7 @@ logo: https://logzbucket.s3.eu-west-1.amazonaws.com/logz-docs/shipper-logos/kafk logs_dashboards: [] logs_alerts: [] logs2metrics: [] -metrics_dashboards: ['1Pm3OYbu1MRGoELc2qhxQ1'] +metrics_dashboards: [] metrics_alerts: [] drop_filter: [] --- diff --git a/docs/shipping/Other/curl.md b/docs/shipping/Other/curl.md index 5a6bbe49..1cb60fbd 100644 --- a/docs/shipping/Other/curl.md +++ b/docs/shipping/Other/curl.md @@ -52,7 +52,7 @@ cat /path/to/log/file | curl -X POST "https://<>:8071?token=< td-agent Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). Fluentd can receive and concatenate multiline logs. To do this, you need to add a parser and concatenation plugin to your Fluentd configuration. diff --git a/docs/shipping/Other/fpm.md b/docs/shipping/Other/fpm.md index fa51eb61..5cf63744 100644 --- a/docs/shipping/Other/fpm.md +++ b/docs/shipping/Other/fpm.md @@ -9,7 +9,7 @@ logo: https://logzbucket.s3.eu-west-1.amazonaws.com/logz-docs/shipper-logos/phpf logs_dashboards: [] logs_alerts: [] logs2metrics: [] -metrics_dashboards: ['1Pm3OYbu1MRGoELc2qhxQ1'] +metrics_dashboards: [] metrics_alerts: [] drop_filter: [] --- diff --git a/docs/shipping/Other/heroku.md b/docs/shipping/Other/heroku.md index 0cdfbcbd..ed1d1919 100644 --- a/docs/shipping/Other/heroku.md +++ b/docs/shipping/Other/heroku.md @@ -62,7 +62,7 @@ heroku drains:add "https://<>:8081?token=<>&< Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd) to confirm you're shipping logs. -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). ## Metrics diff --git a/docs/shipping/Other/intercom.md b/docs/shipping/Other/intercom.md index ce74844f..4a94bc0b 100644 --- a/docs/shipping/Other/intercom.md +++ b/docs/shipping/Other/intercom.md @@ -57,6 +57,6 @@ Deploy this integration to ship Intercom events from your Intercom account to Lo Give your data some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). diff --git a/docs/shipping/Other/invoke-restmethod.md b/docs/shipping/Other/invoke-restmethod.md index 6a990da6..869953b5 100644 --- a/docs/shipping/Other/invoke-restmethod.md +++ b/docs/shipping/Other/invoke-restmethod.md @@ -54,4 +54,4 @@ Invoke-RestMethod -method POST -Uri https://<>:8071?token=<" + ] + }, + { + "Effect": "Allow", + "Action": [ + "s3:GetObject" + ], + "Resource": [ + "arn:aws:s3:::/*" + ] + } + ] +} +``` + +* Replace `` with the name of your S3 bucket. + +:::note +Note that the ListBucket permission is set to the entire bucket and the GetObject permission ends with a /* suffix, so we can get files in subdirectories. +::: + +### Create the user + +Browse to the [IAM users](https://console.aws.amazon.com/iam/home#/users) +and click **Add user**. +You're taken to the _Add user_ wizard. + +![Create an IAM role for another AWS account](https://dytvr9ot2sszz.cloudfront.net/logz-docs/aws/iam--add-user.png) + +Assign a **User name**. + +Under _Select AWS access type_, select **Programmatic access**. + +Click **Next: Permissions** to continue. + +### Create the policy + +In the _Set permissions_ section, click **Attach existing policies directly > Create policy**. +The _Create policy_ page loads in a new tab. + +![Create policy](https://dytvr9ot2sszz.cloudfront.net/logz-docs/aws/create-policy-visual-editor.png) + +Set these permissions: + +* **Service**: + Choose **S3** +* **Actions**: + Select **List > ListBucket** and **Read > GetObject** +* **Resources > bucket**: + Click **Add ARN** to open the _Add ARN_ dialog. + Type the intended **Bucket name**, and then click **Add**. +* **Resources > object**: + Click **Add ARN** to open the _Add ARN(s)_ dialog. + Add the intended **Bucket name**, + then select **Object name > Any**. + Click **Add**. + +Click **Review policy** to continue. + +Give the policy a **Name** and optional **Description**, and then click **Create policy**. + +Remember the policy's name—you'll need this in the next step. + +Close the tab to return to the _Add user_ page. + +### Attach the policy to the user + +Click refresh, and then type your new policy's name in the search box. + +Find your policy in the filtered list and select its check box. + +Click **Next: Tags**, +and then click **Next: Review** to continue to the _Review_ screen. + +### Finalize the user + +Give the user a **Name** and optional **Description**, +and then click **Create user**. + +You're taken to a success page. + +### Add the bucket to Logz.io + +Add the **S3 bucket name** and **Prefix** + +Copy the _Access key ID_ and _Secret access key_, or click **Download .csv**. + +In Logz.io, paste the **Access key** and **Secret key**, +and then click **Save**. diff --git a/docs/user-guide/admin/give-aws-access-with-iam-roles.md b/docs/user-guide/admin/give-aws-access-with-iam-roles.md new file mode 100644 index 00000000..46671752 --- /dev/null +++ b/docs/user-guide/admin/give-aws-access-with-iam-roles.md @@ -0,0 +1,331 @@ +--- +sidebar_position: 9 +title: Give AWS access with IAM roles +image: https://dytvr9ot2sszz.cloudfront.net/logz-docs/social-assets/docs-social.jpg +description: Connect Logz.io to your AWS account to control IAM roles +keywords: [AWS, IAM roles, Logz.io aws] +--- + + + + +You can connect Logz.io to your AWS account more securely using IAM roles. + +This gives Logz.io the appropriate level of access +while keeping your AWS account secure. + + +## Connecting an S3 bucket to Logz.io {#grant-access-to-an-s3-bucket} + + + +### Enable Logz.io to access your S3 bucket + +Logz.io will need the following permissions to your S3 bucket: + +* **s3:ListBucket** - to know which files are in your bucket and to thereby keep track of which files have already been ingested +* **s3:GetObject** - to download your files and ingest them to your account + +To do this, add the following to your IAM policy: + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "s3:ListBucket" + ], + "Resource": [ + "arn:aws:s3:::" + ] + }, + { + "Effect": "Allow", + "Action": [ + "s3:GetObject" + ], + "Resource": [ + "arn:aws:s3:::/*" + ] + } + ] +} +``` + +* Replace `` with the name of your S3 bucket. + +:::note +Note that the ListBucket permission is set to the entire bucket and the GetObject permission ends with a /* suffix, so we can get files in subdirectories. +::: + +### Create a Logz.io-AWS connector + +In your Logz.io app, go to **Send your data**. +Select the relevant AWS resource from the left menu. + +Click **+ Add a bucket** and select the option to **Authenticate with a role** + +![Connect Logz.io to an AWS resource](https://dytvr9ot2sszz.cloudfront.net/logz-docs/access-and-authentication/configure-s3-bucket.png) + +Copy and paste the **Account ID** in your text editor. + +Fill in the form to create a new connector. + +Enter the **S3 bucket name** and, if needed, +the **Prefix** where your logs are stored. + +Click **Get the role policy**. +You can review the role policy to confirm the permissions that will be needed. +Paste the policy in your text editor. + +Keep this information available so you can use it in AWS. + +### Create the IAM Role in AWS + +Go to your [IAM roles](https://console.aws.amazon.com/iam/home#/roles) page in your AWS admin console. + +Click **Create role**. +You're taken to the _Create role_ wizard. + +![Create an IAM role for another AWS account](https://dytvr9ot2sszz.cloudfront.net/logz-docs/aws/iam--create-role.png) + +Click **Another AWS account**. + +Paste the **Account ID** you copied from Logz.io. + +Select **Require external ID**, +and then paste the following value into the **External ID** field: `Logzio:aws:extid:7d420c4cccf77013384751185ac12722` + +Click **Next: Permissions** to continue. + +### Create the policy + +In the _Create role_ screen, click **Create policy**. +The _Create policy_ page loads in a new tab. + +In the **JSON** tab, +replace the default JSON with the policy you copied from Logz.io. + +Click **Review policy** to continue. + +Give the policy a **Name** and optional **Description**, +and then click **Create policy**. + +Remember the policy's name—you'll need this in the next step. + +Close the tab to return to the _Create role_ page. + +### Attach the policy to the role + +Click refresh, and then type your new policy's name in the search box. + +Find your policy in the filtered list and select its check box. + +Click **Next: Tags**, +and then click **Next: Review** to continue to the _Review_ screen. + +### Finalize the role + +Give the role a **Name** and optional **Description**. +We recommend beginning the name with "logzio-" +so that it's clear you're using this role with Logz.io. + +Click **Create role** when you're done. + +### Copy the ARN to Logz.io + +In the _IAM roles_ screen, type your new role's name in the search box. + +Find your role in the filtered list and click it to go to its summary page. + +Copy the role ARN (top of the page). +In Logz.io, paste the ARN in the **Role ARN** field, and then click **Save**. + + +## Migrating to a new external ID {#new-external-id} + +If you previously set up an IAM role with your own external ID, +we recommend updating your Logz.io and AWS configurations +to use a Logz.io-generated external ID. +This adds security to your AWS account +by removing the predictability +of any internal naming conventions +your company might have. + +Before you migrate, +you'll need to know where the existing IAM role is used in Logz.io. +This is because you'll need to replace any +[S3 fetcher](https://app.logz.io/#/dashboard/send-your-data/log-sources/s3-bucket) +and +[Archive & restore](https://app.logz.io/#/dashboard/tools/archive-and-restore) +configurations that use the existing role. + +:::note +In case your S3 bucket is encrypted, you need to add `kms:Decrypt` to the policy on the ARN of the KMS key used to encrypt the bucket. +::: + + + +* **If the role is used in a single Logz.io account**: + You can update the external ID + and replace current Logz.io configurations. + See + [_Migrate to the Logz.io external ID in the same role_](#migrate-with-same-role) + (below). +* **If the role is used with multiple Logz.io accounts**: + You'll need to create a new role for each account + and replace current Logz.io configurations. + See + [_Migrate to new IAM roles_](#migrate-to-new-roles) + (below). + +## Migrate to the Logz.io external ID in the same role {#migrate-with-same-role} + +In this procedure, you'll: + +* Replace Logz.io configurations to use the new external ID +* Update the external ID in your IAM role's trust policy + +Follow this process only if the IAM role is used in a single Logz.io account. + +:::caution Important +When you update your IAM role to the Logz.io external ID, all Logz.io configurations that rely on that role will stop working. Before you begin, make sure you know everywhere your existing IAM role is used in Logz.io. +::: + + +### Delete an S3 configuration from Logz.io + +Choose an +[S3 fetcher](https://app.logz.io/#/dashboard/send-your-data/log-sources/s3-bucket) +or +[Archive & restore](https://app.logz.io/#/dashboard/tools/archive-and-restore) +configuration to replace. + +Copy the **S3 bucket name** and **Role ARN** to your text editor, +and make a note of the **Bucket region**. +If this is an S3 fetcher, copy the path **Prefix** as well, +and make a note of the **Log type**. + +Delete the configuration. + +### Replace the configuration + +If this is for an S3 fetcher, click **Add a bucket**, +and click **Authenticate with a role**. + +![S3 fetcher and archive configuration screens](https://dytvr9ot2sszz.cloudfront.net/logz-docs/archive-and-restore/s3-fetcher-and-archive-config-external-id.png) + +Recreate your configuration with the values you copied in step 1, +and copy the **External ID** (you'll paste it in AWS in the next step). + +### Replace the external ID in your IAM role + +Browse to the [IAM roles](https://console.aws.amazon.com/iam/home#/roles) page. +Open the role used by the configuration you deleted in step 1. + +![IAM role summary page, trust relationships tab](https://dytvr9ot2sszz.cloudfront.net/logz-docs/aws/iam-role-edit-trust-relationship.png) + +Open the **Trust relationships** tab +and click **Edit trust relationship** to open the policy document JSON. + +Find the line with the key `sts:ExternalId`, +and replace the value with the Logz.io external ID you copied in step 2. + +For example, +if your account's external ID is +`logzio:aws:extid:example0nktixxe8q`, +you would see this: + +```text +"sts:ExternalId": "logzio:aws:extid:example0nktixxe8q" +``` + +:::caution Important +Saving the trust policy at this point +will immediately change your role's external ID. +Any other Logz.io configurations that use this role +will stop working until you update them. +::: + +Click **Update Trust Policy** to use the Logz.io external ID for this role. + +### Save the new S3 configuration in Logz.io + +Save the configuration in Logz.io: + +* **For an S3 fetcher**: Click **Save** +* **For Archive & restore**: Click **Start archiving** + +You'll see a success message if Logz.io authenticated and connected to your S3 bucket. + +If the connection failed, +double-check your credentials in Logz.io and AWS. + +### _(If needed)_ Replace other configurations that use this role + +If there are other S3 fetcher or Archive & restore configurations +in this account that use the same role, +replace those configurations with the updated external ID. + +Logz.io generates one external ID per account, +so you won't need to change the role again. + + + +## Migrate to new IAM roles {#migrate-to-new-roles} + +In this procedure, you'll: + +* Create a new IAM role with the new external ID +* Replace Logz.io configurations to use the new role + +You'll repeat this procedure for each Logz.io account +where you need to fetch or archive logs in an S3 bucket. + + +### Delete an S3 configuration from Logz.io + +Choose an +[S3 fetcher](https://app.logz.io/#/dashboard/send-your-data/log-sources/s3-bucket) +or +[Archive & restore](https://app.logz.io/#/dashboard/tools/archive-and-restore) +configuration to replace. + +Copy the **S3 bucket name** to your text editor, +and make a note of the **Bucket region**. +If this is an S3 fetcher, copy the path **Prefix** as well, +and make a note of the **Log type**. + +Delete the configuration. + +### Replace the configuration + +If this is for an S3 fetcher, click **Add a bucket**, +and click **Authenticate with a role**. + +![S3 fetcher and archive configuration screens](https://dytvr9ot2sszz.cloudfront.net/logz-docs/archive-and-restore/s3-fetcher-and-archive-config-external-id.png) + +Recreate your configuration with the values you copied in step 1, +and copy the **External ID** (you'll paste it in AWS later). + +### Set up your new IAM role + +Using the information you copied in step 1, +follow the steps in +[_Grant access to an S3 bucket_](#grant-access-to-an-s3-bucket) +(near the top of this page). + +Continue with this procedure when you're done. + +### _(If needed)_ Replace other configurations that use this role + +If there are other S3 fetcher or Archive & restore configurations +in this account that use the same role, +repeat steps 1 and 2, +and use the role ARN from step 3. + +For configurations in other Logz.io accounts, +repeat this procedure from the beginning. + diff --git a/docs/user-guide/admin/hosting-regions/listener-ip-addresses.md b/docs/user-guide/admin/hosting-regions/listener-ip-addresses.md index 426e02e5..8f748374 100644 --- a/docs/user-guide/admin/hosting-regions/listener-ip-addresses.md +++ b/docs/user-guide/admin/hosting-regions/listener-ip-addresses.md @@ -3,7 +3,7 @@ sidebar_position: 3 title: Listener IP Addresses --- -If you're having trouble sending your data (logs, metrics, and traces) to Logz.io, you may need to open your firewall to Logz.io listener servers. To see if you need to change your firewall configuration, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). +If you're having trouble sending your data (logs, metrics, and traces) to Logz.io, you may need to open your firewall to Logz.io listener servers. To see if you need to change your firewall configuration, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). :::note Send your data to the listener URL, not to individual IP addresses. diff --git a/docs/user-guide/admin/logzio-accounts/accounts.md b/docs/user-guide/admin/logzio-accounts/accounts.md index d15b006b..0d55b9d7 100644 --- a/docs/user-guide/admin/logzio-accounts/accounts.md +++ b/docs/user-guide/admin/logzio-accounts/accounts.md @@ -67,8 +67,8 @@ Once you toggle the 2FA option, you'll receive an email guiding you on setting u Account admins have various options when it comes to managing the account. For further information, check out the following guides: -* [Manage your **Log Management** account](./manage-the-main-account-and-sub-accounts.html#logs) -* [Manage your **Cloud SIEM** account](./manage-the-main-account-and-sub-accounts.html#siem) -* [Manage your **Infrastructure Monitoring** (Metrics) account](./manage-the-main-account-and-sub-accounts.html#metrics) -* [Manage your **Distributed Tracing** account](./manage-the-main-account-and-sub-accounts.html#tracing) -* [Manage your **Timeless** account](./manage-the-main-account-and-sub-accounts.html#timeless) \ No newline at end of file +* [Manage your **Log Management** account](/docs/user-guide/admin/logzio-accounts/manage-the-main-account-and-sub-accounts#logs) +* [Manage your **Cloud SIEM** account](/docs/user-guide/admin/logzio-accounts/manage-the-main-account-and-sub-accounts#siem) +* [Manage your **Infrastructure Monitoring** (Metrics) account](/docs/user-guide/admin/logzio-accounts/manage-the-main-account-and-sub-accounts#metrics) +* [Manage your **Distributed Tracing** account](/docs/user-guide/admin/logzio-accounts/manage-the-main-account-and-sub-accounts#tracing) +* [Manage your **Timeless** account](/docs/user-guide/admin/logzio-accounts/manage-the-main-account-and-sub-accounts#timeless) \ No newline at end of file diff --git a/docs/user-guide/admin/logzio-accounts/manage-the-main-account-and-sub-accounts.md b/docs/user-guide/admin/logzio-accounts/manage-the-main-account-and-sub-accounts.md index 561d52f9..19595a2a 100644 --- a/docs/user-guide/admin/logzio-accounts/manage-the-main-account-and-sub-accounts.md +++ b/docs/user-guide/admin/logzio-accounts/manage-the-main-account-and-sub-accounts.md @@ -1,6 +1,6 @@ --- sidebar_position: 2 -title: Manage Log, Metrics, Tracing, and SIEM accounts +title: Manage Log, Metrics, Tracing, Timeless and SIEM accounts --- diff --git a/docs/user-guide/admin/sso/auth0-sso-guide.md b/docs/user-guide/admin/sso/auth0-sso-guide.md index 722efe74..c4523edb 100644 --- a/docs/user-guide/admin/sso/auth0-sso-guide.md +++ b/docs/user-guide/admin/sso/auth0-sso-guide.md @@ -24,7 +24,7 @@ To kick off this process, send an email to [help@logz.io](mailto:help@logz.io). Write that you want to set up Auth0 SSO for Logz.io. Include these items in the message: -* Your Logz.io [account ID](https://docs.logz.io/user-guide/accounts/finding-your-account-id.html) +* Your Logz.io [account ID](https://app.logz.io/#/dashboard/settings/general) * The last six characters of your [account token](https://app.logz.io/#/dashboard/settings/manage-accounts) The Support Team will respond with the connection information you'll need to give in Auth0. diff --git a/docs/user-guide/admin/sso/aws-sso.md b/docs/user-guide/admin/sso/aws-sso.md index b1e6e4e6..264ccc02 100644 --- a/docs/user-guide/admin/sso/aws-sso.md +++ b/docs/user-guide/admin/sso/aws-sso.md @@ -17,7 +17,7 @@ To set up your AWS SSO, you'll first need to email [help@logz.io](mailto:help@lo Include these items in the message: -* Your Logz.io [account ID]({{site.baseurl}}/user-guide/accounts/finding-your-account-id.html) +* Your Logz.io account ID. * The last six characters of your [account token](https://app.logz.io/#/dashboard/settings/manage-accounts) The Support team will respond with the connection information needed to set up your AWS SSO. diff --git a/docs/user-guide/admin/sso/azure-sso.md b/docs/user-guide/admin/sso/azure-sso.md index a3c24f01..74a800fb 100644 --- a/docs/user-guide/admin/sso/azure-sso.md +++ b/docs/user-guide/admin/sso/azure-sso.md @@ -17,7 +17,7 @@ To kick off this process, send an email to [help@logz.io](mailto:help@logz.io). Write that you want to set up Azure SAML SSO for Logz.io. Include these items in the message: -* Your Logz.io [account ID](https://docs.logz.io/user-guide/accounts/finding-your-account-id.html) +* Your Logz.io [account ID](https://app.logz.io/#/dashboard/settings/general) * The last six characters of your [account token](https://app.logz.io/#/dashboard/settings/manage-accounts) The Support team will respond with the connection information you'll need to give in Azure. diff --git a/docs/user-guide/admin/sso/google-workspace-sso.md b/docs/user-guide/admin/sso/google-workspace-sso.md index ebdf4afe..eef5173b 100644 --- a/docs/user-guide/admin/sso/google-workspace-sso.md +++ b/docs/user-guide/admin/sso/google-workspace-sso.md @@ -16,7 +16,7 @@ To set up your Google Workspace SSO, you'll first need to email [help@logz.io](m Include these items in the message: -* Your Logz.io [account ID](https://docs.logz.io/user-guide/accounts/finding-your-account-id.html) +* Your Logz.io [account ID](https://app.logz.io/#/dashboard/settings/general) * The last six characters of your [account token](https://app.logz.io/#/dashboard/settings/manage-accounts) The Support team will respond with the connection information needed to set up your Google Workspace SSO. @@ -61,15 +61,22 @@ When [creating access groups for Google Workspace](https://support.google.com/a/ First, **[add a custom attribute](https://support.google.com/a/answer/6208725?hl=en)** and link it to your SAML app. -Open your SAML app, navigate to the **Add a new custom attribute** section, and add the following configuration: +Open your SAML app, navigate to the **SAML attribute mapping** section, and click **Configure SAML attribute mapping**. -* Name: groups -* Info type: Text -* Visibility: Visible to user and admin -* Number of values: Multi-value +In the **Attributes** section, add the custom attribute you've just created and set the app attribute of your choice. -![Set group SSO](https://dytvr9ot2sszz.cloudfront.net/logz-docs/sso-providers/google/add-custom-fields.png) +:::note +When using a Custom Attribute, you must update **every user** manually with a string. +::: + +![Set group SSO](https://dytvr9ot2sszz.cloudfront.net/logz-docs/sso-providers/google/google-sso-saml-group.png) + +Next, you can add Group membership information by selecting the relevant groups from your account. + +![Set groups](https://dytvr9ot2sszz.cloudfront.net/logz-docs/sso-providers/google/group-membership.png) + +Click **Save** to apply the changes. -Next, configure the groups attribute to be sent across as part of the SAML login. + \ No newline at end of file diff --git a/docs/user-guide/admin/sso/okta-sso.md b/docs/user-guide/admin/sso/okta-sso.md index 1b287493..f0aa60db 100644 --- a/docs/user-guide/admin/sso/okta-sso.md +++ b/docs/user-guide/admin/sso/okta-sso.md @@ -15,7 +15,7 @@ To kick off this process, send an email to [help@logz.io](mailto:help@logz.io). Write that you want to set up Okta SAML SSO for Logz.io. Include these items in the message: -* Your Logz.io [account ID](https://docs.logz.io/user-guide/accounts/finding-your-account-id.html) +* Your Logz.io [account ID](https://app.logz.io/#/dashboard/settings/general) * The last six characters of your [account token](https://app.logz.io/#/dashboard/settings/manage-accounts) The Support team will respond with the connection information you'll need to give in Okta. diff --git a/docs/user-guide/admin/sso/onelogin-sso.md b/docs/user-guide/admin/sso/onelogin-sso.md index 9c8b9603..c12bdc47 100644 --- a/docs/user-guide/admin/sso/onelogin-sso.md +++ b/docs/user-guide/admin/sso/onelogin-sso.md @@ -14,7 +14,7 @@ To kick off this process, send an email to [help@logz.io](mailto:help@logz.io). Write that you want to set up OneLogin SAML SSO for Logz.io. Include these items in the message: -* Your Logz.io [account ID](https://docs.logz.io/user-guide/accounts/finding-your-account-id.html) +* Your Logz.io [account ID](https://app.logz.io/#/dashboard/settings/general) * The last six characters of your [account token](https://app.logz.io/#/dashboard/settings/manage-accounts) The Support team will respond with the connection information you'll need to give in OneLogin. diff --git a/docs/user-guide/cloud-siem/about-siem/why-siem.md b/docs/user-guide/cloud-siem/about-siem/why-siem.md index 6f545acd..4c701d44 100644 --- a/docs/user-guide/cloud-siem/about-siem/why-siem.md +++ b/docs/user-guide/cloud-siem/about-siem/why-siem.md @@ -10,7 +10,7 @@ Logz.io Cloud SIEM (_Security Information and Event Management_) aggregates secu Cloud SIEM comes fully configured to save your team the effort of configuring hundreds of alerts and dashboards. Each integration includes a set of pre-configured security rules and dashboards that can form the basis of your security operations. You can further edit and expand on any existing rules, dashboards, and reports as you see fit. -Logz.io Cloud SIEM integrates with dozens of security services, including firewalls, end point security, network security, identity management security, and even container security. Step-by-step shipping [instructions]({{site.baseurl}}/user-guide/cloud-siem/integrations/) make it simple to get the data into Logz.io. +Logz.io Cloud SIEM integrates with dozens of security services, including firewalls, end point security, network security, identity management security, and even container security. As soon as you start using Cloud SIEM, your logs will be cross referenced against multiple Threat Intelligence feeds to flag malicious IPs, DNSs, and URLs and identify the method of attack. Cloud SIEM can help your team drastically reduce threat detection time and oversee the security of even the largest environments at scale, regardless of whether they are on-prem, cloud, or hybrid environments. diff --git a/docs/user-guide/cloud-siem/quick-guide/create-jira-ticket-alert.md b/docs/user-guide/cloud-siem/quick-guide/create-jira-ticket-alert.md index b68d0a0c..7d113ed1 100644 --- a/docs/user-guide/cloud-siem/quick-guide/create-jira-ticket-alert.md +++ b/docs/user-guide/cloud-siem/quick-guide/create-jira-ticket-alert.md @@ -45,7 +45,7 @@ To add a pre-configured notification endpoint: * Select **POST** from the **Method** menu. - * Enter the following header into the **Headers** field: `authorization: Basic `. Replace `` with the API token to your Atlassian account. + * Enter the following header into the **Headers** field: `authorization= Basic `. Replace `` with the API token to your Atlassian account. * Add the following code as the payload: diff --git a/docs/user-guide/data-hub/archive-restore/configure-archiving.md b/docs/user-guide/data-hub/archive-restore/configure-archiving.md index 9af3fac1..14614f55 100644 --- a/docs/user-guide/data-hub/archive-restore/configure-archiving.md +++ b/docs/user-guide/data-hub/archive-restore/configure-archiving.md @@ -41,9 +41,9 @@ we recommend authenticating with an IAM role. * To set up an IAM role, see - [_Give AWS access with IAM roles_]({{site.baseurl}}/user-guide/give-aws-access-with-iam-roles/). + [_Give AWS access with IAM roles_](/docs/user-guide/admin/give-aws-access-with-iam-roles). * To set up an access key, see - [_Give AWS access with access keys_]({{site.baseurl}}/user-guide/give-aws-access-with-access-keys/). + [_Give AWS access with access keys_](/docs/user-guide/admin/give-aws-access-with-access-keys). :::caution Important Select a path to the **root of an S3 bucket**, to support data restore options. Data cannot be restored from a sub-bucket path. diff --git a/docs/user-guide/data-hub/archive-restore/restore-archived-logs.md b/docs/user-guide/data-hub/archive-restore/restore-archived-logs.md index d2e4f743..f5ac0425 100644 --- a/docs/user-guide/data-hub/archive-restore/restore-archived-logs.md +++ b/docs/user-guide/data-hub/archive-restore/restore-archived-logs.md @@ -34,7 +34,7 @@ Your existing drop filters **will not apply** when restoring data. Instead, use ::: -### 1. Name your restored account and set the desired time range +### Name your restored account and set the desired time range In the _Restore data_ tab, give your restored account a **Name**, and choose a **Time range** of up to 24 hours. @@ -45,7 +45,7 @@ In the _Restore data_ tab, give your restored account a **Name**, and choose a * There are a few things you need to check before you begin the process. - -### 2. Apply power search and filters +### Apply power search and filters You can control and limit which data you'd like to restore by applying **filters**, using **Power search**, or both. Your restored logs will only include data that matches all of your filters and your exact search term. @@ -85,7 +85,7 @@ Power search lets you apply a text search directly on your archived data before **Before using Power search:** -Power search requires [additional permissions](/user-guide/archive-and-restore/set-s3-permissions.html#add-power-search-permissions) to run. +Power search requires [additional permissions](/docs/user-guide/data-hub/archive-restore/set-s3-permissions/#add-power-search-permissions) to run. @@ -128,7 +128,7 @@ If you want to remove one of the filters you've created, click on the **X** next -### 3. Restore your data +### Restore your data To continue, click on the **Proceed** button. You'll see a summary of your restore settings and be asked to approve them to continue the process. @@ -145,7 +145,7 @@ Once the restored account is ready, you'll receive a notification via email. Your restored account will remain available for 5 days. If you want to explore the data after the restore has expired, you'll have to restore it again. -### 4. Explore the restored account in OpenSearch Dashboards +### Explore the restored account in OpenSearch Dashboards You can view your restored account directly from the Restored accounts page by clicking on the **View logs** option next to the relevant account. diff --git a/docs/user-guide/data-hub/archive-restore/set-s3-permissions.md b/docs/user-guide/data-hub/archive-restore/set-s3-permissions.md index 95788c42..86958de2 100644 --- a/docs/user-guide/data-hub/archive-restore/set-s3-permissions.md +++ b/docs/user-guide/data-hub/archive-restore/set-s3-permissions.md @@ -114,5 +114,44 @@ Logz.io can archive and restore your logs with these credentials. Setting up your Power search permissions: +1. Navigate to your AWS account and search for S3. + + ![Select S3](https://dytvr9ot2sszz.cloudfront.net/logz-docs/power-search/select-s3.png) + +2. Choose the relevant bucket on which you want to apply these permissions. **It should be the same bucket you've used when setting up your S3 permissions.** Once inside, click on Permissions, scroll down to **Bucket policy** and click on **Edit**. + + ![Select S3](https://dytvr9ot2sszz.cloudfront.net/logz-docs/power-search/permission-policy.png) + +3. Paste the following code inside the policy. **Replace the `XXX` with your bucket's name.** + + *If you don't have an existing policy, paste this code inside the editor. Otherwise, add this code to the bottom of the page.* + + ```yaml + { + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Principal": { + "AWS": "arn:aws:iam::406095609952:user/search-archive-restore-user" + }, + "Action": [ + "s3:GetObject", + "s3:ListBucket" + ], + "Resource": [ + "arn:aws:s3:::XXX", #replace XXX with your bucket's name + "arn:aws:s3:::XXX/*" #replace XXX with your bucket's name + ] + } + ] + } + ``` + + + ![Edit bucket policy](https://dytvr9ot2sszz.cloudfront.net/logz-docs/power-search/edit-bucket-policy.png) + +4. Click on **Save changes** to apply the new policy. It might take a few minutes for Logz.io to identify the new policy. + -Once the new policy is updated, you'll be able to [use Power search](/user-guide/archive-and-restore/restore-archived-logs.html#apply-power-search) when restoring archived logs. \ No newline at end of file +Once the new policy is updated, you'll be able to [use Power search](/docs/user-guide/data-hub/archive-restore/restore-archived-logs#apply-power-search-and-filters) when restoring archived logs. \ No newline at end of file diff --git a/docs/user-guide/integrations/api-cookbook/who-am-i.md b/docs/user-guide/integrations/api-cookbook/who-am-i.md index 5d820f7b..3780abfc 100644 --- a/docs/user-guide/integrations/api-cookbook/who-am-i.md +++ b/docs/user-guide/integrations/api-cookbook/who-am-i.md @@ -85,5 +85,5 @@ Copy a valid [API token](https://app.logz.io/#/dashboard/settings/manage-tokens/ You're using a valid API token, but you sent the request to the wrong account region. Change the API region to your account region and try again. -For more information on finding your account's region, see [Account region]({{site.baseurl}}/user-guide/accounts/account-region.html). +For more information on finding your account's region, see [Account region](/docs/user-guide/admin/hosting-regions/account-region). \ No newline at end of file diff --git a/docs/user-guide/integrations/notification-endpoints/endpoints.md b/docs/user-guide/integrations/notification-endpoints/endpoints.md index 365fdab3..c68e93cc 100644 --- a/docs/user-guide/integrations/notification-endpoints/endpoints.md +++ b/docs/user-guide/integrations/notification-endpoints/endpoints.md @@ -62,7 +62,7 @@ name the endpoint, and fill in your Opsgenie API key. :::caution Important -Note that currently only Opsgenie accounts hosted by the US data center are supported. You'll know you're in the US region if you log into your Opsgenie account at https://app.opsgenie.com. If you're hosted in another region, you can configure a [custom endpoint]({{site.baseurl}}/user-guide/integrations/custom-endpoints.html). +Note that currently only Opsgenie accounts hosted by the US data center are supported. You'll know you're in the US region if you log into your Opsgenie account at https://app.opsgenie.com. If you're hosted in another region, you can configure a [custom endpoint](/docs/user-guide/integrations/notification-endpoints/custom-endpoints/). ::: ![Opsgenie endpoints](https://dytvr9ot2sszz.cloudfront.net/logz-docs/notification-endpoints/opsgenie-endpoint07-2021.png) diff --git a/docs/user-guide/k8s-360/overview.md b/docs/user-guide/k8s-360/overview.md index cf941953..a7ed7907 100644 --- a/docs/user-guide/k8s-360/overview.md +++ b/docs/user-guide/k8s-360/overview.md @@ -18,7 +18,7 @@ The platform utilizes Kubernetes' numerous advantages for R&D and dev teams, all To activate your Kubernetes 360 dashboard, connect your Kubernetes data quickly and easily through Logz.io's **[Telemetry Collector](https://app.logz.io/#/dashboard/send-your-data/agent/new)**. -If you already have Kubernetes 360 data in your Logz.io account or prefer connecting Kubernetes manually, follow our **[Kubernetes 360 Prerequisite](./kubernetes-360-pre/)** guide. +If you already have Kubernetes 360 data in your Logz.io account or prefer connecting Kubernetes manually, follow our **[Kubernetes 360 Prerequisite](/docs/user-guide/k8s-360/kubernetes-360-pre)** guide. Once everything is up and running, you can use your Kubernetes 360 application. diff --git a/docs/user-guide/k8s-360/unified-helm-chart.md b/docs/user-guide/k8s-360/unified-helm-chart.md index 8cd7d525..b94a9ac2 100644 --- a/docs/user-guide/k8s-360/unified-helm-chart.md +++ b/docs/user-guide/k8s-360/unified-helm-chart.md @@ -4,189 +4,4 @@ title: Unified Helm Chart --- - - - -The `logzio-monitoring` Helm Chart ships your Kubernetes telemetry (logs, metrics, traces and security reports) to your Logz.io account. - -:::note -Please be aware that this project is presently in its beta stage, and as such, it may undergo alterations. -::: - -:::tip -To get the most out of Kubernetes 360, try out dedicated [dashboard](./kubernetes-360-pre). Log in to your Logz.io account and navigate to the current instructions page [inside the Logz.io app](https://app.logz.io/#/dashboard/send-your-data/collection?tag=all&collection=prometheus-sources). Install the pre-built dashboard to enhance the observability of your metrics. To view the metrics on the main dashboard, log in to your Logz.io Metrics account, and open the [Logz.io Metrics tab](https://app.logz.io/#/dashboard/metrics). -::: - - - -### Default Helm Chart configuration -##### 1. Check if you have any taints on your nodes - -```shell -kubectl get nodes -o json | jq '"\(.items[].metadata.name) \(.items[].spec.taints)"' -``` - -If you want to ship logs from any of the nodes that have a taint, make sure that the taint key values are listed in your in your daemonset/deployment configuration as follows: - -```yaml -tolerations: -- key: - operator: - value: - effect: -``` - -:::node -You need to use `Helm` client with version `v3.9.0` or above. -::: - -##### 2. Add the Helm Chart - -```shell -helm repo add logzio-helm https://logzio.github.io/logzio-helm -helm repo update -``` - -##### 3. Deploy the Chart - -```shell -helm install -n monitoring \ ---set logs.enabled=true \ ---set logzio-fluentd.secrets.logzioShippingToken="<>" \ ---set logzio-fluentd.secrets.logzioListener="<>" \ ---set logzio-fluentd.env_id="<>" \ ---set metricsOrTraces.enabled=true \ ---set logzio-k8s-telemetry.metrics.enabled=true \ ---set logzio-k8s-telemetry.secrets.MetricsToken="<>" \ ---set logzio-k8s-telemetry.secrets.ListenerHost="https://<>:8053" \ ---set logzio-k8s-telemetry.secrets.p8s_logzio_name="<>" \ ---set logzio-k8s-telemetry.traces.enabled=true \ ---set logzio-k8s-telemetry.secrets.TracesToken="<>" \ ---set logzio-k8s-telemetry.secrets.LogzioRegion="<>" \ -logzio-monitoring logzio-helm/logzio-monitoring -``` - -| Parameter | Description | -| --- | --- | -| `<>` | Replace `<>` with the token of the account you want to ship to. | -| `<>` | Replace `<>` with the host [for your region](../../admin/hosting-regions/account-region/#available-regions). For example, `listener.logz.io` if your account is hosted on AWS US East, or `listener-nl.logz.io` if hosted on Azure West Europe. The required port depends whether HTTP or HTTPS is used: HTTP = 8070, HTTPS = 8071. | -| `<>` | Your [metrics shipping token](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping). | -| `<>` | The name for the environment's metrics, to easily identify the metrics for each environment. | -| `<>` | The name for your environment's identifier, to easily identify the telemetry data for each environment. | -| `<>` | Your custom name for the environment's metrics, to easily identify the metrics for each environment. | -| `<>` | Replace `<>` with the [token](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping?product=tracing) of the account you want to ship to. | -| `<>` | Name of your Logz.io traces region e.g `us` or `eu`. You can find your region code in the [Regions and URLs](https://docs.logz.io/user-guide/accounts/account-region.html#regions-and-urls) table. | - -##### 4. Check Logz.io for your data - -Give your data some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). - -Log in to your Logz.io account and navigate to the current instructions page [inside the Logz.io app](https://app.logz.io/#/dashboard/send-your-data/prometheus-sources/). Install the pre-built dashboard to enhance the observability of your metrics. - -To view the metrics on the main dashboard, log in to your Logz.io Metrics account, and open the [Logz.io Metrics tab](https://app.logz.io/#/dashboard/metrics/). - - -### Advanced Helm Chart configuration - -#### Collecting span metrics and Trivy reports - -To enable the `logzio-monitoring` Helm Chart collect span metrics and Trivy reports, add the following command to the default deployment: - -```shell -helm install -n monitoring \ ---set logzio-k8s-telemetry.secrets.SpmToken=<> \ ---set securityReport.enabled=true \ ---set logzio-trivy.env_id="<>" \ ---set logzio-trivy.secrets.logzioShippingToken="<>" \ ---set logzio-trivy.secrets.logzioListener="<>" \ -logzio-monitoring logzio-helm/logzio-monitoring -``` - -| Parameter | Description | -| --- | --- | -| `<>` | Your [span metrics shipping token](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping). | -| `<>` | The name for your environment's identifier, to easily identify the telemetry data for each environment. | -| `<>` | Replace `<>` with the token of the account you want to ship to. | -| `<>` | | - - -#### Further configuration - -You can modify the default `logzio-monitoring` Helm Chart by using the `--set` flag in your `helm install` command: - -| Parameter | Description | Default | -| --- | --- | --- | -| `logs.enabled` | Enable to send k8s logs | `false` | -| `metricsOrTraces` | Enable to send k8s metrics or traces | `false` | - -##### Modifying the configuration for logs - -You can see a full list of the possible configuration values in the [logzio-fluentd Chart folder](https://github.com/logzio/logzio-helm/tree/master/charts/fluentd#configuration). - -If you would like to modify any of the values found in the `logzio-fluentd` folder, use the `--set` flag with the `logzio-fluentd` prefix. - -For instance, if there is a parameter called `someField` in the `logzio-telemetry`'s `values.yaml` file, you can set it by adding the following to the `helm install` command: - -```sh ---set logzio-fluentd.someField="my new value" -``` -You can add `log_type` annotation with a custom value, which will be parsed into a `log_type` field with the same value. - - -##### Modifying the configuration for metrics and traces - -You can see a full list of the possible configuration values in the [logzio-telemetry Chart folder](https://github.com/logzio/logzio-helm/tree/master/charts/logzio-telemetry). - -If you would like to modify any of the values found in the `logzio-telemetry` folder, use the `--set` flag with the `logzio-k8s-telemetry` prefix. - -For instance, if there is a parameter called `someField` in the `logzio-telemetry`'s `values.yaml` file, you can set it by adding the following to the `helm install` command: - - -```sh ---set logzio-k8s-telemetry.someField="my new value" -``` - -#### Sending telemetry data from eks on fargate - -To ship logs from pods running on Fargate, set the `fargateLogRouter.enabled` value to `true`. Doing so will deploy a dedicated `aws-observability` namespace and a `configmap` for the Fargate log router. For more information on EKS Fargate logging, please refer to the [official AWS documentation](https://docs.aws.amazon.com/eks/latest/userguide/fargate-logging.html). - -```shell -helm install -n monitoring \ ---set logs.enabled=true \ ---set logzio-fluentd.fargateLogRouter.enabled=true \ ---set logzio-fluentd.secrets.logzioShippingToken="<>" \ ---set logzio-fluentd.secrets.logzioListener="<>" \ ---set metricsOrTraces.enabled=true \ ---set logzio-k8s-telemetry.metrics.enabled=true \ ---set logzio-k8s-telemetry.secrets.MetricsToken="<>" \ ---set logzio-k8s-telemetry.secrets.ListenerHost="https://<>:8053" \ ---set logzio-k8s-telemetry.secrets.p8s_logzio_name="<>" \ ---set logzio-k8s-telemetry.traces.enabled=true \ ---set logzio-k8s-telemetry.secrets.TracesToken="<>" \ ---set logzio-k8s-telemetry.secrets.LogzioRegion="<>" \ -logzio-monitoring logzio-helm/logzio-monitoring -``` - -| Parameter | Description | -| --- | --- | -| `<>` | | -| `<>` | | -| `<>` | Your [metrics shipping token](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping). | -| `<>` | The name for the environment's metrics, to easily identify the metrics for each environment. | -| `<>` | The name for your environment's identifier, to easily identify the telemetry data for each environment. | -| `<>` | Your custom name for the environment's metrics, to easily identify the metrics for each environment. | -| `<>` | Replace `<>` with the [token](https://app.logz.io/#/dashboard/settings/manage-tokens/data-shipping?product=tracing) of the account you want to ship to. | -| `<>` | Name of your Logz.io traces region e.g `us` or `eu`. You can find your region code in the [Regions and URLs](https://docs.logz.io/user-guide/accounts/account-region.html#regions-and-urls) table. | - -#### Handling image pull rate limit - -In certain situations, such as with spot clusters where pods/nodes are frequently replaced, you may encounter the pull rate limit for images fetched from Docker Hub. This could result in the following error: Y`ou have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limits`. - -To address this issue, you can use the `--set` commands provided below in order to access an alternative image repository: - -```shell ---set logzio-k8s-telemetry.image.repository=ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector-contrib ---set logzio-k8s-telemetry.prometheus-pushgateway.image.repository=public.ecr.aws/logzio/prom-pushgateway ---set logzio-fluentd.image=public.ecr.aws/logzio/logzio-fluentd ---set logzio-trivy.image=public.ecr.aws/logzio/trivy-to-logzio -``` +{@include: ../../_include/general-shipping/k8s.md} \ No newline at end of file diff --git a/docs/user-guide/log-management/cold-tier.md b/docs/user-guide/log-management/cold-tier.md index 26285fb7..695f636d 100644 --- a/docs/user-guide/log-management/cold-tier.md +++ b/docs/user-guide/log-management/cold-tier.md @@ -12,6 +12,10 @@ With Cold Tier, you can seamlessly search cold storage data you've archived, vie To start using Cold Tier, navigate to [Log analytics > Cold Tier](https://app.logz.io/#/dashboard/osd/discover/). +:::caution Important +Cold tier is currently only available for users using **Amazon S3 archive**. +::: +