From f49f54a69982e4554a39e4277fa06336a6c7ee07 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E2=80=9Cnico-shishkin=E2=80=9D?= <“nicoshishkinatlogz@outlook.com”> Date: Wed, 8 Nov 2023 12:43:50 +0100 Subject: [PATCH] DOC-632 --- docs/shipping/AWS/aws-amplify.md | 92 +++++++++++++++++++++++++++++++- 1 file changed, 91 insertions(+), 1 deletion(-) diff --git a/docs/shipping/AWS/aws-amplify.md b/docs/shipping/AWS/aws-amplify.md index 136c86cc..d62c3e55 100644 --- a/docs/shipping/AWS/aws-amplify.md +++ b/docs/shipping/AWS/aws-amplify.md @@ -1,7 +1,7 @@ --- id: AWS-Amplify title: AWS Amplify -overview: This is an AWS Lambda function that collects Amplify access logs and sends them to Logz.io in bulk over HTTP. +overview: This is an integration that collects Amplify access logs and sends them to Logz.io. product: ['logs'] os: [] filters: ['AWS', 'CI/CD'] @@ -14,6 +14,96 @@ metrics_alerts: [] drop_filter: [] --- + +:::tip +For a much easier and more efficient way to collect and send telemetry, consider using the [Logz.io telemetry collector](https://app.logz.io/#/dashboard/send-your-data/agent/new). +::: + +## Logs + +Deploy this integration to forward your AWS Amplify logs to Logz.io using AWS Firehose + + +### Auto-deploy the Stack in the relevant region + +This integration will deploy a Firehose connection with your AWS services to forward logs to Logz.io +To deploy this project, click the button that matches the region you wish to deploy your Stack to: + +| Region | Deployment | +|------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `us-east-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-us-east-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `us-east-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=us-east-2#/stacks/create/review?templateURL=https://logzio-aws-integrations-us-east-2.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `us-west-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=us-west-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-us-west-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `us-west-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/create/review?templateURL=https://logzio-aws-integrations-us-west-2.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `eu-central-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-central-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-eu-central-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `eu-north-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-north-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-eu-north-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `eu-west-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-eu-west-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `eu-west-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-2#/stacks/create/review?templateURL=https://logzio-aws-integrations-eu-west-2.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `eu-west-3` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-3#/stacks/create/review?templateURL=https://logzio-aws-integrations-eu-west-3.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `sa-east-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=sa-east-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-sa-east-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `ap-northeast-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-northeast-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `ap-northeast-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-2#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-northeast-2.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `ap-northeast-3` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-3#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-northeast-3.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `ap-south-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-south-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-south-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `ap-southeast-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-southeast-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `ap-southeast-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-2#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-southeast-2.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `ca-central-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ca-central-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-ca-central-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | + +#### Specify stack details + +Specify the stack details as per the table below, check the checkboxes and select **Create stack**. + +| Parameter | Description | Required/Default | +|--------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------| +| `logzioToken` | The [token](https://app.logz.io/#/dashboard/settings/general) of the account you want to ship logs to. | **Required** | +| `logzioListener` | Listener host. | **Required** | +| `logzioType` | The log type you'll use with this Lambda. This can be a [built-in log type](https://docs.logz.io/user-guide/log-shipping/built-in-log-types.html), or a custom log type. | `logzio_firehose` | +| `services` | A comma-seperated list of services you want to collect logs from. Supported options are: `apigateway`, `rds`, `cloudhsm`, `cloudtrail`, `codebuild`, `connect`, `elasticbeanstalk`, `ecs`, `eks`, `aws-glue`, `aws-iot`, `lambda`, `macie`, `amazon-mq`. | - | +| `customLogGroups` | A comma-seperated list of custom log groups you want to collect logs from | - | +| `triggerLambdaTimeout` | The amount of seconds that Lambda allows a function to run before stopping it, for the trigger function. | `60` | +| `triggerLambdaMemory` | Trigger function's allocated CPU proportional to the memory configured, in MB. | `512` | +| `triggerLambdaLogLevel` | Log level for the Lambda function. Can be one of: `debug`, `info`, `warn`, `error`, `fatal`, `panic` | `info` | +| `httpEndpointDestinationIntervalInSeconds` | The length of time, in seconds, that Kinesis Data Firehose buffers incoming data before delivering it to the destination | `60` | +| `httpEndpointDestinationSizeInMBs` | The size of the buffer, in MBs, that Kinesis Data Firehose uses for incoming data before delivering it to the destination | `5` | + + +:::caution Important +AWS limits every log group to have up to 2 subscription filters. If your chosen log group already has 2 subscription filters, the trigger function won't be able to add another one. +::: + + + + + +#### Send logs + +Give the stack a few minutes to be deployed. + +Once new logs are added to your chosen log group, they will be sent to your Logz.io account. + + +:::caution Important +If you've used the `services` field, you'll have to **wait 6 minutes** before creating new log groups for your chosen services. This is due to cold start and custom resource invocation, that can cause the Lambda to behave unexpectedly. +::: + +#### Check Logz.io for your logs + +Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). + +If you still don't see your logs, see [log shipping troubleshooting]({{site.baseurl}}/user-guide/log-shipping/log-shipping-troubleshooting.html). + + + + + + + + + + + + + This is an AWS Lambda function that collects Amplify access logs and sends them to Logz.io in bulk over HTTP. ## Configuration with a Lambda function