From 6a8d939710286a57e2a0a514ca18c021b4ac837b Mon Sep 17 00:00:00 2001 From: mirii1994 Date: Mon, 4 Dec 2023 16:51:58 +0200 Subject: [PATCH] update guardduty docs --- docs/shipping/AWS/aws-guardduty.md | 255 +++++++++-------------------- 1 file changed, 74 insertions(+), 181 deletions(-) diff --git a/docs/shipping/AWS/aws-guardduty.md b/docs/shipping/AWS/aws-guardduty.md index 2526fb80..96ed4f81 100644 --- a/docs/shipping/AWS/aws-guardduty.md +++ b/docs/shipping/AWS/aws-guardduty.md @@ -15,197 +15,90 @@ drop_filter: [] --- - -## Manual Lambda configuration - -{@include: ../../_include/log-shipping/note-lambda-test.md} - - - - -### Create a new Kinesis data stream - -If you're not already sending your GuardDuty logs through a Kinesis data stream, create one using the [Kinesis console](https://console.aws.amazon.com/kinesis). - -Save the name of the data stream—you'll need this in the next step. - -### Configure CloudWatch Events - -In the [CloudWatch console](https://console.aws.amazon.com/cloudwatch/) left menu, click **Events > Rules**, and then click **Create rule**. - -In the Event Source panel (on the left), set these options: - -* Choose **Event Pattern**. -* In the **Build event pattern** section, choose **GuardDuty** from the **Service Name** list. - You can choose any **Event Type** that you need. - -In the Targets panel (on the right), click **Add target**, and choose **Kinesis stream**. -Choose the Kinesis data stream from step 1 from the **Stream** list. - -Click **Configure details** (lower right corner). - -### Create a new IAM role - -Create a new IAM role and attach the **AWSLambdaKinesisExecutionRole** policy to the new role. - -### Create a new Lambda function - -This Lambda function will collect CloudWatch logs and sends them to Logz.io in bulk over HTTP. - -Open the AWS Lambda Console, and click **Create function**. -Choose **Author from scratch**, and use this information: - -* **Name**: - We suggest adding the log type to the name, but you can name this function whatever you want. -* **Runtime**: - Choose **Python 3.7** -* **Role**: - Select **Use an existing role**. - Then, select the role that you created in the previous step. It should have the **AWSLambdaKinesisExecutionRole** policy. - -Click **Create Function** (bottom right corner of the page). -After a few moments, you'll see configuration options for your Lambda function. - -You'll need this page later on, so keep it open. - -### Download the Kinesis stream shipper - -Download the latest Kinesis stream shipper zip file from the [Logz.io GitHub page](https://github.com/logzio/logzio_aws_serverless/releases). - -By default, the zip file will be named `logzio-kinesis-0.0.2.zip`. - -### Upload the zip file - -In the _Function_ code section of Lambda, open the **Code entry type** list and select **Upload a .ZIP file**. - -Click **Upload** and select the zip file you created in the previous step (`logzio-kinesis-0.0.2.zip`). - -### Set environment variables - -In the _Environment variables_ section, set your Logz.io account token, URL, and log type, and any other variables that you need to use. - -### Environment variables - -| Parameter | Description | Required/Default | -|---|---|---| -| TOKEN (Required) | Your Logz.io account token. {@include: ../../_include/log-shipping/log-shipping-token.html} | Required | -| REGION | Two-letter region code, or blank for US East (Northern Virginia). This determines your listener URL (where you're shipping the logs to) and API URL. You can find your region code in the [Regions and URLs](https://docs.logz.io/user-guide/accounts/account-region.html#regions-and-urls) table. | Default: *blank* (US East)| -| URL (Deprecated)| Use REGION instead. Protocol, listener host, and port (for example, `https://<>:8071`). {@include: ../../_include/log-shipping/listener-var.html} | Required | -| TYPE | The log type you'll use with this Lambda. This can be a [built-in log type](/docs/user-guide/data-hub/log-parsing/default-parsing/#built-in-log-types), or a custom log type. You should create a new Lambda for each log type you use. | `"guardduty"` | -| FORMAT | `"json"` or `"text"`. If `"json"`, the Lambda function will attempt to parse the message field as JSON and populate the event data with the parsed fields. | `"text"` | -| COMPRESS | Set to `true` to compress logs before sending them. Set to `false` to send uncompressed logs. | `false` | - - -### Configure the function's basic settings - -In Basic settings, we recommend starting with these settings: - -* **Memory**: 512 MB -* **Timeout**: 1 min 0 sec - -:::note -These default settings are just a starting point. Check your Lambda usage regularly, and adjust these values if you need to. +## Logs + +### Create an EventBridge rule + +You'll need to create a new EventBridge rule that will send your GuardDuty findings to a Cloudwatch Log Group. + +1. In your **AWS Console** go to ** Amazon EventBridge** service. +2. In the left Amazon EventBridge menu choose **Rules**, then click on **Create rule**. +3. Insert the name of your new rule, and click **Next**. +4. Scroll down to the **Event pattern** panel. For **AWS service** field, choose **GuardDuty**. For **Event type** field choose **All Events**, and click **Next**. +5. For **Select a target** field choose **CloudWatch log group**. For **Log Group** field, choose the first option (`/aws/events`) and type the name you'd like to give your new log group. Click **Next**. +6. Add tags to your event rule, if you want to. Click **Next**. +7. Review the details and click **Create rule**. + + +### Auto-deploy the Stack in the relevant region + +This integration will deploy a Firehose connection with your AWS services to forward logs to Logz.io +To deploy this project, click the button that matches the region you wish to deploy your Stack to: + +| Region | Deployment | +|------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `us-east-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-us-east-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `us-east-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=us-east-2#/stacks/create/review?templateURL=https://logzio-aws-integrations-us-east-2.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `us-west-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=us-west-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-us-west-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `us-west-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/create/review?templateURL=https://logzio-aws-integrations-us-west-2.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `eu-central-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-central-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-eu-central-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `eu-north-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-north-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-eu-north-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `eu-west-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-eu-west-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `eu-west-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-2#/stacks/create/review?templateURL=https://logzio-aws-integrations-eu-west-2.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `eu-west-3` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-3#/stacks/create/review?templateURL=https://logzio-aws-integrations-eu-west-3.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `sa-east-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=sa-east-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-sa-east-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `ap-northeast-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-northeast-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `ap-northeast-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-2#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-northeast-2.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `ap-northeast-3` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-3#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-northeast-3.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `ap-south-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-south-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-south-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `ap-southeast-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-southeast-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `ap-southeast-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-2#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-southeast-2.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | +| `ca-central-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ca-central-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-ca-central-1.s3.amazonaws.com/firehose-logs/0.0.2/sam-template.yaml&stackName=logzio-firehose¶m_logzioToken=<>¶m_logzioListener=https://aws-firehose-logs-<>) | + +#### Specify stack details + +Specify the stack details as per the table below, check the checkboxes and select **Create stack**. +Add the CloudWatch log group name you created in the first step to field `customLogGroups`. + +| Parameter | Description | Required/Default | +|--------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------| +| `logzioToken` | The [token](https://app.logz.io/#/dashboard/settings/general) of the account you want to ship logs to. | **Required** | +| `logzioListener` | Listener host. | **Required** | +| `logzioType` | The log type you'll use with this Lambda. This can be a [built-in log type](https://docs.logz.io/user-guide/log-shipping/built-in-log-types.html), or a custom log type. | `logzio_firehose` | +| `services` | A comma-seperated list of services you want to collect logs from. Supported options are: `apigateway`, `rds`, `cloudhsm`, `cloudtrail`, `codebuild`, `connect`, `elasticbeanstalk`, `ecs`, `eks`, `aws-glue`, `aws-iot`, `lambda`, `macie`, `amazon-mq`. | - | +| `customLogGroups` | A comma-seperated list of custom log groups you want to collect logs from | - | +| `triggerLambdaTimeout` | The amount of seconds that Lambda allows a function to run before stopping it, for the trigger function. | `60` | +| `triggerLambdaMemory` | Trigger function's allocated CPU proportional to the memory configured, in MB. | `512` | +| `triggerLambdaLogLevel` | Log level for the Lambda function. Can be one of: `debug`, `info`, `warn`, `error`, `fatal`, `panic` | `info` | +| `httpEndpointDestinationIntervalInSeconds` | The length of time, in seconds, that Kinesis Data Firehose buffers incoming data before delivering it to the destination | `60` | +| `httpEndpointDestinationSizeInMBs` | The size of the buffer, in MBs, that Kinesis Data Firehose uses for incoming data before delivering it to the destination | `5` | + + +:::caution Important +AWS limits every log group to have up to 2 subscription filters. If your chosen log group already has 2 subscription filters, the trigger function won't be able to add another one. ::: - - -### Set the Kinesis event trigger - -Find the **Add triggers** list (left side of the Designer panel). -Choose **Kinesis** from this list. - -Below the Designer, you'll see the Configure triggers panel. -Choose the **Kinesis stream** that the Lambda function will watch. - -Click **Add**, and then click **Save** at the top of the page. - -### Check Logz.io for your logs - -Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). -If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). - - - - -## Automated CloudFormation deployment +#### Send logs -{@include: ../../_include/log-shipping/note-lambda-test.md} - - -**Before you begin, you'll need**: -AWS CLI, -an S3 bucket to store the CloudFormation package - - +Give the stack a few minutes to be deployed. -### Create a new Kinesis data stream +Once new logs are added to your chosen log group, they will be sent to your Logz.io account. -If you're not already sending your GuardDuty logs through a Kinesis data stream, create one using the [Kinesis console](https://console.aws.amazon.com/kinesis). +Your GuardDuty logs will be sent in accordance to your GuardDuty configuration. +GuardDuty publishes its findings to EventBridge every 6 hours. If you want to configure it differently: +1. Go to your GuardDuty settings. +2. Scroll down to **Findings export options**. Click on **Edit** of **Frequency**. +3. Choose your prefered exporting frequency to export GuardDuty findings. -Select the button below according to the region where you need to deploy the stack. +You can export a sample finding by going to GuardDuty **settings** and clicking the **Generate sample findings**. -| REGION | DEPLOYMENT | -| ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `us-east-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create/template?templateURL=https://logzio-aws-integrations-us-east-1.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `us-east-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=us-east-2#/stacks/create/template?templateURL=https://logzio-aws-integrations-us-east-2.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `us-west-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-1#/stacks/create/template?templateURL=https://logzio-aws-integrations-us-west-1.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `us-west-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-2#/stacks/create/template?templateURL=https://logzio-aws-integrations-us-west-2.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `eu-central-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-central-1#/stacks/create/template?templateURL=https://logzio-aws-integrations-eu-central-1.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `eu-north-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-north-1#/stacks/create/template?templateURL=https://logzio-aws-integrations-eu-north-1.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `eu-west-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-1#/stacks/create/template?templateURL=https://logzio-aws-integrations-eu-west-1.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `eu-west-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-2#/stacks/create/template?templateURL=https://logzio-aws-integrations-eu-west-2.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `eu-west-3` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-3#/stacks/create/template?templateURL=https://logzio-aws-integrations-eu-west-3.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `sa-east-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=sa-east-1#/stacks/create/template?templateURL=https://logzio-aws-integrations-sa-east-1.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `ca-central-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ca-central-1#/stacks/create/template?templateURL=https://logzio-aws-integrations-ca-central-1.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-shipper¶m_LogzioTOKEN=<>) | -| `ap-northeast-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-1#/stacks/create/template?templateURL=https://logzio-aws-integrations-ap-northeast-1.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `ap-northeast-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-2#/stacks/create/template?templateURL=https://logzio-aws-integrations-ap-northeast-2.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `ap-northeast-3` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-3#/stacks/create/template?templateURL=https://logzio-aws-integrations-ap-northeast-3.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `ap-south-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-south-1#/stacks/create/template?templateURL=https://logzio-aws-integrations-ap-south-1.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `ap-southeast-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-1#/stacks/create/template?templateURL=https://logzio-aws-integrations-ap-southeast-1.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | -| `ap-southeast-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-2#/stacks/create/template?templateURL=https://logzio-aws-integrations-ap-southeast-2.s3.amazonaws.com/aws-kinesis/0.0.2/auto-deployment.yaml&stackName=guardduty-log-shipper¶m_LogzioTOKEN=<>) | - -![Create stack](https://dytvr9ot2sszz.cloudfront.net/logz-docs/guardduty/first.png) - -Keep the default setting in the **Create stack** screen and select **Next**. - -### Specify the stack details - -![Specify stack details](https://dytvr9ot2sszz.cloudfront.net/logz-docs/guardduty/second.png) - - -Specify the stack details as per the table below and select **Next**. - - -| Parameter | Description | Required/Default | -| ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------- | -| LogzioTOKEN | Your Logz.io account token. {@include: ../../_include/log-shipping/log-shipping-token.html} | Required | -| KinesisStream | The name of the Kinesis stream where this function will listen for updates. | Required | -| LogzioREGION | Two-letter region code, or blank for US East (Northern Virginia). This determines your listener URL (where you're shipping the logs to) and API URL. You can find your region code in the [Regions and URLs](https://docs.logz.io/user-guide/accounts/account-region.html#regions-and-urls) table. | Default: _blank_ (US East) | -| LogzioURL (Deprecated) | Use LogzioREGION instead. Protocol, listener host, and port (for example, `https://<>:8071`). {@include: ../../_include/log-shipping/listener-var.html} | Required | -| LogzioTYPE | The log type you'll use with this Lambda. This can be a [built-in log type](/docs/user-guide/data-hub/log-parsing/default-parsing/#built-in-log-types), or a custom log type. You should create a new Lambda for each log type you use. | `guardduty` | -| LogzioFORMAT | `"json"` or `"text"`. If `"json"`, the Lambda function will attempt to parse the message field as JSON and populate the event data with the parsed fields. | `"text"` | -| LogzioCOMPRESS | Set to `true` to compress logs before sending them. Set to `false` to send uncompressed logs. | `false` | -| KinesisStreamBatchSize | The largest number of records to read from your stream at one time. | `100` | -| KinesisStreamStartingPosition | The position in the stream to start reading from. For more information, see [ShardIteratorType](https://docs.aws.amazon.com/kinesis/latest/APIReference/API_GetShardIterator.html) in the Amazon Kinesis API Reference. | `"LATEST"` | - -### Configure the stack options - -![Configure stack options](https://dytvr9ot2sszz.cloudfront.net/logz-docs/guardduty/third.png) - -Specify the **Key** and **Value** parameters for the **Tags** and select **Next**. - -### Review the deployment - -![Review deployment](https://dytvr9ot2sszz.cloudfront.net/logz-docs/guardduty/fourth.png) - -Confirm that you acknowledge that AWS CloudFormation might create IAM resources and select **Create stack**. +:::caution Important +If you've used the `services` field, you'll have to **wait 6 minutes** before creating new log groups for your chosen services. This is due to cold start and custom resource invocation, that can cause the Lambda to behave unexpectedly. +::: -### Check Logz.io for your logs +#### Check Logz.io for your logs Give your logs some time to get from your system to ours, and then open [Open Search Dashboards](https://app.logz.io/#/dashboard/osd). If you still don't see your logs, see [log shipping troubleshooting](/docs/user-guide/log-management/troubleshooting/log-shipping-troubleshooting/). - - - \ No newline at end of file