diff --git a/docs/shipping/AWS/aws-cloudformation.md b/docs/shipping/AWS/aws-cloudformation.md new file mode 100644 index 00000000..7f221c34 --- /dev/null +++ b/docs/shipping/AWS/aws-cloudformation.md @@ -0,0 +1,93 @@ +--- +id: AWS-CloudFormation +title: AWS CloudFormation +overview: Send your Amazon CloudFront metrics to Logz.io. +product: ['logs' , 'metrics'] +os: [] +filters: ['AWS', 'Network'] +logo: https://logzbucket.s3.eu-west-1.amazonaws.com/logz-docs/shipper-logos/aws-cloudformation.svg +logs_dashboards: [] +logs_alerts: [] +logs2metrics: [] +metrics_dashboards: [] +metrics_alerts: [] +drop_filter: [] +--- + + + +:::note +For a much easier and more efficient way to collect and send metrics, consider using the [Logz.io telemetry collector](https://app.logz.io/#/dashboard/send-your-data/agent/new). +::: + + + + +This CloudFormation template allows shipping AWS Logs and Metrics to logz.io. + +It will deploy the following projects: + +- AWS metrics to Logzio (See here the function's repo)[https://github.com/logzio/cloudwatch-metrics-helpers]. +- AWS logs to Logz.io (See here the function's repo)[https://github.com/logzio/cloudwatch-shipper-trigger]. + + +##### Instructions + +To deploy this integrion, click the button that matches the region you wish to deploy your Stack to: + +| Region | Deployment | +|------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `us-east-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-us-east-1.s3.amazonaws.com/cloudwatch-logs-metrics/0.0.1/sam-template.yaml&stackName=logs-metrics-logzio) | +| `us-east-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=us-east-2#/stacks/create/review?templateURL=https://logzio-aws-integrations-us-east-2.s3.amazonaws.com/cloudwatch-logs-metrics/0.0.1/sam-template.yaml&stackName=logs-metrics-logzio) | +| `us-west-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=us-west-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-us-west-1.s3.amazonaws.com/cloudwatch-logs-metrics/0.0.1/sam-template.yaml&stackName=logs-metrics-logzio) | +| `us-west-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=us-west-2#/stacks/create/review?templateURL=https://logzio-aws-integrations-us-west-2.s3.amazonaws.com/cloudwatch-logs-metrics/0.0.1/sam-template.yaml&stackName=logs-metrics-logzio) | +| `eu-central-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-central-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-eu-central-1.s3.amazonaws.com/cloudwatch-logs-metrics/0.0.1/sam-template.yaml&stackName=logs-metrics-logzio) | +| `eu-north-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-north-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-eu-north-1.s3.amazonaws.com/cloudwatch-logs-metrics/0.0.1/sam-template.yaml&stackName=logs-metrics-logzio) | +| `eu-west-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-eu-west-1.s3.amazonaws.com/cloudwatch-logs-metrics/0.0.1/sam-template.yaml&stackName=logs-metrics-logzio) | +| `eu-west-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-2#/stacks/create/review?templateURL=https://logzio-aws-integrations-eu-west-2.s3.amazonaws.com/cloudwatch-logs-metrics/0.0.1/sam-template.yaml&stackName=logs-metrics-logzio) | +| `eu-west-3` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=eu-west-3#/stacks/create/review?templateURL=https://logzio-aws-integrations-eu-west-3.s3.amazonaws.com/cloudwatch-logs-metrics/0.0.1/sam-template.yaml&stackName=logs-metrics-logzio) | +| `sa-east-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=sa-east-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-sa-east-1.s3.amazonaws.com/cloudwatch-logs-metrics/0.0.1/sam-template.yaml&stackName=logs-metrics-logzio) | +| `ap-northeast-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-northeast-1.s3.amazonaws.com/cloudwatch-logs-metrics/0.0.1/sam-template.yaml&stackName=logs-metrics-logzio) | +| `ap-northeast-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-2#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-northeast-2.s3.amazonaws.com/cloudwatch-logs-metrics/0.0.1/sam-template.yaml&stackName=logs-metrics-logzio) | +| `ap-northeast-3` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-northeast-3#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-northeast-3.s3.amazonaws.com/cloudwatch-logs-metrics/0.0.1/sam-template.yaml&stackName=logs-metrics-logzio) | +| `ap-south-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-south-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-south-1.s3.amazonaws.com/cloudwatch-logs-metrics/0.0.1/sam-template.yaml&stackName=logs-metrics-logzio) | +| `ap-southeast-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-southeast-1.s3.amazonaws.com/cloudwatch-logs-metrics/0.0.1/sam-template.yaml&stackName=logs-metrics-logzio) | +| `ap-southeast-2` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ap-southeast-2#/stacks/create/review?templateURL=https://logzio-aws-integrations-ap-southeast-2.s3.amazonaws.com/cloudwatch-logs-metrics/0.0.1/sam-template.yaml&stackName=logs-metrics-logzio) | +| `ca-central-1` | [![Deploy to AWS](https://dytvr9ot2sszz.cloudfront.net/logz-docs/lights/LightS-button.png)](https://console.aws.amazon.com/cloudformation/home?region=ca-central-1#/stacks/create/review?templateURL=https://logzio-aws-integrations-ca-central-1.s3.amazonaws.com/cloudwatch-logs-metrics/0.0.1/sam-template.yaml&stackName=logs-metrics-logzio) | + + +| Parameter | Description | Required/Default | +|------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------| +| `logzioLogsToken` | Replace `<>` with the [token](https://app.logz.io/#/dashboard/settings/general) of the account you want to ship to. | **Required** | +| `logzioListener` | The Logz.io listener URL for your region. (For more details, see the [regions page](https://docs.logz.io/user-guide/accounts/account-region.html). For example - `https://listener.logz.io:8053` | **Required** | +| `logzioMetricsToken` | Your Logz.io metrics shipping token. | **Required** | +| `awsNamespaces` | Comma-separated list of the AWS namespaces you want to monitor. See [this list]((https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/aws-services-cloudwatch-metrics.html) of namespaces. If you want to automatically add all namespaces, use value `all-namespaces`. | **Required** | +| `logzioMetricsDestination` | Your Logz.io destination URL. | **Required** | +| `logzioLogsType` | The log type you'll use with this Lambda. This can be a [built-in log type](https://docs.logz.io/user-guide/log-shipping/built-in-log-types.html), or a custom log type.
You should create a new Lambda for each log type you use. | Default: `logzio_cloudwatch_lambda` | +| `logzioLogsFormat` | `json` or `text`. If `json`, the Lambda function will attempt to parse the message field as JSON and populate the event data with the parsed fields. | Default: `text` | +| `logzioLogsCompress` | Set to `true` to compress logs before sending them. Set to `false` to send uncompressed logs. | Default: `true` | +| `logzioLogsEnrich` | Enrich CloudWatch events with custom properties, formatted as `key1=value1;key2=value2`. | - | +| `shipperLambdaTimeout` | The number of seconds that Lambda allows a function to run before stopping it, for the shipper function. | Default: `60` | +| `shipperLambdaMemory` | Shipper function's allocated CPU proportional to the memory configured, in MB. | Default: `512` | +| `services` | A comma-seperated list of services you want to collect logs from. Supported options are: `apigateway`, `rds`, `cloudhsm`, `cloudtrail`, `codebuild`, `connect`, `elasticbeanstalk`, `ecs`, `eks`, `aws-glue`, `aws-iot`, `lambda`, `macie`, `amazon-mq`. | - | +| `customLogGroups` | A comma-seperated list of custom log groups you want to collect logs from | - | +| `triggerLambdaTimeout` | The amount of seconds that Lambda allows a function to run before stopping it, for the trigger function. | Default: `60` | +| `triggerLambdaMemory` | Trigger function's allocated CPU proportional to the memory configured, in MB. | Default: `512` | +| `triggerLambdaLogLevel` | Log level for the Lambda function. Can be one of: `debug`, `info`, `warn`, `error`, `fatal`, `panic` | Default: `info` | +| `httpEndpointDestinationIntervalInSeconds` | The length of time, in seconds, that Kinesis Data Firehose buffers incoming data before delivering it to the destination. | Default: `60` | +| `httpEndpointDestinationSizeInMBs` | The size of the buffer, in MBs, that Kinesis Data Firehose uses for incoming data before delivering it to the destination. | Default: `5` | + + + +:::note +AWS limits every log group to have up to 2 subscription filters. If your chosen log group already has 2 subscription filters, the trigger function won't be able to add another one. +::: + +:::note +If you've used the `services` field, you'll have to **wait 6 minutes** before creating new log groups for your chosen services. This is due to cold start and custom resource invocation, that can cause the cause Lambda to behave unexpectedly. +::: + + +##### Check Logz.io for your metrics + +Give your data some time to get from your system to ours, then log in to your Logz.io Metrics account, and open [the Logz.io Metrics tab](https://app.logz.io/#/dashboard/metrics/). diff --git a/static/_redirects b/static/_redirects index b933511a..738dc7de 100644 --- a/static/_redirects +++ b/static/_redirects @@ -553,4 +553,5 @@ /shipping/log-sources/vector.html /docs/shipping/Other/Vector-data /shipping/log-sources/vpc.html /docs/shipping/Network/VPC-network /shipping/log-sources/windows.html /docs/shipping/Operating-Systems/Windows -/api https://logz-docs-api.netlify.app/api \ No newline at end of file +/api https://logz-docs-api.netlify.app/api +/shipping/prometheus-sources/aws-cloudformation.html /docs/shipping/AWS/aws-cloudformation \ No newline at end of file