From 77f24c9b244df87a2f27bf5f430b9871af522229 Mon Sep 17 00:00:00 2001 From: Greg Furman Date: Thu, 12 Dec 2024 18:57:48 +0200 Subject: [PATCH 1/3] [ESM] Add table for feature-specific coverage --- content/en/user-guide/aws/lambda/index.md | 53 ++++++++++++++++++----- 1 file changed, 42 insertions(+), 11 deletions(-) diff --git a/content/en/user-guide/aws/lambda/index.md b/content/en/user-guide/aws/lambda/index.md index dcd125d8bd..24cf0cb6f7 100644 --- a/content/en/user-guide/aws/lambda/index.md +++ b/content/en/user-guide/aws/lambda/index.md @@ -165,20 +165,51 @@ The product of 10 and 10 is 100% [Lambda event source mappings](https://docs.aws.amazon.com/lambda/latest/dg/invocation-eventsourcemapping.html) allows you to connect Lambda functions to other AWS services. The following event sources are supported in LocalStack: +- [Simple Queue Service (SQS)](https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html) - [DynamoDB](https://docs.aws.amazon.com/lambda/latest/dg/with-ddb.html) - [Kinesis](https://docs.aws.amazon.com/lambda/latest/dg/with-kinesis.html) - [Managed Streaming for Apache Kafka (MSK)](https://docs.aws.amazon.com/lambda/latest/dg/with-msk.html) -- [Simple Queue Service (SQS)](https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html) - -The limitations compared to AWS include: -- Lambda Success Destinations are not supported. -- Only basic validations are performed upon creating and updating ESM. -- Streaming Pollers for Kinesis and DynamoDB do not implement the following batching behavior configurations: - - `BisectBatchOnFunctionError` - - `MaximumBatchingWindowInSeconds` - - `ParallelizationFactor` - - `ScalingConfig` - - `TumblingWindowInSeconds`. +- [Self-Managed Apache Kafka](https://docs.aws.amazon.com/lambda/latest/dg/with-kafka.html) + +### Coverage + +The coverage table below shows feature-level coverage for all supported event sources for the latest version of LocalStack. + +Where necessary, footnotes are used to provide additional context. + +#### Legend +- ⭐️ Only Available in Pro +- 🟢 Fully Implemented +- 🟡 Partially Implemented +- 🟠 Not Implemented +- ➖ Not Applicable (Not Supported by AWS) + +| | SQS Stream Kafka ⭐️ +|--------------------------------|-------------------------------------------------|:--------:|:----:|:---------:|:----------:|:----------:|:------------:| +| **Parameter** | **Description** | **Standard** | **FIFO** | **Kinesis** | **DynamoDB** | **Amazon MSK** | **Self-Managed** | +| BatchSize | Batching events by count. | 🟡 [^1] | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | +| ➖ | Batch when ≥ 6 MB limit. | 🟠 | 🟠 | 🟠 | 🟠 | 🟢 | 🟢 | +| MaximumBatchingWindowInSeconds | Batch by Time Window. | 🟠 | ➖ | 🟠 | 🟠 | 🟢 | 🟢 | +| MaximumRetryAttempts | Discard after N retries. | ➖ | ➖ | 🟢 | 🟢 | ➖ | ➖ | +| MaximumRecordAgeInSeconds | Discard records older than time `t`. | ➖ | ➖ | 🟢 | 🟢 | ➖ | ➖ | +| Enabled | Enabling/Disabling. | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | +| FilterCriteria | Filter criteria object. | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | +| FunctionResponseTypes | Enabling ReportBatchItemFailures. | 🟢 | 🟢 | 🟢 | 🟢 | ➖ | ➖ | +| BisectBatchOnFunctionError | Bisect a batch on error and retry. | ➖ | ➖ | 🟠 | 🟠 | ➖ | ➖ | +| ScalingConfig | The scaling configuration for the event source. | 🟠 | 🟠 | ➖ | ➖ | ➖ | ➖ | +| ParallelizationFactor | Parallel batch processing by shard. | ➖ | ➖ | 🟠 | 🟠 | ➖ | ➖ | +| DestinationConfig.OnFailure | SQS Failure Destination. | ➖ | ➖ | 🟢 | 🟢 | 🟢 | 🟢 | +| | SNS Failure Destination. | ➖ | ➖ | 🟢 | 🟢 | 🟢 | 🟢 | +| | S3 Failure Destination. | ➖ | ➖ | 🟠 | 🟠 | 🟠 | 🟠 | +| DestinationConfig.OnSuccess | Success Destinations. | ➖ | ➖ | ➖ | ➖ | ➖ | ➖ | +| MetricsConfig | CloudWatch metrics. | 🟠 | 🟠 | 🟠 | 🟠 | 🟠 | 🟠 | +| ProvisionedPollerConfig | Control throughput via min-max limits. | ➖ | ➖ | ➖ | ➖ | 🟠 | 🟠 | +| StartingPosition | Position to start reading from. | ➖ | ➖ | 🟢 | 🟢 | 🟢 | 🟢 | +| StartingPositionTimestamp | Timestamp to start reading from. | ➖ | ➖ | 🟢 | ➖ | 🟢 | 🟢 | +| TumblingWindowInSeconds | Duration (seconds) of a processing window. | ➖ | ➖ | 🟠 | 🟠 | ➖ | ➖ | +| Topics ⭐️ | Kafka topics to read from. | ➖ | ➖ | ➖ | ➖ | 🟢 | 🟢 | + +[^1]: SQS event-source mappings are limited to sending batches of up to `10` records at a time when invoking Lambda functions. Create a [GitHub issue](https://github.com/localstack/localstack/issues/new/choose) or reach out to [LocalStack support](https://docs.localstack.cloud/getting-started/help-and-support/) if you experience any challenges. From d96673d4617babe97b9851954458bbfbb89a50c8 Mon Sep 17 00:00:00 2001 From: Greg Furman Date: Mon, 16 Dec 2024 15:47:28 +0200 Subject: [PATCH 2/3] Address comments --- content/en/user-guide/aws/lambda/index.md | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/content/en/user-guide/aws/lambda/index.md b/content/en/user-guide/aws/lambda/index.md index 24cf0cb6f7..fe0e17623a 100644 --- a/content/en/user-guide/aws/lambda/index.md +++ b/content/en/user-guide/aws/lambda/index.md @@ -168,32 +168,36 @@ The following event sources are supported in LocalStack: - [Simple Queue Service (SQS)](https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html) - [DynamoDB](https://docs.aws.amazon.com/lambda/latest/dg/with-ddb.html) - [Kinesis](https://docs.aws.amazon.com/lambda/latest/dg/with-kinesis.html) -- [Managed Streaming for Apache Kafka (MSK)](https://docs.aws.amazon.com/lambda/latest/dg/with-msk.html) -- [Self-Managed Apache Kafka](https://docs.aws.amazon.com/lambda/latest/dg/with-kafka.html) +- [Managed Streaming for Apache Kafka (MSK)](https://docs.aws.amazon.com/lambda/latest/dg/with-msk.html) ⭐️ +- [Self-Managed Apache Kafka](https://docs.aws.amazon.com/lambda/latest/dg/with-kafka.html) ⭐️ -### Coverage +### Behaviour Coverage -The coverage table below shows feature-level coverage for all supported event sources for the latest version of LocalStack. +The table below shows feature coverage for all supported event sources for the latest version of LocalStack. + +Unlike [API operation coverage](https://docs.localstack.cloud/references/coverage/coverage_lambda/), this table illustrates the **functional and behavioural coverage** of LocalStack's Lambda Event Source Mapping implementation. Where necessary, footnotes are used to provide additional context. -#### Legend -- ⭐️ Only Available in Pro +{{< callout >}} +Feature availability and coverage is categorized with the following system: +- ⭐️ Only Available in LocalStack Pro image - 🟢 Fully Implemented - 🟡 Partially Implemented - 🟠 Not Implemented - ➖ Not Applicable (Not Supported by AWS) +{{}} | | SQS Stream Kafka ⭐️ |--------------------------------|-------------------------------------------------|:--------:|:----:|:---------:|:----------:|:----------:|:------------:| | **Parameter** | **Description** | **Standard** | **FIFO** | **Kinesis** | **DynamoDB** | **Amazon MSK** | **Self-Managed** | | BatchSize | Batching events by count. | 🟡 [^1] | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | -| ➖ | Batch when ≥ 6 MB limit. | 🟠 | 🟠 | 🟠 | 🟠 | 🟢 | 🟢 | +| *Not Configurable* | Batch when ≥ 6 MB limit. | 🟠 | 🟠 | 🟠 | 🟠 | 🟢 | 🟢 | | MaximumBatchingWindowInSeconds | Batch by Time Window. | 🟠 | ➖ | 🟠 | 🟠 | 🟢 | 🟢 | | MaximumRetryAttempts | Discard after N retries. | ➖ | ➖ | 🟢 | 🟢 | ➖ | ➖ | | MaximumRecordAgeInSeconds | Discard records older than time `t`. | ➖ | ➖ | 🟢 | 🟢 | ➖ | ➖ | | Enabled | Enabling/Disabling. | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | -| FilterCriteria | Filter criteria object. | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | +| FilterCriteria | Filter pattern evaluating. [^2] [^3] | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | | FunctionResponseTypes | Enabling ReportBatchItemFailures. | 🟢 | 🟢 | 🟢 | 🟢 | ➖ | ➖ | | BisectBatchOnFunctionError | Bisect a batch on error and retry. | ➖ | ➖ | 🟠 | 🟠 | ➖ | ➖ | | ScalingConfig | The scaling configuration for the event source. | 🟠 | 🟠 | ➖ | ➖ | ➖ | ➖ | @@ -210,6 +214,8 @@ Where necessary, footnotes are used to provide additional context. | Topics ⭐️ | Kafka topics to read from. | ➖ | ➖ | ➖ | ➖ | 🟢 | 🟢 | [^1]: SQS event-source mappings are limited to sending batches of up to `10` records at a time when invoking Lambda functions. +[^2]: Read more at [Control which events Lambda sends to your function](https://docs.aws.amazon.com/lambda/latest/dg/invocation-eventfiltering.html) +[^3]: The available Metadata properties may not have full parity with AWS depending on the event source. Read more at [Understanding event filtering basics](https://docs.aws.amazon.com/lambda/latest/dg/invocation-eventfiltering.html#filtering-basics). Create a [GitHub issue](https://github.com/localstack/localstack/issues/new/choose) or reach out to [LocalStack support](https://docs.localstack.cloud/getting-started/help-and-support/) if you experience any challenges. From c8a2c74caa8f0463d570497a134033205e68e10a Mon Sep 17 00:00:00 2001 From: Greg Furman Date: Mon, 16 Dec 2024 15:50:29 +0200 Subject: [PATCH 3/3] Appease linting gods --- content/en/user-guide/aws/lambda/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/user-guide/aws/lambda/index.md b/content/en/user-guide/aws/lambda/index.md index fe0e17623a..d17d62f664 100644 --- a/content/en/user-guide/aws/lambda/index.md +++ b/content/en/user-guide/aws/lambda/index.md @@ -215,7 +215,7 @@ Feature availability and coverage is categorized with the following system: [^1]: SQS event-source mappings are limited to sending batches of up to `10` records at a time when invoking Lambda functions. [^2]: Read more at [Control which events Lambda sends to your function](https://docs.aws.amazon.com/lambda/latest/dg/invocation-eventfiltering.html) -[^3]: The available Metadata properties may not have full parity with AWS depending on the event source. Read more at [Understanding event filtering basics](https://docs.aws.amazon.com/lambda/latest/dg/invocation-eventfiltering.html#filtering-basics). +[^3]: The available Metadata properties may not have full parity with AWS depending on the event source (read more at [Understanding event filtering basics](https://docs.aws.amazon.com/lambda/latest/dg/invocation-eventfiltering.html#filtering-basics)). Create a [GitHub issue](https://github.com/localstack/localstack/issues/new/choose) or reach out to [LocalStack support](https://docs.localstack.cloud/getting-started/help-and-support/) if you experience any challenges.