Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ESM] Add table for feature-specific coverage #1594

Merged
merged 3 commits into from
Dec 16, 2024
Merged

[ESM] Add table for feature-specific coverage #1594

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
77f24c9
[ESM] Add table for feature-specific coverage
gregfurman Dec 12, 2024
d96673d
Address comments
gregfurman Dec 16, 2024
c8a2c74
Appease linting gods
gregfurman Dec 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 42 additions & 11 deletions content/en/user-guide/aws/lambda/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -165,20 +165,51 @@ The product of 10 and 10 is 100%
[Lambda event source mappings](https://docs.aws.amazon.com/lambda/latest/dg/invocation-eventsourcemapping.html) allows you to connect Lambda functions to other AWS services.
The following event sources are supported in LocalStack:

- [Simple Queue Service (SQS)](https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html)
- [DynamoDB](https://docs.aws.amazon.com/lambda/latest/dg/with-ddb.html)
- [Kinesis](https://docs.aws.amazon.com/lambda/latest/dg/with-kinesis.html)
- [Managed Streaming for Apache Kafka (MSK)](https://docs.aws.amazon.com/lambda/latest/dg/with-msk.html)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we should add Self-Managed Kafka and the Pro image star ⭐ or is the table sufficient? It might be easier to see a quick list at first glance at the cost of some redundancy 🤷‍♂️

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

- [Simple Queue Service (SQS)](https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html)

The limitations compared to AWS include:
- Lambda Success Destinations are not supported.
- Only basic validations are performed upon creating and updating ESM.
- Streaming Pollers for Kinesis and DynamoDB do not implement the following batching behavior configurations:
- `BisectBatchOnFunctionError`
- `MaximumBatchingWindowInSeconds`
- `ParallelizationFactor`
- `ScalingConfig`
- `TumblingWindowInSeconds`.
- [Self-Managed Apache Kafka](https://docs.aws.amazon.com/lambda/latest/dg/with-kafka.html)

### Coverage
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From @joe4dev:

@HarshCasper How do we best present such a feature model to the user?

We are trying to improve the transparency regarding the supported behavioral features of Lambda Event Source Mapping beyond the simple list of supported sources we currently have in the Lambda docs here. Should we just add the table below to the Lambda docs (similar than the API coverage docs, but for behavior)


The coverage table below shows feature-level coverage for all supported event sources for the latest version of LocalStack.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Helpful clarification 👍

Can we add a link to the API coverage and a sentence clarifying the difference between API coverage and behavior/feature coverage?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a sentance for this!


Where necessary, footnotes are used to provide additional context.

#### Legend
- ⭐️ Only Available in Pro
- 🟢 Fully Implemented
- 🟡 Partially Implemented
- 🟠 Not Implemented
- ➖ Not Applicable (Not Supported by AWS)

| | <th colspan="2" style="text-align:center;">SQS</th> <th colspan="2" style="text-align:center;">Stream</th> <th colspan="2" style="text-align:center;">Kafka ⭐️</th>
|--------------------------------|-------------------------------------------------|:--------:|:----:|:---------:|:----------:|:----------:|:------------:|
| **Parameter** | **Description** | **Standard** | **FIFO** | **Kinesis** | **DynamoDB** | **Amazon MSK** | **Self-Managed** |
| BatchSize | Batching events by count. | 🟡 [^1] | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| ➖ | Batch when ≥ 6 MB limit. | 🟠 | 🟠 | 🟠 | 🟠 | 🟢 | 🟢 |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: remove ➖

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replaced with *Not Configurable*

| MaximumBatchingWindowInSeconds | Batch by Time Window. | 🟠 | ➖ | 🟠 | 🟠 | 🟢 | 🟢 |
| MaximumRetryAttempts | Discard after N retries. | ➖ | ➖ | 🟢 | 🟢 | ➖ | ➖ |
| MaximumRecordAgeInSeconds | Discard records older than time `t`. | ➖ | ➖ | 🟢 | 🟢 | ➖ | ➖ |
| Enabled | Enabling/Disabling. | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
| FilterCriteria | Filter criteria object. | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to distinguish between Filter engine, filter validations, and event parity (as discussed in Notion)? The current description "Filter criteria object." sounds quite API like

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For posterity: we added a footnote showing certain Metadata fields are unsupported as well as a link to the filter criteria docs on AWS.

| FunctionResponseTypes | Enabling ReportBatchItemFailures. | 🟢 | 🟢 | 🟢 | 🟢 | ➖ | ➖ |
| BisectBatchOnFunctionError | Bisect a batch on error and retry. | ➖ | ➖ | 🟠 | 🟠 | ➖ | ➖ |
| ScalingConfig | The scaling configuration for the event source. | 🟠 | 🟠 | ➖ | ➖ | ➖ | ➖ |
| ParallelizationFactor | Parallel batch processing by shard. | ➖ | ➖ | 🟠 | 🟠 | ➖ | ➖ |
| DestinationConfig.OnFailure | SQS Failure Destination. | ➖ | ➖ | 🟢 | 🟢 | 🟢 | 🟢 |
| | SNS Failure Destination. | ➖ | ➖ | 🟢 | 🟢 | 🟢 | 🟢 |
| | S3 Failure Destination. | ➖ | ➖ | 🟠 | 🟠 | 🟠 | 🟠 |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great solution 👏

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🫡

| DestinationConfig.OnSuccess | Success Destinations. | ➖ | ➖ | ➖ | ➖ | ➖ | ➖ |
| MetricsConfig | CloudWatch metrics. | 🟠 | 🟠 | 🟠 | 🟠 | 🟠 | 🟠 |
| ProvisionedPollerConfig | Control throughput via min-max limits. | ➖ | ➖ | ➖ | ➖ | 🟠 | 🟠 |
| StartingPosition | Position to start reading from. | ➖ | ➖ | 🟢 | 🟢 | 🟢 | 🟢 |
| StartingPositionTimestamp | Timestamp to start reading from. | ➖ | ➖ | 🟢 | ➖ | 🟢 | 🟢 |
| TumblingWindowInSeconds | Duration (seconds) of a processing window. | ➖ | ➖ | 🟠 | 🟠 | ➖ | ➖ |
| Topics ⭐️ | Kafka topics to read from. | ➖ | ➖ | ➖ | ➖ | 🟢 | 🟢 |

[^1]: SQS event-source mappings are limited to sending batches of up to `10` records at a time when invoking Lambda functions.

Create a [GitHub issue](https://github.com/localstack/localstack/issues/new/choose) or reach out to [LocalStack support](https://docs.localstack.cloud/getting-started/help-and-support/) if you experience any challenges.

Expand Down
Loading