From 27afd5f43ca248be793e479294eac58d377e7696 Mon Sep 17 00:00:00 2001 From: Joel Scheuner Date: Thu, 9 Nov 2023 00:33:29 +0100 Subject: [PATCH] Remove lambda legacy options (#899) --- content/en/references/configuration.md | 34 +--------- content/en/user-guide/aws/lambda/index.md | 7 +- .../transparent-endpoint-injection/_index.md | 21 ++++-- .../patched-sdks.md | 66 ------------------- 4 files changed, 23 insertions(+), 105 deletions(-) delete mode 100644 content/en/user-guide/tools/transparent-endpoint-injection/patched-sdks.md diff --git a/content/en/references/configuration.md b/content/en/references/configuration.md index d447c3bc39..a78f0295bb 100644 --- a/content/en/references/configuration.md +++ b/content/en/references/configuration.md @@ -146,7 +146,7 @@ See [here](#opensearch). ### Lambda {{< alert title="Note" >}} -New [Lambda]({{< ref "user-guide/aws/lambda" >}}) implementation active since LocalStack 2.0 (Docker `latest` since 2023-03-23). +The legacy [Lambda]({{< ref "user-guide/aws/lambda" >}}) implementation has been removed since LocalStack 3.0 (Docker `latest` since 2023-11-09). Please consult the [migration guide]({{< ref "user-guide/aws/lambda#migrating-to-lambda-v2" >}}) for more information. {{}} @@ -168,38 +168,6 @@ Please consult the [migration guide]({{< ref "user-guide/aws/lambda#migrating-to | `LAMBDA_RUNTIME_IMAGE_MAPPING` | [base images for Lambda](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-images.html) (default) | Customize the Docker image of Lambda runtimes, either by:
a) pattern with `` placeholder, e.g. `custom-repo/lambda-:2022`
b) json dict mapping the `` to an image, e.g. `{"python3.9": "custom-repo/lambda-py:thon3.9"}` | | `LAMBDA_SYNCHRONOUS_CREATE` | `0` (default) | Set to `1` to create lambda functions synchronously (not recommended). | | `LAMBDA_TRUNCATE_STDOUT` | `2000` (default) | Allows increasing the default char limit for truncation of lambda log lines when printed in the console. This does not affect the logs processing in CloudWatch. | -| `PROVIDER_OVERRIDE_LAMBDA` | `v2` (default) | Currently supported implementation of our [local Lambda service]({{< ref "user-guide/aws/lambda" >}}) active since LocalStack 2.0 (Docker `latest` since 2023-03-23). | -| | `legacy` (**deprecated**) | Use the old lambda provider (not recommended).
**See [Lambda providers]({{< ref "user-guide/aws/lambda" >}}).** | - -### Lambda (Legacy) - -The old lambda provider is temporarily available in Localstack v2 using `PROVIDER_OVERRIDE_LAMBDA=legacy` but we highly recommend [migrating]({{< ref "user-guide/aws/lambda" >}}) to the new Lambda provider. - -| Variable| Example Values | Description | -| - | - | - | -| `LAMBDA_EXECUTOR` | | Method to use for executing Lambda functions. For `docker` and `docker-reuse`, if LocalStack itself is started inside Docker, then the `docker` command needs to be available inside the container (usually requires to run the container in privileged mode). More information in Lambda Executor Modes.
**Removed in new provider. Mount the Docker socket or see [migration guide]({{< ref "user-guide/aws/lambda" >}}).** | -| | `docker` (default) | Run each function invocation in a separate Docker container. | -| | `local` (fallback) | Run Lambda functions in a temporary directory on the local machine. | -| | `docker-reuse` | Create one Docker container per function and reuse it across invocations. | -| `LAMBDA_STAY_OPEN_MODE` | `1` (default) | Usage of the stay-open mode of Lambda containers. Only applicable if `LAMBDA_EXECUTOR=docker-reuse`. Set to `0` if you want to use [Hot Reloading]({{< ref "hot-reloading" >}}).
**Removed in new provider because stay-open mode is the default behavior. `LAMBDA_KEEPALIVE_MS` can be used to configure how long containers should be kept running in-between invocations.** | -| `LAMBDA_REMOTE_DOCKER` | | determines whether Lambda code is copied or mounted into containers.
**Removed in new provider because zip file copying is used by default and hot reloading automatically configures mounting.** | -| | `true` (default) | your Lambda function definitions will be passed to the container by copying the zip file (potentially slower). It allows for remote execution, where the host and the client are not on the same machine.| -| | `false` | your Lambda function definitions will be passed to the container by mounting a volume (potentially faster). This requires to have the Docker client and the Docker host on the same machine. | -| `LAMBDA_TRUNCATE_STDOUT` | `2000` (default) | Allows increasing the default char value for truncation of lambda logs. This does not affect the logs processing in CloudWatch.
**Still supported in new provider.** | -| `BUCKET_MARKER_LOCAL` | `__local__` (default) | Optional bucket name for running lambdas locally.
**Still supported in new provider but default changed to `hot-reload`.** | -| `LAMBDA_CODE_EXTRACT_TIME` | `25` (default) | Time in seconds to wait at max while extracting Lambda code. By default, it is 25 seconds for limiting the execution time to avoid client/network timeout issues.
**Removed in new provider because function creation happens asynchronously.**| -| `LAMBDA_DOCKER_NETWORK` | | Optional Docker network for the container running your lambda function. This configuration value also applies to ECS containers. Needs to be set to the network the LocalStack container is connected to.
**Still supported in new provider.** | -| `LAMBDA_DOCKER_DNS` | | Optional DNS server for the container running your lambda function.
**Supported in new provider since LocalStack 2.2.** | -| `LAMBDA_DOCKER_FLAGS` | `-e KEY=VALUE`, `-v host:container`, `-p host:container`, `--add-host domain:ip` | Additional flags passed to Docker `run`\|`create` commands. Supports environment variables, ports, volume mounts, extra hosts, networks, labels, user, platform and privileged mode.
**Still supported in new provider.** | -| `LAMBDA_CONTAINER_REGISTRY` | `lambci/lambda` (default) | An alternative docker registry from where to pull lambda execution containers.
**Replaced by `LAMBDA_RUNTIME_IMAGE_MAPPING` in new provider.** | -| `LAMBDA_REMOVE_CONTAINERS` | `1` (default) | Whether to remove any Lambda Docker containers. Only applicable when using docker-reuse executor.
**Still supported in new provider.** | -| `LAMBDA_FALLBACK_URL` | | Fallback URL to use when a non-existing Lambda is invoked. Either records invocations in DynamoDB (value `dynamodb://`) or forwards invocations as a POST request (value `http(s)://...`).
**Removed in new provider.** | -| `LAMBDA_FORWARD_URL` | | URL used to forward all Lambda invocations (useful to run Lambdas via an external service).
**Removed in new provider.** | -| `LAMBDA_JAVA_OPTS` | `-Xmx512M` | Allow passing custom JVM options to Java Lambdas executed in Docker. Use `_debug_port_` placeholder to configure the debug port, e.g., `-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=_debug_port_`.
**Currently not supported in new provider but possible via custom entrypoint.** | -| `HOSTNAME_FROM_LAMBDA` | `localstack` | Endpoint host under which APIs are accessible from Lambda containers (optional). This can be useful in docker-compose stacks to use the local container hostname if neither IP address nor container name of the main container are available (e.g., in CI). Often used in combination with `LAMBDA_DOCKER_NETWORK`.
**Removed in new provider.** | -| `LAMBDA_XRAY_INIT` | `1` / `0` (default) | Whether to fully initialize XRay daemon for Lambda containers (may increase Lambda startup times).
**Removed in new provider because the X-Ray daemon is always initialized.** | -| `SYNCHRONOUS_KINESIS_EVENTS` | `1` (default) / `0` | Whether or not to handle Kinesis Lambda event sources as synchronous invocations.
**Removed in new provider.** | -| `PROVIDER_OVERRIDE_LAMBDA` | `asf` (optional) | Opt-in to use the new lambda provider beta. See [Lambda providers]({{< ref "user-guide/aws/lambda" >}}).
**Active by default in the new provider.** | ### MWAA diff --git a/content/en/user-guide/aws/lambda/index.md b/content/en/user-guide/aws/lambda/index.md index a7e66fcb5d..5f3af869af 100644 --- a/content/en/user-guide/aws/lambda/index.md +++ b/content/en/user-guide/aws/lambda/index.md @@ -196,7 +196,12 @@ The Resource Browser allows you to perform the following actions: ## Migrating to Lambda v2 -As part of the [LocalStack 2.0 release](https://discuss.localstack.cloud/t/new-lambda-implementation-in-localstack-2-0/258), the Lambda provider has been migrated to `v2` (formerly known as `asf`). With the new implementation, the following changes have been introduced: +{{< alert title="Note" >}} +The legacy Lambda implementation has been removed since LocalStack 3.0 (Docker `latest` since 2023-11-09). +{{}} + +As part of the [LocalStack 2.0 release](https://discuss.localstack.cloud/t/new-lambda-implementation-in-localstack-2-0/258), the Lambda provider has been migrated to `v2` (formerly known as `asf`). +With the new implementation, the following changes have been introduced: - To run Lambda functions in LocalStack, mount the Docker socket into the LocalStack container. Add the following Docker volume mount to your LocalStack startup configuration: `/var/run/docker.sock:/var/run/docker.sock`. You can find an example of this configuration in our official [`docker-compose.yml` file](https://docs.localstack.cloud/getting-started/installation/#starting-localstack-with-docker-compose). - The `v2` provider discontinues Lambda Executor Modes such as `LAMBDA_EXECUTOR=local`. Previously, this mode was used as a fallback when the Docker socket was unavailable in the LocalStack container, but many users unintentionally used it instead of the configured `LAMBDA_EXECUTOR=docker`. The new provider now behaves similarly to the old `docker-reuse` executor and does not require such configuration. diff --git a/content/en/user-guide/tools/transparent-endpoint-injection/_index.md b/content/en/user-guide/tools/transparent-endpoint-injection/_index.md index 9c3eae6978..242ff8912d 100644 --- a/content/en/user-guide/tools/transparent-endpoint-injection/_index.md +++ b/content/en/user-guide/tools/transparent-endpoint-injection/_index.md @@ -8,10 +8,21 @@ aliases: - /tools/local-endpoint-injection/ --- -In the community (open source) edition, the application code needs to configure each AWS SDK client instance with the target `endpoint URL` to point to the APIs on `localhost` or, in the case of Lambdas running in the context of LocalStack, the `endpoint URL` should point to `http://${LOCALSTACK_HOSTNAME}:${EDGE_PORT}`. +In the community (open source) edition, +the application code needs to configure the `endpoint URL` of each AWS SDK client instance to target LocalStack +using the environment variable `AWS_ENDPOINT_URL` available within Lambda functions in LocalStack. +For example, a Python boto3 client can be configured as follows: -The Pro version provides two options for transparently making your application logic speak to the local APIs instead of real AWS (without having to change your production code): -1. integrated DNS server -2. patched AWS SDKs (**deprecated**) +```python +client = boto3.client("lambda", endpoint_url=os.environ['AWS_ENDPOINT_URL']) +``` -More details can be found in the subsections below. +For [supported AWS SDKs](https://docs.aws.amazon.com/sdkref/latest/guide/feature-ss-endpoints.html#ss-endpoints-sdk-compat) +(including boto3 since [1.28.0](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst#L892)), this configuration happens automatically without any custom code changes. + + +In LocalStack Pro, +no application code changes are required to let your application connect to local cloud APIs instead of real AWS because +LocalStack provides an integrated DNS server that resolves AWS API calls to target LocalStack. + +More details can be found in the subsection below. diff --git a/content/en/user-guide/tools/transparent-endpoint-injection/patched-sdks.md b/content/en/user-guide/tools/transparent-endpoint-injection/patched-sdks.md deleted file mode 100644 index ac6e19804b..0000000000 --- a/content/en/user-guide/tools/transparent-endpoint-injection/patched-sdks.md +++ /dev/null @@ -1,66 +0,0 @@ ---- -title: "Patched AWS SDKs for Lambdas (Deprecated)" -categories: ["LocalStack Pro", "Tools"] -weight: 6 -description: > - Using patched SDKs in Lambdas to transparently redirect AWS API calls to LocalStack -aliases: - - /tools/local-endpoint-injection/patched-sdks/ ---- - -{{< alert title="Warning" color="warning">}} -Patched AWS SDKs for Lambdas are **deprecated** and only used by the old lambda provider. -The new lambda provider (active since LocalStack 2.0) uses DNS-based domain resolution (except for the Ruby runtime).
-Please refer to [Lambda providers]({{< ref "user-guide/aws/lambda" >}}) for more details about the new Lambda implementation. -{{< /alert >}} - -The Lambda runtime in LocalStack uses patched AWS SDKs, which are configured to target the local APIs instead of the real AWS. -This behavior is enabled by default for most Lambda runtimes when using LocalStack Pro. - -Assuming you had a Python Lambda handler that attempts to list all S3 buckets. In the past, you had to manually configure the `endpoint_url` parameter on the boto3 client (and potentially use environment switches for dev/prod in your test code): -```python -import boto3 -def handler(event, context): - client = boto3.client("s3", endpoint_url="http://localhost:4566") - print(client.list_buckets()) -``` - -With the patched AWS SDKs, it now becomes possible to deploy your unmodified production code to LocalStack, simply creating a boto3 client with default settings. The invocations of the boto3 client will be automatically forwarded to the local APIs: -```python -import boto3 -def handler(event, context): - client = boto3.client("s3") - print(client.list_buckets()) -``` - -{{< alert title="Note">}} -This functionality only works when using the SDKs provided by the Lambda execution environment itself. -If you choose to ship your own SDKs with your Lambda or using a layer, it will fallback to the [DNS based transparent execution]({{< ref "dns-server" >}}) if enabled, since those SDK versions will not be patched. -{{< /alert >}} - -This feature works by patching the AWS SDKs in the docker images, which provide the execution environment for Lambdas within LocalStack. - -The main advantage of this mode is, that no DNS magic is involved, and SSL certificate checks do not have to be disabled. - -## Configuration - -If you want to disable this behavior, and use the DNS server to resolve the endpoints for AWS, you can disable this behavior by using: - -```bash -TRANSPARENT_LOCAL_ENDPOINTS=0 -``` - -## Supported Runtimes -Currently, LocalStack supports patching the SDKs for the following runtimes: - -* Python (using boto3) -* NodeJS -* Ruby -* Java - -Also, these patched SDKs are only available in the following Lambda execution modes: - -* docker -* docker-reuse - -This feature is currently not supported for custom Lambda container images.