Update networking documentation for 2.3 (#696)

* Add link to network troubleshooting in getting started

* Add entry in faq about dns server

* Add latest/2.3 toggle for instruction

* Revert "Add latest/2.3 toggle for instruction"

This reverts commit ec945d2.

* Add toggle for old instructions

* Update from your container in endpoint url

* Add information about our DNS server

* Hide old documentation behind summary toggle

* Update the DNS server documentation

* Add item about subdomains

* Remove previous wildcard DNS access guide

content/en/getting-started/faq.md

@@ -27,9 +27,13 @@ localstack update localstack-cli
 
 ### Is using `localhost.localstack.cloud:4566` to set as the endpoint for AWS services recommended?
 
-`localhost.localstack.cloud` is the recommended endpoint - especially for S3, in order to enable host-based bucket endpoints. For most of the other services, it is fine to use `localhost:4566`. Users can rely on the `localhost.localstack.cloud` domain to be publicly resolvable, and we also publish an SSL certificate that is automatically used inside LocalStack, in order to enable HTTPS endpoints with valid certificates.
+`localhost.localstack.cloud` is the recommended endpoint - especially for S3, in order to enable host-based bucket endpoints.
+* When using this domain within LocalStack compute environments like Lambda, ECS or EC2, this domain name resolves to the LocalStack container via our DNS server available in version 2.3.
+* By configuring your environment, your applications can also use `localhost.localstack.cloud` to resolve to the LocalStack container via our DNS server.
+* In addition, we also publish an SSL certificate that is automatically used inside LocalStack, in order to enable HTTPS endpoints with valid certificates.
 
-Across our docs, we use `localhost:4566` instead of `localhost.localstack.cloud`, to provide a fallback option to users. The primary reason being that some users are behind a corporate firewall or an internet service provider that does not allow resolving `localhost.localstack.cloud` properly.
+Across our docs, we use `localhost:4566` instead of `localhost.localstack.cloud`, to provide a fallback option to users.
+The primary reason being that some users are behind a corporate firewall or an internet service provider that does not allow resolving `localhost.localstack.cloud` properly.
 
 ### How should I use the latest LocalStack Docker images?
 

content/en/getting-started/installation.md

@@ -451,6 +451,11 @@ If you have installed the CLI with Brew or directly as a binary, please simply p
   Ensure that the diagnostic endpoint is run after you have tried reproducing the affected task.
   After running the task, run the diagnostic endpoint and share the archive file with your team members or LocalStack Support.
 
+- My application cannot reach LocalStack over the network
+
+  We have [extensive network troubleshooting documentation available]({{< ref "references/network-troubleshooting" >}}).
+  If this does not solve your problem then please reach out for [help and support]({{< ref "help-and-support" >}}).
+
 ## What's next?
 
 Now that you have LocalStack up and running, the following resources might be useful for your next steps:

content/en/references/network-troubleshooting/created-resources/_index.md

@@ -24,7 +24,12 @@ For example, suppose you have created an OpenSearch cluster using LocalStack and
 
 Check out our documentation while [using the endpoint URL]({{<ref "endpoint-url#from-a-container-localstack-created" >}}).
 
-The Lambda service in LocalStack also supports the `HOSTNAME_FROM_LAMBDA` environment variable, which can be handy if LocalStack is reachable through a specific hostname. Suppose you're running LocalStack in a [user-defined network](https://docs.docker.com/network/bridge/) using Docker, where the LocalStack container can be accessed from other containers in the network using its service name. In that case, you can set the `HOSTNAME_FROM_LAMBDA` environment variable to this value to help resolve any issues with lambda functions accessing resources created by LocalStack.
+<details>
+<summary>For LocalStack versions before 2.3.0</summary>
+The Lambda service in LocalStack also supports the <code>HOSTNAME_FROM_LAMBDA</code> environment variable, which can be handy if LocalStack is reachable through a specific hostname.
+Suppose you're running LocalStack in a <a href="https://docs.docker.com/network/bridge/">user-defined network</a> using Docker, where the LocalStack container can be accessed from other containers in the network using its service name.
+In that case, you can set the <code>HOSTNAME_FROM_LAMBDA</code> environment variable to this value to help resolve any issues with lambda functions accessing resources created by LocalStack.
+</details>
 
 ## From your container
 

content/en/references/network-troubleshooting/endpoint-url/_index.md

@@ -24,7 +24,16 @@ You can also use the `EDGE_PORT` [configuration variable]({{< ref "references/co
 
 {{< figure src="../images/4.svg" width="400" >}}
 
-Suppose your code is running inside an ECS container that LocalStack has created. To enable access to the LocalStack instance, it's advisable to start LocalStack in a [user-defined network](https://docs.docker.com/network/bridge/), and then set the `MAIN_DOCKER_NETWORK` environment variable to this network's name. This allows the code running inside the container to access the LocalStack instance using its hostname. For example:
+Suppose your code is running inside an ECS container that LocalStack has created. 
+
+The LocalStack instance is available at the domain `localhost.localstack.cloud`.
+All subdomains of `localhost.localstack.cloud` also resolve to the LocalStack instance, e.g. API Gateway default URLs.
+
+<details>
+<summary>For LocalStack versions before 2.3.0</summary>
+To enable access to the LocalStack instance, it's advisable to start LocalStack in a [user-defined network](https://docs.docker.com/network/bridge/), and then set the `MAIN_DOCKER_NETWORK` environment variable to this network's name.
+This allows the code running inside the container to access the LocalStack instance using its hostname.
+For example:
 
 {{<tabpane>}}
 {{<tab header="CLI" lang="bash">}}
@@ -60,12 +69,99 @@ networks:
 # endpoint url
 {{</tab>}}
 {{</tabpane>}}
+</details>
+
 
 ## From your container
 
 {{< figure src="../images/7.svg" width="400" >}}
 
-Suppose you're accessing AWS resources such as S3 in LocalStack by running your application code in a container. To facilitate access to LocalStack from within the container, it's recommended to start LocalStack in a [user-defined network](https://docs.docker.com/network/bridge/) and set the MAIN_DOCKER_NETWORK environment variable to the network's name. Doing so enables the containerized code to connect to the LocalStack instance using its hostname. For instance:
+Suppose you're accessing AWS resources such as S3 in LocalStack by running your application code in a container.
+Your application container should be configured to use LocalStack as its DNS server.
+Once this is done, the domain name `localhost.localstack.cloud` will resolve to the LocalStack container.
+All subdomains of `localhost.localstack.cloud` will also resolve to the LocalStack instance, e.g. API Gateway default URLs.
+
+To configure your application container:
+
+* add a user-managed docker network;
+* either determine your LocalStack container IP, or configure your LocalStack container to have a fixed known IP address;
+* set the DNS server of your application container to the IP address of the LocalStack container.
+
+{{% tabpane %}}
+{{< tab header="CLI" lang="bash" >}}
+# start localstack
+localstack start -d --network ls
+localstack wait
+
+# get the ip address of the LocalStack container
+docker inspect localstack_main | \
+	jq -r '.[0].NetworkSettings.Networks | to_entries | .[].value.IPAddress'
+# prints 172.27.0.2
+
+# run your application container
+docker run --rm -it --dns 172.27.0.2 --network ls <arguments> <image name>
+{{< / tab >}}
+{{< tab header="Docker" lang="bash" >}}
+# start localstack
+docker network create ls
+docker run --rm -it --network ls --name localstack_main <other flags> localstack/localstack[-pro]
+
+# get the ip address of the LocalStack container
+docker inspect localstack_main | \
+	jq -r '.[0].NetworkSettings.Networks | to_entries | .[].value.IPAddress'
+# prints 172.27.0.2
+
+# run your application container
+docker run --rm -it --dns 172.27.0.2 --network ls <arguments> <image name>
+{{< / tab >}}
+{{< tab header="docker-compose.yml" lang="yaml" >}}
+version: "3.8"
+
+services:
+  localstack:
+    container_name: "${LOCALSTACK_DOCKER_NAME-localstack_main}"
+    image: localstack/localstack
+    ports:
+      # Now only required if you need to access LocalStack from the host
+      - "127.0.0.1:4566:4566"            
+      # Now only required if you need to access LocalStack from the host
+      - "127.0.0.1:4510-4559:4510-4559"
+    environment:
+      - DEBUG=${DEBUG-}
+      - DOCKER_HOST=unix:///var/run/docker.sock
+    volumes:
+      - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack"
+      - "/var/run/docker.sock:/var/run/docker.sock"
+    networks:
+      ls:
+        # Set the container IP address in the 10.0.2.0/24 subnet
+        ipv4_address: 10.0.2.20
+
+  application:
+    image: ghcr.io/localstack/localstack-docker-debug:main
+    entrypoint: ""
+    command: ["sleep", "infinity"]
+    dns:
+      # Set the DNS server to be the LocalStack container
+      - 10.0.2.20
+    networks:
+      - ls
+
+networks:
+  ls:
+    ipam:
+      config:
+        # Specify the subnet range for IP address allocation
+        - subnet: 10.0.2.0/24
+{{< / tab >}}
+{{% / tabpane %}}
+
+
+<details>
+<summary>For LocalStack versions before 2.3.0</summary>
+To facilitate access to LocalStack from within the container, it's recommended to start LocalStack in a <a href="https://docs.docker.com/network/bridge/">user-defined network</a> and set the <code>MAIN_DOCKER_NETWORK</code> environment variable to the network's name.
+Doing so enables the containerized code to connect to the LocalStack instance using its hostname.
+For instance:
 
 {{<tabpane>}}
 {{<tab header="CLI" lang="bash">}}
@@ -108,74 +204,10 @@ networks:
 
 ### Wildcard DNS access
 
-{{<alert title="Note">}}
-The Wildcard DNS access feature is part of [LocalStack's Pro/Team offering](https://localstack.cloud/pricing) and requires an API key to be configured.
-{{</alert>}}
-
-Certain resources created by LocalStack can be accessed using virtual host addressing. For example, an S3 bucket can be accessed at the following address format: `<bucket>.s3.<region>.localhost.localstack.cloud`. 
-
-By default, the LocalStack container cannot be reached from containers running in your Docker network at the above address. This is because any subdomains of `localhost.localstack.cloud` is resolved to `127.0.0.1` within the Docker network. 
-
-If Docker supported wildcard DNS configuration with `--network-alias` (Docker CLI) or `aliases:` (`docker-compose`), this could be solved with Docker configuration alone.
-
-To map more complex domain names to the LocalStack container within the Docker network, the LocalStack container can be utilized as a DNS server. However, this approach requires additional configuration steps. 
-
-Specifically, the LocalStack container must have a static IP address within the network. To set up the LocalStack container as a DNS server, a static IP address must be assigned within the Docker network. 
+LocalStack newer than version 2.3.0 supports wildcard DNS access by default.
+Please update your LocalStack container and see the [instructions]({{< ref "#from-your-container" >}}).
 
-Here is an example of how you can set it up:
-
-{{<tabpane>}}
-{{<tab header="Docker" lang="bash">}}
-# Create the network
-docker network create my-network --subnet <ip address range CIDR>
-
-# Start LocalStack
-docker run --rm -it \
-    --network my-network \
-    --ip 10.0.2.20 \
-    -e DNS_RESOLVE_IP=10.0.2.20 \
-    <other flags> \
-    localstack/localstack-pro
-
-# Start your application container
-docker run --rm -it \
-    --dns 10.0.2.20 \
-    --network my-network \
-    <args>
-# Your code can now access LocalStack at <subdomain>.localhost.localstack.cloud
-{{</tab>}}
-{{<tab header="docker-compose.yml" lang="yaml">}}
-services:
-  localstack:
-    # ... other configuration here
-    environment:
-      - DNS_RESOLVE_IP=10.0.2.20
-    networks:
-      ls:
-        ipv4_address: 10.0.2.20
-
-  application:
-    # ... other configuration here
-    dns:
-      - 10.0.2.20
-    networks:
-      ls:
-
-networks:
-  ls:
-    name: ls
-    ipam:
-      config:
-        - subnet: 10.0.2.0/24
-{{</tab>}}
-{{</tabpane>}}
-
-To access LocalStack resources from the *application* container, you can make requests to the following address format: `<bucket-name>.s3.<region>.localhost.localstack.cloud:4566/<key>`. This will ensure that the requests reach the LocalStack container.
-
-{{<alert title="Note">}}
-For optimal configuration, we recommend using a private IP address range, such as 10.0.0.0/8, for your containers. This helps avoid conflicts with IP addresses assigned by Docker.
-Additionally, it's advisable to avoid using `X.X.X.1` as an IP address, as it is commonly reserved for the host within that subnet.
-{{</alert>}}
+</details>
 
 ## From a separate host
 

content/en/references/network-troubleshooting/transparent-endpoint-injection/_index.md

@@ -13,6 +13,12 @@ Suppose you're attempting to access LocalStack, but you're relying on transparen
 {{< figure src="../images/2.svg" width="400" >}}
 
 If you're using LocalStack with an [API key]({{<ref "getting-started/api-key">}}), then you can utilize the [DNS server]({{<ref "user-guide/tools/transparent-endpoint-injection/dns-server">}}) to perform requests to LocalStack as if it were AWS.
+You need to make two changes:
+
+* Publish port 53 from the LocalStack docker container to your host.
+* Configure your host to use the LocalStack DNS server by default.
+
+For more details, see your [DNS server documentation]({{<ref "user-guide/tools/transparent-endpoint-injection/dns-server">}}).
 
 For the community edition of LocalStack, you can employ your own DNS server to achieve a similar outcome, but it won't be managed by LocalStack. Note that in both cases, SSL verification must be disabled. See the [limitations of the DNS server]({{<ref "user-guide/tools/transparent-endpoint-injection/dns-server#limitations" >}}) for more information.
 

content/en/user-guide/tools/transparent-endpoint-injection/dns-server.md

@@ -8,30 +8,28 @@ aliases:
   - /tools/local-endpoint-injection/dns-server/
 ---
 
-LocalStack Pro supports transparent execution mode, which means that your application code automatically accesses the LocalStack APIs as opposed to the real APIs on AWS.
+All versions of LocalStack include a DNS server that resolves the domain name `localhost.localstack.cloud` to the LocalStack container.
+This enables seamless connectivity from your container to LocalStack, or from created compute resources like Lambda, ECS or EC2 to LocalStack.
+In addition, LocalStack Pro supports transparent execution mode, which means that your application code automatically accesses the LocalStack APIs as opposed to the real APIs on AWS.
 
-When the system starts up, the log output contains the IP address of the local DNS server. Typically, this address by default is either `0.0.0.0` (see example below) or `127.0.0.1` if LocalStack cannot bind to `0.0.0.0` due to a conflicting service.
-
-```text
-Starting DNS servers (tcp/udp port 53 on 0.0.0.0)...
-```
+When the system starts up, the log output contains the IP address of the local DNS server.
+If port 53 can be bound on the host, the LocalStack CLI will publish port 53 from the container to the host on IP address `127.0.0.1`.
+Otherwise it will not publish port 53 to the host.
+Regardless of whether the port can be bound or not, the DNS server is bound to address `0.0.0.0` of the LocalStack container so other containers within the same docker network can use the DNS server.
+See the [Network Troubleshooting guide]({{< ref "references/network-troubleshooting/endpoint-url#from-your-container" >}}) for more details.
 
 ## Configuration
 
-The DNS server can be configured to match your usecase using the `DNS_ADDRESS` environment variable.
-
-To bind the server to `127.0.0.1`, you can set:
-
-```bash
-DNS_ADDRESS=127.0.0.1
-```
-
-You can disable the DNS server (which will prevent LocalStack from binding port 53) using:
+If you experience problems when running LocalStack and the DNS server is the issue, you can disable the DNS server using:
 
 ```bash
 DNS_ADDRESS=0
 ```
 
+{{< alert title="Warning" color="warning" >}}
+We do not recommend this configuration since this disables resolving `localhost.localstack.cloud` to the LocalStack container.
+{{< / alert >}}
+
 You can also specify which exact URLs should be redirected to LocalStack by defining a hostname regex like:
 
 ```bash
@@ -212,6 +210,10 @@ If you rely on your local network's DNS, your router/DNS server might block requ
 This feature is enabled by default in pfSense, OPNSense, OpenWRT, AVM FritzBox, and potentially also other devices.
 Some of the vendors might allow upstream responses in the 127.0.0.0/8 range (like OpenWRT).
 
+{{< alert title="Note" >}}
+If you are using the LocalStack DNS server, DNS rebind protection should not cause any issues.
+{{< / alert >}}
+
 You can check if your DNS setup works correctly by resolving a subdomain of `localhost.localstack.cloud`:
 {{< command "hl_lines=16">}}
 $ dig test.localhost.localstack.cloud
@@ -237,7 +239,7 @@ localhost.localstack.cloud. 389	IN	A	127.0.0.1
 ;; MSG SIZE  rcvd: 90
 {{< /command >}}
 
-If the the DNS resolves the subdomain to your localhost (127.0.0.1), your setup is working.
+If the DNS resolves the subdomain to your localhost (127.0.0.1), your setup is working.
 If not, please check the configuration of your router / DNS if the Rebind Protection is active or [enable the LocalStack DNS on your system]({{< ref "dns-server#system-dns-configuration" >}}).
 
 ## Customizing internal endpoint resolution