Add native Logstash to Logstash documentation (#15346)

Add native Logstash to Logstash documentation, which replaces HTTP Input/Output

docs/static/ls-ls-config.asciidoc

@@ -10,12 +10,23 @@ one Logstash instance, see <<pipeline-to-pipeline>>.
 
 Logstash-to-Logstash communication can be achieved in one of two ways: 
 
+* <<native-considerations,Logstash output to Logstash Input>>
 * <<lumberjack-considerations,Lumberjack output to Beats input>>
-* <<http-considerations,HTTP output to HTTP input>>
+
+[[native-considerations]]*Logstash to Logstash considerations*
+
+This is the preferred method to implement Logstash-to-Logstash. It replaces <<ls-to-ls-http>> and has these considerations:
+
+* It relies on HTTP as the communication protocol between the Input and Output.
+* It does not provide built-in high availability. You will need to implement your own load balancer in between the Logstash output and the Logstash input.
+* If you need a proxy between the Logstash instances, you can use any HTTP proxy.
+* No connection information is added to events.
+
+Ready to see more configuration details? See <<ls-to-ls-native>>.
 
 [[lumberjack-considerations]]*Lumberjack-Beats considerations*
 
-Lumberjack output to Beats input has been our standard approach for {ls}-to-{ls} communication, and may still be the best option for more robust use cases. 
+Lumberjack output to Beats input has been our standard approach for {ls}-to-{ls} communication, but our recommended approach is now <<ls-to-ls-native>>.
 Before you implement the Lumberjack to Beats configuration, keep these points in mind: 
 
 * Lumberjack to Beats provides high availability, but does not provide load balancing. 
@@ -25,20 +36,6 @@ The Lumberjack output plugin allows defining multiple output hosts for high avai
 
 Ready to see more configuration details? See <<ls-to-ls-lumberjack>>.
 
-[[http-considerations]]*HTTP-HTTP considerations* 
-
-This approach relies on the use of <<plugins-outputs-http,http output>> to <<plugins-inputs-http,http input>> plugins.
-Take these considerations into account before you implement:
-
-* HTTP does not provide built-in high availability. You will need to implement your own load balancer in between the HTTP output and the HTTP input. 
-* If you need a proxy between the Logstash instances, you can use any HTTP proxy. 
-* The HTTP input adds connection information to events, and this may be data you don't want.
-
-For now, <<plugins-outputs-http,http output>> to <<plugins-inputs-http,http input>> with manual configuration may be the best path forward if these limitations don't apply to your use case.
-
-Ready to see more configuration details? See <<ls-to-ls-http>>.
-
-NOTE: In the future, we may replace the implementation of Logstash-to-Logstash with a purpose-build HTTP implementation, which would deprecate the use of Lumberjack and Beats, or the use of the HTTP Input and Output plugins.
-
 include::ls-ls-lumberjack.asciidoc[]
 include::ls-ls-http.asciidoc[]
+include::ls-ls-native.asciidoc[]

docs/static/ls-ls-http.asciidoc

@@ -4,9 +4,7 @@
 HTTP output to HTTP input is an alternative to the Lumberjack output to Beats input approach for Logstash-to-Logstash communication. 
 This approach relies on the use of <<plugins-outputs-http,http output>> to <<plugins-inputs-http,http input>> plugins.
 
-NOTE: Check out these <<http-considerations,considerations>> before you implement {ls}-to-{ls} using HTTP. 
-
-For now, <<plugins-outputs-http,http output>> to <<plugins-inputs-http,http input>> with manual configuration may be the best path forward if these limitations don't apply to your use case.
+NOTE: {ls}-to-{ls} using HTTP input/output plugins is now being deprecated in favor of <<ls-to-ls-native>>.
 
 [[overview-http-http]]
 ==== Configuration overview
@@ -60,7 +58,7 @@ output {
 }
 ----
 
-[[securing-logstash-to-logstash]]
+[[securing-logstash-to-logstash-http]]
 ===== Secure Logstash to Logstash
 
 It is important that you secure the communication between Logstash instances. 

docs/static/ls-ls-native.asciidoc

@@ -0,0 +1,128 @@
+[[ls-to-ls-native]]
+=== Logstash-to-Logstash: Output to Input
+
+The Logstash output to Logstash input is the default approach for Logstash-to-Logstash communication.
+
+NOTE: Check out these <<native-considerations,considerations>> before you implement {ls}-to-{ls}.
+
+[[overview-ls-ls]]
+==== Configuration overview
+
+To connect two Logstash instances:
+
+. Configure the downstream (server) Logstash to use Logstash input
+. Configure the upstream (client) Logstash to use Logstash output
+. Secure the communication between Logstash input and Logstash output
+
+[[configure-downstream-logstash-input]]
+===== Configure the downstream Logstash to use Logstash input
+
+Configure the Logstash input on the downstream (receiving) Logstash to receive connections.
+The minimum configuration requires this option:
+
+* `port` - To set a custom port. The default is 9800 if none is provided.
+
+[source,json]
+----
+input {
+    logstash {
+        port => 9800
+    }
+}
+----
+
+[[configure-upstream-logstash-output]]
+===== Configure the upstream Logstash to use Logstash output
+
+In order to obtain the best performance when sending data from one Logstash to another, the data is batched and compressed. As such, the upstream Logstash (the sending Logstash) only needs to be concerned about configuring the receiving endpoint with these options:
+
+* `hosts` - The receiving Logstash and port. If no port specified, 9800 will be used.
+
+NOTE: In the future, {ls} will support multiple output hosts.
+
+[source,json]
+----
+output {
+    logstash {
+        hosts => '10.0.0.123:9800'
+    }
+}
+----
+
+[[securing-logstash-to-logstash]]
+===== Secure Logstash to Logstash
+
+It is important that you secure the communication between Logstash instances.
+Use SSL/TLS mutual authentication in order to ensure that the upstream Logstash instance sends encrypted data to a trusted downstream Logstash instance, and vice versa.
+
+. Create a certificate authority (CA) in order to sign the certificates that you plan to use between Logstash instances. Creating a correct SSL/TLS infrastructure is outside the scope of this document.
++
+TIP: We recommend you use the {ref}/certutil.html[elasticsearch-certutil] tool to generate your certificates.
+
+. Configure the downstream (receiving) Logstash to use SSL.
+Add these settings to the Logstash input configuration:
++
+ * `ssl_enabled`: When set to `true`, it enables Logstash use of SSL/TLS
+ * `ssl_key`: Specifies the key that Logstash uses to authenticate with the client.
+ * `ssl_certificate`: Specifies the certificate that Logstash uses to authenticate with the client.
+ * `ssl_certificate_authorities`: Configures Logstash to trust any certificates signed by the specified CA.
+ * `ssl_client_authentication`: Specifies whether Logstash server verifies the client certificate against the CA.
++
+For example:
++
+[source,json]
+----
+input {
+  logstash {
+    ...
+
+    ssl_enabled => true
+    ssl_key => "server.pkcs8.key"
+    ssl_certificate => "server.crt"
+    ssl_certificate_authorities => "ca.crt"
+    ssl_client_authentication => required
+  }
+}
+----
+
+. Configure the upstream (sending) Logstash to use SSL.
+Add these settings to the Logstash output configuration:
++
+ * `ssl_key`: Specifies the key the Logstash client uses to authenticate with the Logstash server.
+ * `ssl_certificate`: Specifies the certificate that the Logstash client uses to authenticate to the Logstash server.
+ * `ssl_certificate_authorities`: Configures the Logstash client to trust any certificates signed by the specified CA.
++
+For example:
++
+[source,json]
+----
+output {
+  logstash {
+    ...
+
+    ssl_enabled => true
+    ssl_key => "client.pkcs8.key"
+    ssl_certificate => "client.crt"
+    ssl_certificate_authorities => "ca.crt"
+  }
+}
+----
+
+. If you would like an additional authentication step, you can also use basic user/password authentication in both Logstash instances:
++
+ * `username`: Sets the username to use for authentication.
+ * `password`: Sets the password to use for authentication.
++
+For example, you would need to add the following to both Logstash instances:
++
+[source,json]
+----
+...
+  logstash {
+    ...
+
+    username => "your-user"
+    password => "your-secret"
+  }
+...
+----