Merge branch 'main' into kellen/nick-pr-3

clients/ingest-go.md

@@ -71,26 +71,47 @@ Or, set the QDB_CLIENT_CONF environment variable and call
 
 1. Export the configuration string as an environment variable:
    ```bash
-   export QDB_CLIENT_CONF="addr=localhost:9000;username=admin;password=quest;"
+   export QDB_CLIENT_CONF="http::addr=localhost:9000;username=admin;password=quest;"
    ```
 2. Then in your Go code:
    ```Go
    client, err := questdb.LineSenderFromEnv(context.TODO())
    ```
 
+Alternatively, you can use the built-in Go API to specify the connection
+options.
+
+```go
+package main
+
+import (
+       "context"
+       qdb "github.com/questdb/go-questdb-client/v3"
+)
+
+
+func main() {
+       ctx := context.TODO()
+
+       client, err := qdb.NewLineSender(context.TODO(), qdb.WithHttp(), qdb.WithAddress("localhost:9000"), qdb.WithBasicAuth("admin", "quest"))
+```
+
+When using QuestDB Enterprise, authentication can also be done via REST token.
+Please check the [RBAC docs](/docs/operations/rbac/#authentication) for more
+info.
+
 ## Basic Insert
 
-Example: inserting data from a temperature sensor.
+Example: inserting executed trades for cryptocurrencies.
 
-Without authentication:
+Without authentication and using the current timestamp:
 
 ```Go
 package main
 
 import (
 	"context"
 	"github.com/questdb/go-questdb-client/v3"
-	"time"
 )
 
 func main() {
@@ -101,12 +122,12 @@ func main() {
 		panic("Failed to create client")
 	}
 
-	timestamp := time.Now()
-	err = client.Table("sensors").
-		Symbol("id", "toronto1").
-		Float64Column("temperature", 20.0).
-		Float64Column("humidity", 0.5).
-		At(ctx, timestamp)
+	err = client.Table("trades").
+		Symbol("symbol", "ETH-USD").
+		Symbol("side", "sell").
+		Float64Column("price", 2615.54).
+		Float64Column("amount", 0.00044).
+		AtNow(ctx)
 
 	if err != nil {
 		panic("Failed to insert data")
@@ -119,57 +140,82 @@ func main() {
 }
 ```
 
-## Limitations
+In this case, the designated timestamp will be the one at execution time. Let's
+see now an example with an explicit timestamp, custom auto-flushing, and basic
+auth.
 
-### Transactionality
+```Go
+package main
 
-The Go client does not support full transactionality:
+import (
+	"context"
+	"github.com/questdb/go-questdb-client/v3"
+	"time"
+)
 
-- Data for the first table in an HTTP request will be committed even if the
-  second table's commit fails.
-- An implicit commit occurs each time a new column is added to a table. This
-  action cannot be rolled back if the request is aborted or encounters parse
-  errors.
+func main() {
+	ctx := context.TODO()
 
-### Timestamp column
+	client, err := questdb.LineSenderFromConf(ctx, "http::addr=localhost:9000;username=admin;password=quest;auto_flush_rows=100;auto_flush_interval=1000;")
+	if err != nil {
+		panic("Failed to create client")
+	}
 
-QuestDB's underlying InfluxDB Line Protocol (ILP) does not name timestamps,
-leading to an automatic column name of timestamp. To use a custom name,
-pre-create the table with the desired timestamp column name:
+	timestamp := time.Now()
+	err = client.Table("trades").
+		Symbol("symbol", "ETH-USD").
+		Symbol("side", "sell").
+		Float64Column("price", 2615.54).
+		Float64Column("amount", 0.00044).
+		At(ctx, timestamp)
+
+	if err != nil {
+		panic("Failed to insert data")
+	}
 
-```sql
-CREATE TABLE temperatures (
-    ts timestamp,
-    sensorID symbol,
-    sensorLocation symbol,
-    reading double
-) timestamp(my_ts);
+	err = client.Flush(ctx)
+	// You can flush manually at any point.
+	// If you don't flush manually, the client will flush automatically
+	// when a row is added and either:
+	//   * The buffer contains 75000 rows (if HTTP) or 600 rows (if TCP)
+	//   * The last flush was more than 1000ms ago.
+	// Auto-flushing can be customized via the `auto_flush_..` params.
+
+	if err != nil {
+		panic("Failed to flush data")
+	}
+}
 ```
 
-## Health check
+We recommended to use User-assigned timestamps when ingesting data into QuestDB.
+Using the current timestamp hinder the ability to deduplicate rows which is
+[important for exactly-once processing](/docs/reference/api/ilp/overview/#exactly-once-delivery-vs-at-least-once-delivery).
+
+## Configuration options
 
-To monitor your active connection, there is a `ping` endpoint:
+The minimal configuration string needs to have the protocol, host, and port, as
+in:
 
-```shell
-curl -I http://localhost:9000/ping
+```
+http::addr=localhost:9000;
 ```
 
-Returns (pong!):
+In the Go client, you can set the configuration options via the standard config
+string, which is the same across all clients, or using
+[the built-in API](https://pkg.go.dev/github.com/questdb/go-questdb-client/v3#LineSenderOption).
 
-```shell
-HTTP/1.1 204 OK
-Server: questDB/1.0
-Date: Fri, 2 Feb 2024 17:09:38 GMT
-Transfer-Encoding: chunked
-Content-Type: text/plain; charset=utf-8
-X-Influxdb-Version: v2.7.4
-```
+For all the extra options you can use, please check
+[the client docs](https://pkg.go.dev/github.com/questdb/go-questdb-client/v3#LineSenderFromConf)
 
-Determine whether an instance is active and confirm the version of InfluxDB Line
-Protocol with which you are interacting.
+Alternatively, for a breakdown of Configuration string options available across
+all clients, see the [Configuration string](/docs/configuration-string/) page.
 
 ## Next Steps
 
+Please refer to the [ILP overview](/docs/reference/api/ilp/overview) for details
+about transactions, error control, delivery guarantees, health check, or table
+and column auto-creation.
+
 Explore the full capabilities of the Go client via
 [Go.dev](https://pkg.go.dev/github.com/questdb/go-questdb-client/v3).
 

clients/ingest-node.md

@@ -41,10 +41,41 @@ Install the QuestDB Node.js client via npm:
 npm i -s @questdb/nodejs-client
 ```
 
-## Basic Usage
+## Authentication
 
-A simple example to connect to QuestDB, insert some data into a table, and flush
-the data:
+Passing in a configuration string with basic auth:
+
+```javascript
+const { Sender } = require("@questdb/nodejs-client");
+
+const conf = "http::addr=localhost:9000;username=admin;password=quest;"
+const sender = Sender.fromConfig(conf);
+    ...
+```
+
+Passing via the `QDB_CLIENT_CONF` env var:
+
+```bash
+export QDB_CLIENT_CONF="http::addr=localhost:9000;username=admin;password=quest;"
+```
+
+```javascript
+const { Sender } = require("@questdb/nodejs-client");
+
+
+const sender = Sender.fromEnv();
+    ...
+```
+
+When using QuestDB Enterprise, authentication can also be done via REST token.
+Please check the [RBAC docs](/docs/operations/rbac/#authentication) for more
+info.
+
+## Basic insert
+
+Example: inserting executed trades for cryptocurrencies.
+
+Without authentication and using the current timestamp.
 
 ```javascript
 const { Sender } = require("@questdb/nodejs-client")
@@ -55,42 +86,99 @@ async function run() {
 
   // add rows to the buffer of the sender
   await sender
-    .table("prices")
-    .symbol("instrument", "EURUSD")
-    .floatColumn("bid", 1.0195)
-    .floatColumn("ask", 1.0221)
-    .at(Date.now(), "ms")
-  await sender
-    .table("prices")
-    .symbol("instrument", "GBPUSD")
-    .floatColumn("bid", 1.2076)
-    .floatColumn("ask", 1.2082)
-    .at(Date.now(), "ms")
+    .table("trades")
+    .symbol("symbol", "ETH-USD")
+    .symbol("side", "sell")
+    .floatColumn("price", 2615.54)
+    .floatColumn("amount", 0.00044)
+    .atNow()
 
   // flush the buffer of the sender, sending the data to QuestDB
   // the buffer is cleared after the data is sent, and the sender is ready to accept new data
   await sender.flush()
 
-  // add rows to the buffer again, and send it to the server
+  // close the connection after all rows ingested
+  // unflushed data will be lost
+  await sender.close()
+}
+
+run().then(console.log).catch(console.error)
+```
+
+In this case, the designated timestamp will be the one at execution time. Let's
+see now an example with an explicit timestamp, custom auto-flushing, and basic
+auth.
+
+```javascript
+const { Sender } = require("@questdb/nodejs-client")
+
+async function run() {
+  // create a sender using HTTP protocol
+  const sender = Sender.fromConfig(
+    "http::addr=localhost:9000;username=admin;password=quest;auto_flush_rows=100;auto_flush_interval=1000;",
+  )
+
+  // Calculate the current timestamp. You could also parse a date from your source data.
+  const timestamp = Date.now()
+
+  // add rows to the buffer of the sender
+  await sender
+    .table("trades")
+    .symbol("symbol", "ETH-USD")
+    .symbol("side", "sell")
+    .floatColumn("price", 2615.54)
+    .floatColumn("amount", 0.00044)
+    .at(timestamp, "ms")
+
+  // add rows to the buffer of the sender
   await sender
-    .table("prices")
-    .symbol("instrument", "EURUSD")
-    .floatColumn("bid", 1.0197)
-    .floatColumn("ask", 1.0224)
-    .at(Date.now(), "ms")
+    .table("trades")
+    .symbol("symbol", "BTC-USD")
+    .symbol("side", "sell")
+    .floatColumn("price", 39269.98)
+    .floatColumn("amount", 0.001)
+    .at(timestamp, "ms")
+
+  // flush the buffer of the sender, sending the data to QuestDB
+  // the buffer is cleared after the data is sent, and the sender is ready to accept new data
   await sender.flush()
 
   // close the connection after all rows ingested
+  // unflushed data will be lost
   await sender.close()
 }
 
 run().then(console.log).catch(console.error)
 ```
 
+As you can see, both events now are using the same timestamp. We recommended to
+use the original event timestamps when ingesting data into QuestDB. Using the
+current timestamp hinder the ability to deduplicate rows which is
+[important for exactly-once processing](/docs/reference/api/ilp/overview/#exactly-once-delivery-vs-at-least-once-delivery).
+
+## Configuration options
+
+The minimal configuration string needs to have the protocol, host, and port, as
+in:
+
+```
+http::addr=localhost:9000;
+```
+
+For all the extra options you can use, please check
+[the client docs](https://questdb.github.io/nodejs-questdb-client/SenderOptions.html)
+
+Alternatively, for a breakdown of Configuration string options available across
+all clients, see the [Configuration string](/docs/configuration-string/) page.
+
 ## Next Steps
 
-Dive deeper into the Node.js client capabilities by exploring more examples
-provided in the
+Please refer to the [ILP overview](/docs/reference/api/ilp/overview) for details
+about transactions, error control, delivery guarantees, health check, or table
+and column auto-creation.
+
+Dive deeper into the Node.js client capabilities, including TypeScript and
+Worker Threads examples, by exploring the
 [GitHub repository](https://github.com/questdb/nodejs-questdb-client).
 
 To learn _The Way_ of QuestDB SQL, see the