Jv/ingestion examples

clients/ingest-go.md

@@ -71,26 +71,46 @@ Or, set the QDB_CLIENT_CONF environment variable and call
 
 1. Export the configuration string as an environment variable:
    ```bash
-   export QDB_CLIENT_CONF="addr=localhost:9000;username=admin;password=quest;"
+   export QDB_CLIENT_CONF="http::addr=localhost:9000;username=admin;password=quest;"
    ```
 2. Then in your Go code:
    ```Go
    client, err := questdb.LineSenderFromEnv(context.TODO())
    ```
 
+Alternatively, you can use the built-in Go API to specify the connection options.
+
+ ```go
+ package main
+
+import (
+        "context"
+        qdb "github.com/questdb/go-questdb-client/v3"
+)
+
+
+func main() {
+        ctx := context.TODO()
+
+        client, err := qdb.NewLineSender(context.TODO(), qdb.WithHttp(), qdb.WithAddress("localhost:9000"), qdb.WithBasicAuth("admin", "quest"))
+```
+
+
+When using QuestDB Enterprise, authentication can also be done via REST token.
+Please check the [RBAC docs](/docs/operations/rbac/#authentication) for more info.
+
 ## Basic Insert
 
-Example: inserting data from a temperature sensor.
+Example: inserting executed trades for cryptocurrencies.
 
-Without authentication:
+Without authentication and using the current timestamp:
 
 ```Go
 package main
 
 import (
 	"context"
 	"github.com/questdb/go-questdb-client/v3"
-	"time"
 )
 
 func main() {
@@ -101,12 +121,12 @@ func main() {
 		panic("Failed to create client")
 	}
 
-	timestamp := time.Now()
-	err = client.Table("sensors").
-		Symbol("id", "toronto1").
-		Float64Column("temperature", 20.0).
-		Float64Column("humidity", 0.5).
-		At(ctx, timestamp)
+	err = client.Table("trades").
+		Symbol("symbol", "ETH-USD").
+		Symbol("side", "sell").
+		Float64Column("price", 2615.54).
+		Float64Column("amount", 0.00044).
+		AtNow(ctx)
 
 	if err != nil {
 		panic("Failed to insert data")
@@ -119,57 +139,74 @@ func main() {
 }
 ```
 
-## Limitations
+In this case, the designated timestamp will be the one at execution time. Let's see now an example with an explicit timestamp, custom auto-flushing, and basic auth.
+
+```Go
+package main
+
+import (
+	"context"
+	"github.com/questdb/go-questdb-client/v3"
+	"time"
+)
 
-### Transactionality
+func main() {
+	ctx := context.TODO()
 
-The Go client does not support full transactionality:
+	client, err := questdb.LineSenderFromConf(ctx, "http::addr=localhost:9000;username=admin;password=quest;auto_flush_rows=100;auto_flush_interval=1000;")
+	if err != nil {
+		panic("Failed to create client")
+	}
 
-- Data for the first table in an HTTP request will be committed even if the
-  second table's commit fails.
-- An implicit commit occurs each time a new column is added to a table. This
-  action cannot be rolled back if the request is aborted or encounters parse
-  errors.
+	timestamp := time.Now()
+	err = client.Table("trades").
+		Symbol("symbol", "ETH-USD").
+		Symbol("side", "sell").
+		Float64Column("price", 2615.54).
+		Float64Column("amount", 0.00044).
+		At(ctx, timestamp)
 
-### Timestamp column
+	if err != nil {
+		panic("Failed to insert data")
+	}
 
-QuestDB's underlying InfluxDB Line Protocol (ILP) does not name timestamps,
-leading to an automatic column name of timestamp. To use a custom name,
-pre-create the table with the desired timestamp column name:
+	err = client.Flush(ctx)
+	// You can flush manually at any point.
+	// If you don't flush manually, the client will flush automatically
+	// when a row is added and either:
+	//   * The buffer contains 75000 rows (if HTTP) or 600 rows (if TCP)
+	//   * The last flush was more than 1000ms ago.
+	// Auto-flushing can be customized via the `auto_flush_..` params.
 
-```sql
-CREATE TABLE temperatures (
-    ts timestamp,
-    sensorID symbol,
-    sensorLocation symbol,
-    reading double
-) timestamp(my_ts);
+	if err != nil {
+		panic("Failed to flush data")
+	}
+}
 ```
+We recommended to use User-assigned timestamps when ingesting data into QuestDB.
+ Using the current timestamp hinder the ability to deduplicate rows which is
+[important for exactly-once processing](/docs/reference/api/ilp/overview/#exactly-once-delivery-vs-at-least-once-delivery).
 
-## Health check
+## Configuration options
 
-To monitor your active connection, there is a `ping` endpoint:
+The minimal configuration string needs to have the protocol, host, and port, as in:
 
-```shell
-curl -I http://localhost:9000/ping
+```
+http::addr=localhost:9000;
 ```
 
-Returns (pong!):
+In the Go client, you can set the configuration options via the standard config string,
+which is the same across all clients, or using [the built-in API](https://pkg.go.dev/github.com/questdb/go-questdb-client/v3#LineSenderOption).
 
-```shell
-HTTP/1.1 204 OK
-Server: questDB/1.0
-Date: Fri, 2 Feb 2024 17:09:38 GMT
-Transfer-Encoding: chunked
-Content-Type: text/plain; charset=utf-8
-X-Influxdb-Version: v2.7.4
-```
+For all the extra options you can use, please check [the client docs](https://pkg.go.dev/github.com/questdb/go-questdb-client/v3#LineSenderFromConf)
 
-Determine whether an instance is active and confirm the version of InfluxDB Line
-Protocol with which you are interacting.
 
 ## Next Steps
 
+Please refer to the [ILP overview](/docs/reference/api/ilp/overview) for details
+about transactions, error control, delivery guarantees, health check, or table and
+column auto-creation.
+
 Explore the full capabilities of the Go client via
 [Go.dev](https://pkg.go.dev/github.com/questdb/go-questdb-client/v3).
 

clients/ingest-node.md

@@ -41,10 +41,41 @@ Install the QuestDB Node.js client via npm:
 npm i -s @questdb/nodejs-client
 ```
 
-## Basic Usage
+## Authentication
+
+Passing in a configuration string with basic auth:
+
+```javascript
+const { Sender } = require("@questdb/nodejs-client");
+
+const conf = "http::addr=localhost:9000;username=admin;password=quest;"
+const sender = Sender.fromConfig(conf);
+    ...
+```
+
+Passing via the `QDB_CLIENT_CONF` env var:
+
+```bash
+export QDB_CLIENT_CONF="http::addr=localhost:9000;username=admin;password=quest;"
+```
+
+```javascript
+const { Sender } = require("@questdb/nodejs-client");
+
+
+const sender = Sender.fromEnv();
+    ...
+```
+
+When using QuestDB Enterprise, authentication can also be done via REST token.
+Please check the [RBAC docs](/docs/operations/rbac/#authentication) for more info.
+
+## Basic insert
+
+Example: inserting executed trades for cryptocurrencies.
+
+Without authentication and using the current timestamp.
 
-A simple example to connect to QuestDB, insert some data into a table, and flush
-the data:
 
 ```javascript
 const { Sender } = require("@questdb/nodejs-client")
@@ -55,42 +86,96 @@ async function run() {
 
   // add rows to the buffer of the sender
   await sender
-    .table("prices")
-    .symbol("instrument", "EURUSD")
-    .floatColumn("bid", 1.0195)
-    .floatColumn("ask", 1.0221)
-    .at(Date.now(), "ms")
-  await sender
-    .table("prices")
-    .symbol("instrument", "GBPUSD")
-    .floatColumn("bid", 1.2076)
-    .floatColumn("ask", 1.2082)
-    .at(Date.now(), "ms")
+    .table("trades")
+    .symbol("symbol", "ETH-USD")
+    .symbol("side", "sell")
+    .floatColumn("price", 2615.54)
+    .floatColumn("amount", 0.00044)
+    .atNow()
 
   // flush the buffer of the sender, sending the data to QuestDB
   // the buffer is cleared after the data is sent, and the sender is ready to accept new data
   await sender.flush()
 
-  // add rows to the buffer again, and send it to the server
+  // close the connection after all rows ingested
+  // unflushed data will be lost
+  await sender.close()
+}
+
+run().then(console.log).catch(console.error)
+```
+
+In this case, the designated timestamp will be the one at execution time. Let's see now an example with an explicit
+timestamp, custom auto-flushing, and basic auth.
+
+
+```javascript
+const { Sender } = require("@questdb/nodejs-client")
+
+async function run() {
+  // create a sender using HTTP protocol
+  const sender = Sender.fromConfig(
+    "http::addr=localhost:9000;username=admin;password=quest;auto_flush_rows=100;auto_flush_interval=1000;"
+    )
+
+  // Calculate the current timestamp. You could also parse a date from your source data.
+    const timestamp = Date.now();
+
+  // add rows to the buffer of the sender
+  await sender
+    .table("trades")
+    .symbol("symbol", "ETH-USD")
+    .symbol("side", "sell")
+    .floatColumn("price", 2615.54)
+    .floatColumn("amount", 0.00044)
+    .at(timestamp, "ms")
+
+  // add rows to the buffer of the sender
   await sender
-    .table("prices")
-    .symbol("instrument", "EURUSD")
-    .floatColumn("bid", 1.0197)
-    .floatColumn("ask", 1.0224)
-    .at(Date.now(), "ms")
+    .table("trades")
+    .symbol("symbol", "BTC-USD")
+    .symbol("side", "sell")
+    .floatColumn("price", 39269.98)
+    .floatColumn("amount", 0.001)
+    .at(timestamp, "ms")
+
+
+  // flush the buffer of the sender, sending the data to QuestDB
+  // the buffer is cleared after the data is sent, and the sender is ready to accept new data
   await sender.flush()
 
+
   // close the connection after all rows ingested
+  // unflushed data will be lost
   await sender.close()
 }
 
 run().then(console.log).catch(console.error)
 ```
 
+As you can see, both events now are using the same timestamp. We recommended to use the original event timestamps when
+ingesting data into QuestDB. Using the current timestamp hinder the ability to deduplicate rows which is
+[important for exactly-once processing](/docs/reference/api/ilp/overview/#exactly-once-delivery-vs-at-least-once-delivery).
+
+
+## Configuration options
+
+The minimal configuration string needs to have the protocol, host, and port, as in:
+
+```
+http::addr=localhost:9000;
+```
+
+For all the extra options you can use, please check [the client docs](https://questdb.github.io/nodejs-questdb-client/SenderOptions.html)
+
+
 ## Next Steps
 
-Dive deeper into the Node.js client capabilities by exploring more examples
-provided in the
+Please refer to the [ILP overview](/docs/reference/api/ilp/overview) for details
+about transactions, error control, delivery guarantees, health check, or table and
+column auto-creation.
+
+Dive deeper into the Node.js client capabilities, including TypeScript and Worker Threads examples, by exploring the
 [GitHub repository](https://github.com/questdb/nodejs-questdb-client).
 
 To learn _The Way_ of QuestDB SQL, see the