diff --git a/clients/date-to-timestamp-conversion.md b/clients/date-to-timestamp-conversion.md new file mode 100644 index 00000000..18898178 --- /dev/null +++ b/clients/date-to-timestamp-conversion.md @@ -0,0 +1,394 @@ +--- +title: Date to Timestamp Conversion +sidebar_label: Date to Timestamp +description: Python, Go, JAVA, JavaScript, C/C++, Rust, .Net, PHP, or Ruby. +--- + + +# Date to Timestamp Conversion in Different Programming Languages + +Most languages have a dedicated type for dates or timestamps, with the notable exception of C. In this guide, we show how to convert from a literal string representing a date into the native `Date` type, and then +into a `Timestamp` type using Python, Go, Java, C, C++, Rust, C#/.NET, JavaScript/Node.js, Ruby, and PHP. + +QuestDB offers clients for Python, Go, Java, C, C++, Rust, C#/.NET, and JavaScript/Node.js. Some of the clients +can directly use a `Timestamp` type when using the client, while others need to convert the timestamp into a +long representing the epoch time in microseconds. We add such required conversions into the snippets. + +Please refer to the [ingestion overview](https://questdb.io/docs/ingestion-overview/) to learn more about the details of the client library for your language. + +## Date to Timestamp in Python + +The `datetime.date` type stores only date information, while `datetime.datetime` stores both date and time information. +The QuestDB Python client accepts either a `datetime.datetime` object, or a `pandas.timestamp`. + +```python +from datetime import datetime, date +import pandas as pd +from questdb.ingress import Sender + +date_str = '2024-08-05' + +# We could parse the string directly into a datetime object, +# but for this example, we will show how to convert it via a date. +date_obj = datetime.strptime(date_str, '%Y-%m-%d').date() +print(f"Date object: {date_obj}") + +# Convert to datetime object. This object can be passed as a column value in the QuestDB Python client's row() API. +datetime_obj = datetime.combine(date_obj, datetime.min.time()) +print(f"DateTime object: {datetime_obj}") + +# Now you can pass the datetime_obj to the QuestDB sender, as in +# ... +# columns={'NonDesignatedTimestampColumnName': datetime_obj, +# ... + + +# Optionally, you can convert to a pandas Timestamp. The QuestDB Python client offers the dataframe() API, which accepts pd.Timestamp columns +pd_timestamp = pd.Timestamp(datetime_obj) +print(f"Pandas Timestamp: {pd_timestamp}") + +``` +Learn more about the [QuestDB Python Client](/docs/clients/ingest-python/) + +## Date to Timestamp in Go + +The `time.Time` type stores both date and time information. It is used for most time-related tasks in Go, such as +parsing dates, formatting dates, and time arithmetic. + +The QuestDB Go client expects the timestamp as an `int64`, in unix microseconds, so we will need to convert it. + +```go +package main + +import ( + "fmt" + "time" + "github.com/questdb/go-questdb-client/v3" +) + +func main() { + dateStr := "2024-08-05" + + // Layout string to match the format of dateStr + layout := "2006-01-02" + + // Parse the date string into a time.Time object + dateObj, err := time.Parse(layout, dateStr) + if err != nil { + fmt.Println("Error parsing date:", err) + return + } + + // Convert the dateObj to a timestamp in microseconds + timestamp := dateObj.UnixNano() / int64(time.Microsecond) + + fmt.Println("Date:", dateObj) + fmt.Println("Timestamp (microseconds):", timestamp) + + // Now you can call add the column to the QuestDB client, as in + // ... + // TimestampColumn("NonDesignatedTimestampColumnName", timestamp). + // ... +} + +``` +Learn more about the [QuestDB Go Client](/docs/clients/ingest-go/) + +## Date to Timestamp in Java + +The `java.time.LocalDate` type stores only date information, while `java.time.LocalDateTime` stores both date and time +information. + +The QuestDB Java Client expects either a `java.time.Instant` object or a `long` value representing an epoch number. When using a `long`, you must provide the time unit using `java.time.temporal.ChronoUnit`. + +Example using `Instant` + +```java +import java.time.LocalDate; +import java.time.ZoneId; +import java.time.Instant; +import java.time.format.DateTimeFormatter; +import io.questdb.client.Sender; + +public class Main { + public static void main(String[] args) { + String dateStr = "2024-08-05"; + DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd"); + // Note we are only converting to Date for educational purposes, but we really need the timestamp for QuestDB + LocalDate date = LocalDate.parse(dateStr, formatter); + + // Convert LocalDate to Instant at the start of the day in the default time zone + Instant instant = date.atStartOfDay(ZoneId.systemDefault()).toInstant(); + + System.out.println("Date: " + date); + System.out.println("Instant: " + instant); + + // Example method call using QuestDB API + try (Sender sender = Sender.fromConfig("http::addr=localhost:9000;")) { + sender.table("trades") + .symbol("symbol", "ETH-USD") + .symbol("side", "sell") + .timestampColumn("NonDesignatedTimestampColumnName", instant) + .doubleColumn("price", 2615.54) + .doubleColumn("amount", 0.00044) + .atNow(); + } +} +``` + +Example using a `long` with epoch microseconds and `java.time.temporal.ChronoUnit` + +```java +import java.time.LocalDate; +import java.time.LocalDateTime; +import java.time.ZoneOffset; +import java.time.format.DateTimeFormatter; +import java.time.temporal.ChronoUnit; +import io.questdb.client.Sender; + +public class Main { + public static void main(String[] args) { + String dateStr = "2024-08-05"; + DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd"); + // Note we are only converting to Date for educational purposes, but we really need the timestamp for QuestDB + LocalDate date = LocalDate.parse(dateStr, formatter); + + // Convert LocalDate to LocalDateTime at the start of the day + LocalDateTime dateTime = date.atStartOfDay(); + + // Convert LocalDateTime to epoch microseconds + long epochMicro = dateTime.toInstant(ZoneOffset.UTC).toEpochMilli() * 1000; + + System.out.println("Epoch in Microseconds: " + epochMicro); + + // Example method call using QuestDB API + try (Sender sender = Sender.fromConfig("http::addr=localhost:9000;")) { + sender.table("trades") + .symbol("symbol", "ETH-USD") + .symbol("side", "sell") + .timestampColumn("NonDesignatedTimestampColumnName", epochMicro, ChronoUnit.MICROS) + .doubleColumn("price", 2615.54) + .doubleColumn("amount", 0.00044) + .atNow(); + + } + +} +``` + +Learn more about the [QuestDB Java Client](/docs/clients/java-ilp/) + +## Date to Timestamp in C + +Standard C does not have a native `Date type`, but `struct tm` in `` can store both date and time information. + +The QuestDB C client expects timestamp to be an `int64_t` in microseconds, so we will need to convert it. + +```c +#include +#include +#include +#include + + +int main() { + char dateStr[] = "2024-08-05"; + struct tm tm = {0}; + strptime(dateStr, "%Y-%m-%d", &tm); + + // Convert to time_t (seconds since Epoch) + time_t seconds = mktime(&tm); + + // Convert to microseconds + int64_t microseconds = (int64_t)seconds * 1000000; + printf("Date: %s, Timestamp (microseconds): %ld +", dateStr, microseconds); + + // you can now pass the microseconds variable to the QuestDB client line_sender_buffer_column_ts_micros function + + return 0; +} +``` + +Learn more about the [QuestDB C Client](/docs/clients/ingest-c-and-cpp/#c-1) + +## Date to Timestamp in C++ + +The `std::chrono::year_month_da`y type from C++20 stores only date information, while `std::chrono::time_point` stores +both date and time information. + +The QuestDB C++ client accepts an int64_t as microseconds, but also a +`std::chrono::time_point` or a `std::chrono::time_point`. + +```cpp +#include +#include +#include +#include +#include + + +int main() { + std::string dateStr = "2024-08-05"; + std::istringstream iss(dateStr); + std::tm tm = {}; + iss >> std::get_time(&tm, "%Y-%m-%d"); + + // Convert to time_point (timestamp) + auto tp = std::chrono::system_clock::from_time_t(std::mktime(&tm)); + auto microseconds = std::chrono::duration_cast(tp.time_since_epoch()).count(); + + std::cout << "Date: " << std::put_time(&tm, "%Y-%m-%d") << ", Timestamp (microseconds): " << microseconds << std::endl; + + // You can now pass the microseconds variable to the QuestDB column function, converting to int64_t using the + // timestamp_micros function + // .column("NotDesignatedTimestampColumnName",questdb::ingress::timestamp_micros(microseconds) + + return 0; +} +``` +Learn more about the [QuestDB C++ Client](/docs/clients/ingest-c-and-cpp/) + +## Date to Timestamp in Rust + +The `chrono::NaiveDate` type stores only date information, while `chrono::NaiveDateTime` stores both date and time +information. + +The QuestDB Rust client accepts either a `i64` Epoch in microseconds, or a `chrono::Datetime`. + +```rust +extern crate chrono; +use questdb::ingress::{Sender, Buffer, TimestampMicros} +use chrono::{NaiveDate, NaiveDateTime}; + +fn main() { + let date_str = "2024-08-05"; + let date_obj = NaiveDate::parse_from_str(date_str, "%Y-%m-%d").expect("Failed to parse date"); + + // Convert to NaiveDateTime for timestamp + let datetime = date_obj.and_hms(0, 0, 0); + let timestamp = datetime.timestamp_micros(); + + println!("Date: {}", date_obj); + println!("Timestamp (microseconds): {}", timestamp); + + // You can now use this timestamp to call the .column_ts QuestDB API + // .column_ts("NonDesignatedTimestampColumnName", TimestampMicros::new(timestamp))? + + + +} +``` +Learn more about the [QuestDB Rust Client](/docs/clients/ingest-rust/) + +## Date to Timestamp in C#/.NET + +The `System.DateTime` type stores both date and time information. There is also `System.DateOnly` for only date information +in .NET 6 and later. + +The QuestDB Dotnet client accepts `DateTime` or `DateTimeOffset` objects. Showing both options below. + +```csharp +using System; +using QuestDB; + + +class Program +{ + static void Main() + { + string dateStr = "2024-08-05"; + + // Parse the date string into a DateTime object + DateTime date = DateTime.ParseExact(dateStr, "yyyy-MM-dd", null); + Console.WriteLine("DateTime: " + date); + + // You can now call the QuestDB API adding a column, as in + // sender.Column("NonDesignatedTimestampColumnName", date); + + + // Parse the date string into a DateTimeOffset object + DateTimeOffset dateOffset = DateTimeOffset.ParseExact(dateStr, "yyyy-MM-dd", null); + Console.WriteLine("DateTimeOffset: " + dateOffset); + + // You can now call the QuestDB API adding a column, as in + // sender.Column("NonDesignatedTimestampColumnName", dateOffset); + } +} + +``` +Learn more about the [QuestDB .NET Client](/docs/clients/ingest-dotnet/) + +## Date to Timestamp in JavasScript/Node.js + +The Date type stores both date and time information. + +The QuestDB Node.js client accepts an epoch in microseconds, which can be a `number` or `bigint`. + +```javascript +const { Sender } = require("@questdb/nodejs-client") + +const dateStr = '2024-08-05'; +const dateObj = new Date(dateStr + 'T00:00:00Z'); + +// Convert to timestamp (milliseconds since Epoch) then convert to microseconds +const timestamp = BigInt(dateObj.getTime()) * 1000n; +console.log("Date:", dateObj.toISOString().split('T')[0]); +console.log("Timestamp (microseconds):", timestamp.toString()); + +// You can now add the column using QuestDB client, as in +// .timestampColumn("NonDesignatedTimestampColumnName", timestamp) +``` + +Learn more about the [QuestDB Node.js Client](/docs/clients/ingest-node/) + +## Date to Timestamp in Ruby + +The `Date` class stores only date information, while the `DateTime` class stores both date and time information. + +QuestDB does not have an official Ruby client, but you can send a POST request comprising the ILP messages. Within this messages, you can pass an epoch timestamp in nanoseconds as the designated timestamp, and pass epoch timestamps in microseconds for other timestamp columns. + +Alternatively, you can use the [InfluxDB Ruby Client](https://github.com/influxdata/influxdb-client-ruby), which is compatible with QuestDB. + +```ruby +require 'date' + +date_str = '2024-08-05' +date_obj = Date.parse(date_str) + +# Convert to DateTime for timestamp in microseconds +datetime_obj = DateTime.parse(date_str) +timestamp = (datetime_obj.to_time.to_i * 1_000_000) + (datetime_obj.to_time.usec) + +puts "Date: #{date_obj}" +puts "Timestamp (microseconds): #{timestamp}" +``` + +Learn more about the [ILP text format](/docs/reference/api/ilp/advanced-settings/). + + +## Date to Timestamp in PHP + +Both of the `DateTime` and `DateTimeImmutable` classes store date and time information + +QuestDB does not have an official PHP client, but you can send a POST request comprising the ILP messages. Within this messages, you can pass an epoch timestamp in nanoseconds as the designated timestamp, and pass epoch timestamps in microseconds for other timestamp columns. + +Alternatively, you can use the [InfluxDB PHP Client](https://github.com/influxdata/influxdb-client-php), which is compatible with QuestDB. + +```php +getTimestamp() * 1000000; + +echo "Date: " . $date_obj->format('Y-m-d') . " +"; +echo "Timestamp (microseconds): " . $timestamp . " +"; +?> +``` + +Learn more about the [ILP text format](/docs/reference/api/ilp/advanced-settings/). + diff --git a/reference/function/date-time.md b/reference/function/date-time.md index e37c02f6..949f3723 100644 --- a/reference/function/date-time.md +++ b/reference/function/date-time.md @@ -65,9 +65,37 @@ The interpretation of `y` depends on the input digit number: | `005-03` | `0005-03-01T00:00:00.000000Z` | Greedily parsing the number as it is | | `0005-03` | `0005-03-01T00:00:00.000000Z` | Greedily parsing the number as it is | +## Timestamp to Date conversion + +As described at the [data types section](/docs/reference/sql/datatypes), the only difference between `Timestamp` and `Date` +in QuestDB type system is the resolution. Whilst `Timestamp` stores resolution as an offset from Unix epoch in microseconds, +`Date` stores the offset in milliseconds. + +Since both types are backed by a signed long, this means the `DATE` type has a +wider range. A `DATE` column can store about ±2.9 million years from the Unix epoch, whereas a `TIMESTAMP` has an approximate +range of ±290,000 years. + +For most purposes a `TIMESTAMP` is preferred, as it offers a wider range of functions whilst still being 8 bytes in size. + +Be aware that, when using a `TIMESTAMP` as the designated timestamp, you cannot set it to any value before the Unix epoch +(`1970-01-01T00:00:00.000000Z`). + +To explicitely convert from `TIMESTAMP` to `DATE` you can use `CAST(ts_column AS DATE)`. To convert +from `DATE` to `TIMESTAMP` you can `CAST(date_column AS TIMESTAMP)`. + +### Programmatically convert from language-specific datetimes into QuestDB timestamps + +Different programming languages use different types of objects to represent the `Date` type. To learn how to convert +from the `Date` type into a `Timestamp` object in Python, Go, Java, JavaScript, C/C++, Rust, or C#/.NET, please visit +our [Date to Timestamp conversion](/docs/clients/date-to-timestamp-conversion) reference. + +--- + +# Function Reference + ## date_trunc -`date_trunc(unit, timestamp)` - returns a timestamps truncated to the selected +`date_trunc(unit, timestamp)` - returns a timestamp truncated to the specified precision **Arguments:** @@ -299,6 +327,33 @@ SELECT to_str(ts,'EE'),day_of_week_sunday_first(ts) FROM myTable; | Saturday | 7 | | Sunday | 1 | +## days_in_month + +`days_in_month(value)` - returns the number of days in a month from a given +timestamp or date. + +**Arguments:** + +- `value` is any `timestamp` or `date` + +**Return value:** + +Return value type is `int` + +**Examples:** + +```questdb-sql +SELECT month(ts), days_in_month(ts) FROM myTable; +``` + +| month | days_in_month | +| :---- | :------------ | +| 4 | 30 | +| 5 | 31 | +| 6 | 30 | +| 7 | 31 | +| 8 | 31 | + ## extract `extract (unit, timestamp)` - returns the selected time unit from the input @@ -413,33 +468,6 @@ SELECT year(ts), is_leap_year(ts) FROM myTable; | 2024 | true | | 2025 | false | -## days_in_month - -`days_in_month(value)` - returns the number of days in a month from a provided -timestamp or date. - -**Arguments:** - -- `value` is any `timestamp` or `date` - -**Return value:** - -Return value type is `int` - -**Examples:** - -```questdb-sql -SELECT month(ts), days_in_month(ts) FROM myTable; -``` - -| month | days_in_month | -| :---- | :------------ | -| 4 | 30 | -| 5 | 31 | -| 6 | 30 | -| 7 | 31 | -| 8 | 31 | - ## micros `micros(value)` - returns the `micros` of the millisecond for a given date or @@ -708,83 +736,83 @@ SELECT second(ts), count() FROM transactions; | 58 | 9876 | | 59 | 2567 | -## systimestamp +## sysdate -`systimestamp()` - offset from UTC Epoch in microseconds. Calculates -`UTC timestamp` using system's real time clock. The value is affected by -discontinuous jumps in the system time (e.g., if the system administrator -manually changes the system time). +`sysdate()` - returns the timestamp of the host system as a `date` with +`millisecond` precision. -`systimestamp()` value can change within the query execution timeframe and -should **NOT** be used in WHERE clause to filter designated timestamp column. +Calculates `UTC date` with millisecond precision using system's real time clock. +The value is affected by discontinuous jumps in the system time (e.g., if the +system administrator manually changes the system time). + +`sysdate()` value can change within the query execution timeframe and should +**NOT** be used in WHERE clause to filter designated timestamp column. :::tip -Use now() with WHERE clause filter. +Use `now()` with WHERE clause filter. ::: **Arguments:** -- `systimestamp()` does not accept arguments. +- `sysdate()` does not accept arguments. **Return value:** -Return value type is `timestamp`. +Return value type is `date`. **Examples:** -```questdb-sql title="Insert current system timestamp" +```questdb-sql title="Insert current system date along with a value" INSERT INTO readings -VALUES(systimestamp(), 123.5); +VALUES(sysdate(), 123.5); ``` -| ts | reading | +| sysdate | reading | | :-------------------------- | :------ | | 2020-01-02T19:28:48.727516Z | 123.5 | -## sysdate +```questdb-sql title="Query based on last minute" +SELECT * FROM readings +WHERE date_time > sysdate() - 60000000L; +``` -`sysdate()` - returns the timestamp of the host system as a `date` with -`millisecond` precision. +## systimestamp -Calculates `UTC date` with millisecond precision using system's real time clock. -The value is affected by discontinuous jumps in the system time (e.g., if the -system administrator manually changes the system time). +`systimestamp()` - offset from UTC Epoch in microseconds. Calculates +`UTC timestamp` using system's real time clock. The value is affected by +discontinuous jumps in the system time (e.g., if the system administrator +manually changes the system time). -`sysdate()` value can change within the query execution timeframe and should -**NOT** be used in WHERE clause to filter designated timestamp column. +`systimestamp()` value can change within the query execution timeframe and +should **NOT** be used in WHERE clause to filter designated timestamp column. :::tip -Use `now()` with WHERE clause filter. +Use now() with WHERE clause filter. ::: **Arguments:** -- `sysdate()` does not accept arguments. +- `systimestamp()` does not accept arguments. **Return value:** -Return value type is `date`. +Return value type is `timestamp`. **Examples:** -```questdb-sql title="Insert current system date along with a value" +```questdb-sql title="Insert current system timestamp" INSERT INTO readings -VALUES(sysdate(), 123.5); +VALUES(systimestamp(), 123.5); ``` -| sysdate | reading | +| ts | reading | | :-------------------------- | :------ | | 2020-01-02T19:28:48.727516Z | 123.5 | -```questdb-sql title="Query based on last minute" -SELECT * FROM readings -WHERE date_time > sysdate() - 60000000L; -``` - ## timestamp_ceil `timestamp_ceil(unit, timestamp)` - performs a ceiling calculation on a diff --git a/sidebars.js b/sidebars.js index e5536b2c..3c6de3ea 100644 --- a/sidebars.js +++ b/sidebars.js @@ -68,6 +68,11 @@ module.exports = { type: "doc", label: ".NET", }, + { + id: "clients/date-to-timestamp-conversion", + type: "doc", + label: "Date to Timestamp Conversion", + }, ], }, {