feat: cm360 batching support #2651
Conversation
aashishmalik
force-pushed
the
fix.cm360-timestamp
branch
from
3771b3d to
19f9829
Compare
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## develop #2651 +/- ##
===========================================
+ Coverage 86.93% 86.94% +0.01%
===========================================
Files 605 605
Lines 28167 28223 +56
Branches 6703 6715 +12
===========================================
+ Hits 24486 24539 +53
- Misses 3348 3350 +2
- Partials 333 334 +1
☔ View full report in Codecov by Sentry. |
src/v0/destinations/campaign_manager/data/CampaignManagerTrackConfig.json
Show resolved
Hide resolved
| requestJson.timestampMicros = requestJson.timestampMicros.toString(); | ||
|
|
||
| const date = new Date(requestJson.timestampMicros); | ||
| let unixTimestamp = date.getTime(); |
There was a problem hiding this comment.
Doesn't this always return in milliseconds only irrespective of input passed ?
There was a problem hiding this comment.
If input is 1668624722555000 (this is in microseconds, year 2022) , the date utility still considers it is in milliseconds and convert this to 054846-08-12T03:22:35.000Z (year 5000) , when we call getTime it return 1668624722555000, so doing no of digits check here.
Same case with 1668624722 it considers it in ms, and getTime return it in 1668624722, so need to multiply by 6
There was a problem hiding this comment.
Also, I think one alternative thing we can do if they pass timestamp in properties, we can make it mandatory to pass it in microSeconds and get rid of this conversion at our end
There was a problem hiding this comment.
function convertToMicroseconds(input) {
const timestamp = Date.parse(input);
if (!isNaN(timestamp)) {
// If the input is a valid date string, timestamp will be a number
if (input.includes("Z")) {
// ISO 8601 date string with milliseconds
return timestamp * 1000;
} else {
// ISO 8601 date string without milliseconds
return timestamp * 1000000;
}
} else if (/^\d+$/.test(input)) {
// If the input is a numeric string (assume microseconds or milliseconds)
if (input.length <= 13) {
// Length less than or equal to 13 indicates milliseconds
return parseInt(input) * 1000;
} else {
// Otherwise, assume microseconds
return parseInt(input);
}
} else {
// Invalid input
throw new Error("Invalid input format");
}
}// Example usage:
const input1 = "2023-10-11T12:34:56.789Z"; // ISO 8601 date string with milliseconds
const input2 = "1633956896789"; // Milliseconds
const input3 = "1633956896789000"; // Microseconds
console.log(convertToMicroseconds(input1)); // Output: 1633956896789000
console.log(convertToMicroseconds(input2)); // Output: 1633956896789000
console.log(convertToMicroseconds(input3)); // Output: 1633956896789000
This code defines the convertToMicroseconds function, which first checks if the input is a valid date string (ISO 8601 format). If it is, it checks whether the input has milliseconds or not and converts it accordingly. If the input is a numeric string, it checks its length to determine whether it's in milliseconds or microseconds. If the input doesn't match any of these formats, it throws an error for an invalid input format.
Source: chatgpt
There was a problem hiding this comment.
console.log(convertToMicroseconds("2022-11-17T00:22:02.903+05:30"));
o/p -->>1668624722903000000 (in nanosweconds)
will modify this a bit
aashishmalik
force-pushed
the
fix.cm360-timestamp
branch
from
19f9829 to
3e589e7
Compare
|
Kudos, SonarCloud Quality Gate passed! |
| requestJson.timestampMicros = requestJson.timestampMicros.toString(); | ||
|
|
||
| const date = new Date(requestJson.timestampMicros); | ||
| let unixTimestamp = date.getTime(); |
There was a problem hiding this comment.
function convertToMicroseconds(input) {
const timestamp = Date.parse(input);
if (!isNaN(timestamp)) {
// If the input is a valid date string, timestamp will be a number
if (input.includes("Z")) {
// ISO 8601 date string with milliseconds
return timestamp * 1000;
} else {
// ISO 8601 date string without milliseconds
return timestamp * 1000000;
}
} else if (/^\d+$/.test(input)) {
// If the input is a numeric string (assume microseconds or milliseconds)
if (input.length <= 13) {
// Length less than or equal to 13 indicates milliseconds
return parseInt(input) * 1000;
} else {
// Otherwise, assume microseconds
return parseInt(input);
}
} else {
// Invalid input
throw new Error("Invalid input format");
}
}// Example usage:
const input1 = "2023-10-11T12:34:56.789Z"; // ISO 8601 date string with milliseconds
const input2 = "1633956896789"; // Milliseconds
const input3 = "1633956896789000"; // Microseconds
console.log(convertToMicroseconds(input1)); // Output: 1633956896789000
console.log(convertToMicroseconds(input2)); // Output: 1633956896789000
console.log(convertToMicroseconds(input3)); // Output: 1633956896789000
This code defines the convertToMicroseconds function, which first checks if the input is a valid date string (ISO 8601 format). If it is, it checks whether the input has milliseconds or not and converts it accordingly. If the input is a numeric string, it checks its length to determine whether it's in milliseconds or microseconds. If the input doesn't match any of these formats, it throws an error for an invalid input format.
Source: chatgpt
| events.forEach((ev) => { | ||
| conversions.push(...ev.message.body.JSON.conversions); | ||
| metadata.push(ev.metadata); | ||
| if (ev.message.body.JSON.encryptionInfo) { |
There was a problem hiding this comment.
this will override the encryptionInfo, is this expected?
| const { destination, message } = events[0]; | ||
| // Batch event into dest batch structure | ||
| events.forEach((ev) => { | ||
| conversions.push(...ev.message.body.JSON.conversions); |
There was a problem hiding this comment.
you could keep this ev.message.body.JSON in a variable and reuse it instead of typing long strings
Description of the change
Type of change
Related issues
Checklists
Development
Code review