fix: populate users fields for sentAt, timestamp and originalTimestamp #3753
Conversation
|
Test report for this run is available at: https://test-integrations-dev.s3.amazonaws.com/integrations-test-reports/rudder-transformer/3753/test-report.html |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## hotfix/24092024 #3753 +/- ##
================================================
Coverage 88.69% 88.70%
================================================
Files 594 594
Lines 32413 32416 +3
Branches 7727 7728 +1
================================================
+ Hits 28750 28753 +3
- Misses 3342 3372 +30
+ Partials 321 291 -30 ☔ View full report in Codecov by Sentry. |
|
| @@ -987,11 +987,23 @@ function processWarehouseMessage(message, options) { | |||
|
|
|||
| const usersEvent = { ...commonProps }; | |||
| const usersColumnTypes = {}; | |||
| let userColumnMappingRules = whUserColumnMappingRules; | |||
| if (!isDataLakeProvider(options.provider)) { | |||
There was a problem hiding this comment.
We are populating it for all non Datalake destinations because there are few destinations like CLICKHOUSE, RS, POSTGRES where load files are being used in certain configurations to load the data. Here we can see there are 3 empty fields present which are sentAt, timestamp and originalTimestamp.
- UploadJobUsersAppendRecordsUsingUsersLoadFiles
- UploadJobUsersRecordsUsingUsersLoadFilesForClickhouse
Once these changes are merged we would need to update these test cases as well in rudder-server.
What are the changes introduced in this PR?
sent_at,timestampandoriginal_timestampfor non-datalake warehouses,What is the related Linear task?
Developer checklist
Reviewer checklist