Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: onboard emersys destination #3318

Merged
merged 27 commits into from
May 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
7ee45ee
feat: emersys initial commit
shrouti1507 Apr 26, 2024
4db2602
feat: emersys initial batching commit
shrouti1507 Apr 27, 2024
0a608b8
feat: emersys initial batching commit 2
shrouti1507 Apr 28, 2024
33b87be
feat: unit test cases with minor edits
shrouti1507 Apr 29, 2024
3522abf
fix: emersys unit test cases
shrouti1507 Apr 30, 2024
ed9e0cf
Merge branch 'develop' of github.com:rudderlabs/rudder-transformer in…
shrouti1507 Apr 30, 2024
f20b403
fix: adding support for track call
shrouti1507 Apr 30, 2024
f3fe891
fix: adding network hand;er support
shrouti1507 May 1, 2024
0610d38
fix: track batch handled
shrouti1507 May 2, 2024
04bf32c
fix: latest changes
shrouti1507 May 2, 2024
608469b
refactor: remove cdk v1 imports
koladilip May 2, 2024
68ab695
fix: identify batching final
shrouti1507 May 2, 2024
54ed0b2
fix: all batching final
shrouti1507 May 3, 2024
8b05a62
fix: cide clean up
shrouti1507 May 3, 2024
acf6eeb
fix: router test cases finalised
shrouti1507 May 4, 2024
110aff2
small edit
May 5, 2024
00b5f4f
fix: processor test cases finalised
shrouti1507 May 5, 2024
2bf77a8
fix: data delivery test cases
shrouti1507 May 6, 2024
9fe98aa
fix: adding user deletion implementation
shrouti1507 May 6, 2024
dcde00f
fix: small edit
shrouti1507 May 6, 2024
83cd772
fix: edit in network handler
shrouti1507 May 6, 2024
1751f4a
fix: data delivery test cases finalised
May 6, 2024
3694148
fix: editing unit test cases
shrouti1507 May 7, 2024
5a058bf
fix: resolve fake timer clash
shrouti1507 May 7, 2024
1452ed4
fix: add user deletion test cases
shrouti1507 May 7, 2024
b491435
fix: doc links added
shrouti1507 May 10, 2024
305e231
fix: review comments addressed
shrouti1507 May 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions src/cdk/v2/destinations/emarsys/config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
const ALLOWED_OPT_IN_VALUES = ['1', '2', ''];
const groupedSuccessfulPayload = {
identify: {
method: 'PUT',
batches: [],
},
group: {
method: 'POST',
batches: [],
},
track: {
method: 'POST',
batches: [],
},
};

module.exports = {
MAX_BATCH_SIZE: 1000,
EMAIL_FIELD_ID: 3,
OPT_IN_FILED_ID: 31,
ALLOWED_OPT_IN_VALUES,
MAX_BATCH_SIZE_BYTES: 8000000, // 8 MB,
groupedSuccessfulPayload,
};
88 changes: 88 additions & 0 deletions src/cdk/v2/destinations/emarsys/procWorkflow.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
bindings:
- name: EventType
path: ../../../../constants
- path: ../../bindings/jsontemplate
exportAll: true
- name: removeUndefinedValues
path: ../../../../v0/util
- name: removeUndefinedAndNullValues
path: ../../../../v0/util
- name: defaultRequestConfig
path: ../../../../v0/util
- name: getIntegrationsObj
path: ../../../../v0/util
- name: getFieldValueFromMessage
path: ../../../../v0/util
- name: CommonUtils
path: ../../../../util/common
- path: ./utils
- path: ./config
- path: lodash
name: cloneDeep

steps:
- name: checkIfProcessed
condition: .message.statusCode
template: |
$.batchMode ? .message.body.JSON : .message
onComplete: return
- name: messageType
template: |
.message.type.toLowerCase()
- name: validateInput
template: |
let messageType = $.outputs.messageType;
$.assert(messageType, "Message type is not present. Aborting message.");
$.assert(messageType in {{$.EventType.([.TRACK, .IDENTIFY, .GROUP])}},
"message type " + messageType + " is not supported")
$.assertConfig(.destination.Config.emersysUsername, "Emersys user name is not configured. Aborting");
$.assertConfig(.destination.Config.emersysUserSecret, "Emersys user secret is not configured. Aborting");

- name: validateInputForTrack
description: Additional validation for Track events
condition: $.outputs.messageType === {{$.EventType.TRACK}}
template: |
$.assert(.message.event, "event could not be mapped to conversion rule. Aborting.")
- name: preparePayloadForIdentify
description: |
Builds identify payload. ref: https://dev.emarsys.com/docs/core-api-reference/f8ljhut3ac2i1-update-contacts
condition: $.outputs.messageType === {{$.EventType.IDENTIFY}}
template: |
$.context.payload = $.buildIdentifyPayload(.message, .destination.Config,);
- name: preparePayloadForGroup
description: |
Builds group payload. ref: https://dev.emarsys.com/docs/core-api-reference/1m0m70hy3tuov-add-contacts-to-a-contact-list
condition: $.outputs.messageType === {{$.EventType.GROUP}}
template: |
$.context.payload = $.buildGroupPayload(.message, .destination.Config,);
- name: preparePayloadForTrack
description: |
Builds track payload. ref: https://dev.emarsys.com/docs/core-api-reference/fl0xx6rwfbwqb-trigger-an-external-event
condition: $.outputs.messageType === {{$.EventType.TRACK}}
template: |
const properties = ^.message.properties;
const integrationObject = $.getIntegrationsObj(^.message, 'emarsys');
const emersysIdentifierId = $.deduceCustomIdentifier(integrationObject, ^.destination.Config.emersysCustomIdentifier);
const payload = {
key_id: emersysIdentifierId,
external_id: $.deduceExternalIdValue(^.message,emersysIdentifierId,.destination.Config.fieldMapping),
trigger_id: integrationObject.trigger_id,
data: properties.data,
attachment:$.CommonUtils.toArray(properties.attachment),
event_time: $.getFieldValueFromMessage(^.message, 'timestamp'),
};
$.context.payload = {
eventType: ^.message.type,
destinationPayload: {
payload: $.removeUndefinedAndNullValues(payload),
eventId: $.deduceEventId(^.message,.destination.Config),
},
};
- name: buildResponse
template: |
const response = $.defaultRequestConfig();
response.body.JSON = $.context.payload;
response.endpoint = $.deduceEndPoint($.context.payload,.destination.Config);
response.method = "POST";
response.headers = $.buildHeader(.destination.Config)
response
38 changes: 38 additions & 0 deletions src/cdk/v2/destinations/emarsys/rtWorkflow.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
bindings:
- path: ./utils
- name: handleRtTfSingleEventError
path: ../../../../v0/util/index

steps:
- name: validateInput
template: |
$.assert(Array.isArray(^) && ^.length > 0, "Invalid event array")

- name: transform
externalWorkflow:
path: ./procWorkflow.yaml
bindings:
- name: batchMode
value: true
loopOverInput: true
- name: successfulEvents
template: |
$.outputs.transform#idx.output.({
"message": .[],
"destination": ^ [idx].destination,
"metadata": ^ [idx].metadata
})[]
- name: failedEvents
template: |
$.outputs.transform#idx.error.(
$.handleRtTfSingleEventError(^[idx], .originalError ?? ., {})
)[]

- name: batchSuccessfulEvents
description: Batches the successfulEvents
template: |
$.context.batchedPayload = $.batchResponseBuilder($.outputs.successfulEvents);

- name: finalPayload
template: |
[...$.outputs.failedEvents, ...$.context.batchedPayload]
Loading
Loading