Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: onboard new destination the trade desk #2918

Merged
merged 15 commits into from
Jan 5, 2024
Merged

Conversation

Gauravudia
Copy link
Contributor

@Gauravudia Gauravudia commented Dec 15, 2023

What are the changes introduced in this PR?

Resolves INT-1167

  • Onboard the new destination: The Trade Desk
  • Onboard a destination on transformer proxy, addressing two key issues:
    • Handling error responses in the event of a 200 status code.
    • Resolving an issue caused by the server sorting payload keys, impacting the decryption of the HMAC at the Trade Desk's end. To overcome this challenge, a proxy request has been created, generating the HMAC of the sorted payload JSON keys.
  • Used new audience message type: record. This allows the server to send each record individually, with the transformer responsible for efficiently batching the events.
  • Onboard destination on router to support new audience message type: record
  • Added a utility function, splitItemsBasedOnMaxSizeInBytes, designed to split an items array containing various IDs (TDID, DAID, UID2) based on the maximum base size allowed by Trade Desk (approximately 2.5MB).
  • Implemented a utility function, getSignatureHeader, to generate the HMAC of the payload using secret keys (advertiser secret key and provided data secret key).

Please explain the objectives of your changes below

Put down any required details on the broader aspect of your changes. If there are any dependent changes, mandatorily mention them here

  1. Server changes to pass destination config in metadata is required for this Integration to work correctly. Without this change transformer will throw configuration error.
  2. Adding data provider secret key in transformer env.

Type of change

new feature

Any technical or performance related pointers to consider with the change?

N/A

Any new dependencies introduced with this change?

N/A

Any new generic utility introduced or modified. Please explain the changes.

existing utility: prepareProxyRequest in network.js is modified to return config from request. This config will be later used by calling function proxyRequest in networkHandler.js of Trade Desk

If the PR has changes in more than 10 files, please mention why the changes were not split into multiple PRs.

N/A

If multiple linear tasks are associated with the PR changes, please elaborate on the reason:

N/A


Developer checklist

  • No breaking changes are being introduced.

  • Are all related docs linked with the PR?

  • Are all changes manually tested?

  • Does this change require any documentation changes?

  • Are relevant unit and component test-cases added?

Reviewer checklist

  • Is the type of change in the PR title appropriate as per the changes?

  • Verified that there are no credentials or confidential data exposed with the changes.

Copy link
Contributor

coderabbitai bot commented Dec 15, 2023

Important

Auto Review Skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository.

To trigger a single review, invoke the @coderabbitai review command.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

Tips

Chat

There are 3 ways to chat with CodeRabbit's AI:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit-tests for this file.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit tests for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository from git and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit tests.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid.
    • @coderabbitai read the files in the src/scheduler package and generate README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • The JSON schema for the configuration file is available here.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json

CodeRabbit Discord Community

Join our Discord Community to get help, request features, and share feedback.

Copy link

codecov bot commented Dec 15, 2023

Codecov Report

Attention: 4 lines in your changes are missing coverage. Please review.

Comparison is base (8bada4b) 87.20% compared to head (b6c58a9) 87.28%.

Files Patch % Lines
src/cdk/v2/destinations/the_trade_desk/utils.js 96.72% 2 Missing ⚠️
...c/v0/destinations/the_trade_desk/networkHandler.js 93.54% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #2918      +/-   ##
===========================================
+ Coverage    87.20%   87.28%   +0.07%     
===========================================
  Files          864      872       +8     
  Lines        29373    29500     +127     
  Branches      6854     6865      +11     
===========================================
+ Hits         25616    25749     +133     
+ Misses        3411     3405       -6     
  Partials       346      346              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Gauravudia Gauravudia marked this pull request as ready for review January 2, 2024 07:47
@Gauravudia Gauravudia requested review from a team and sivashanmukh as code owners January 2, 2024 07:47
@Gauravudia Gauravudia requested review from ItsSudip and removed request for utsabc January 2, 2024 07:47
src/cdk/v2/destinations/the_trade_desk/utils.js Outdated Show resolved Hide resolved
src/cdk/v2/destinations/the_trade_desk/utils.js Outdated Show resolved Hide resolved
src/cdk/v2/destinations/the_trade_desk/utils.js Outdated Show resolved Hide resolved
src/v0/destinations/the_trade_desk/networkHandler.js Outdated Show resolved Hide resolved
koladilip
koladilip previously approved these changes Jan 5, 2024
src/features.json Outdated Show resolved Hide resolved
@koladilip
Copy link
Contributor

so we can merge this directly or we need to deploy this only to exinity? cc: @krishna2020

@koladilip koladilip added the DO NOT MERGE Don't merge this PR without having discussion label Jan 5, 2024
@krishna2020
Copy link
Collaborator

so we can merge this directly or we need to deploy this only to exinity? cc: @krishna2020

Requirement is only for exinity customer. This PR has changes only w.r.t TradeDesk destination - so good to merge too. However exinity should have dedicated transformer deployment for enabling v1 proxy - so deploying this branch also works.

yashasvibajpai
yashasvibajpai previously approved these changes Jan 5, 2024
@Gauravudia Gauravudia removed the DO NOT MERGE Don't merge this PR without having discussion label Jan 5, 2024
@Gauravudia Gauravudia dismissed stale reviews from yashasvibajpai and koladilip via b6c58a9 January 5, 2024 07:44
Copy link

sonarqubecloud bot commented Jan 5, 2024

Quality Gate Passed Quality Gate passed

The SonarCloud Quality Gate passed, but some issues were introduced.

1 New issue
0 Security Hotspots
94.1% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

@Gauravudia Gauravudia merged commit f5ad088 into develop Jan 5, 2024
15 checks passed
@Gauravudia Gauravudia deleted the feat/theTradeDesk branch January 5, 2024 09:45
anantjain45823 added a commit that referenced this pull request Jan 9, 2024
* chore(deps): bump actions/stale from 8.0.0 to 9.0.0 (#2892)

Bumps [actions/stale](https://github.com/actions/stale) from 8.0.0 to 9.0.0.
- [Release notes](https://github.com/actions/stale/releases)
- [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md)
- [Commits](actions/stale@v8.0.0...v9.0.0)

---
updated-dependencies:
- dependency-name: actions/stale
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Krishna Chaitanya <[email protected]>

* chore(deps): bump docker/build-push-action from 4.1.1 to 5.1.0 (#2839)

Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 4.1.1 to 5.1.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](docker/build-push-action@v4.1.1...v5.1.0)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Krishna Chaitanya <[email protected]>

* chore(deps): bump docker/setup-buildx-action from 2.9.1 to 3.0.0 (#2777)

Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 2.9.1 to 3.0.0.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](docker/setup-buildx-action@v2.9.1...v3.0.0)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Krishna Chaitanya <[email protected]>

* chore(deps): bump actions/setup-node from 3.7.0 to 4.0.0 (#2775)

Bumps [actions/setup-node](https://github.com/actions/setup-node) from 3.7.0 to 4.0.0.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](actions/setup-node@v3.7.0...v4.0.0)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Krishna Chaitanya <[email protected]>

* chore(deps): bump actions/checkout from 3.5.3 to 4.1.1 (#2776)

Bumps [actions/checkout](https://github.com/actions/checkout) from 3.5.3 to 4.1.1.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v3.5.3...v4.1.1)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* [Snyk] Upgrade @aws-sdk/client-s3 from 3.370.0 to 3.438.0 (#2838)

fix: upgrade @aws-sdk/client-s3 from 3.370.0 to 3.438.0

Snyk has created this PR to upgrade @aws-sdk/client-s3 from 3.370.0 to 3.438.0.

See this package in npm:
https://www.npmjs.com/package/@aws-sdk/client-s3

See this project in Snyk:
https://app.snyk.io/org/datamanagement/project/39f686fe-9e69-4343-a9ce-129e22bff288?utm_source=github&utm_medium=referral&page=upgrade-pr

Co-authored-by: snyk-bot <[email protected]>
Co-authored-by: Sai Kumar Battinoju <[email protected]>

* chore: upgrade delete old branches action

* Update create-hotfix-branch.yml

* chore: move destinations to component test suite (#2894)

* chore: move clevertap, braze and am

* chore: move intercom test cases

* chore: add component test cases for marketo and sendgrid

* chore: resolve conflicts

* chore: add component test cases for destinations without mock (#2910)

chore: add component test cases for ga4, stormly, facebook_conversion

* chore: add component test cases for some destinations part1 (#2915)

* chore: add component test cases for some destinations

* chore: add method to network.ts

* chore: added component test cases for desitnations part3 (#2927)

* chore: added component test cases for desitnations

* chore: added component test cases for wootric

* chore: add component test cases for some destinations part2 (#2917)

* chore: add component test cases for some destinations

* chore: added component test cases for desitnations

* chore: decrease coverageThreshold in jest.config.js

* chore: decrease coverageThreshold in jest.default.config.js

* fix: tiktok add missing field brand (#2942)

* feat: amplitude add support for unset (#2941)

* feat: amplitude add support for unset

* Update src/v0/destinations/am/transform.js

Co-authored-by: Yashasvi Bajpai <[email protected]>

* Update src/v0/destinations/am/transform.js

Co-authored-by: Yashasvi Bajpai <[email protected]>

* chore:comment addresed

* Update transform.js

* chore:comment addresed

* chore: added docs

* Update utils.js

---------

Co-authored-by: Yashasvi Bajpai <[email protected]>

* chore: upgrade node to 18.19 (#2940)

* chore: upgrade node to 18.19

* fix: image name

---------

Co-authored-by: Yashasvi Bajpai <[email protected]>

* feat: pass ip details for factorsAI (#2925)

* feat: handle improper type validation sync vulnerability (#2937)

* feat: handle improper type validation sync vulnerability

* feat: addressed review comments

* feat: move intercom to routerTransform (#2964)

* chore: add endpoint labels (#2944)

* chore: add endpoint labelsx1

* chore: add endpoint labelsx2

* Update src/v0/destinations/active_campaign/transform.js

Co-authored-by: Sudip Paul <[email protected]>

* Update src/v0/destinations/active_campaign/transform.js

Co-authored-by: Sankeerth <[email protected]>

* chore: fix monday path

---------

Co-authored-by: Sudip Paul <[email protected]>
Co-authored-by: Sankeerth <[email protected]>

* chore: add endpoint labels (#2951)

chore: add endpoint labelsx3

Co-authored-by: Sudip Paul <[email protected]>

* fix: error handling when payload contains toString as key (#2954)

* fix: for gainsight px only new users to have default signUp date (#2953)

* fix: only new users to have default signUp date

* fix: only new users to have default createDate

* fix: small edit in comment

* fix: small edit in comment

* fix: small edit in comment

* fix: review comments addressed

* feat(sprig): added user deletion support (#2886)

* feat(sprig): added user deletion support

* chore: destination doc link added for reference

* chore: code review changes

* chore: code review changes

* fix: enhancement and version upgrade of google ads remarketing list (#2945)

* fix: enhancement and version upgrade of garl

* fix: migrating util to google utils and adding test cases

* fix: small edit

* Update src/v0/destinations/google_adwords_remarketing_lists/networkHandler.js

Co-authored-by: Sudip Paul <[email protected]>

* fix: edit test case description

---------

Co-authored-by: Sudip Paul <[email protected]>

* fix: version upgrade of gaec from 14 to 15 (#2966)

* feat: onboard new destination the trade desk (#2918)

* feat: onboard new destination the trade desk

* feat: onboard trade desk on transformer porxy

* feat: add secret from metadata

* refactor: remove secretKey from headers in utils

* feat: using audienceId from config inplace of segmentName

* test: added testcases

* test: added testcases

* refactor: prepareProxyRequest utility and added delivery test cases

* test: add console log

* refactor: address review comments

* refactor: use existing utility to split batch on size

* chore: added mocks

* refactor: throw platform error

* chore(deps): bump actions/setup-node from 4.0.0 to 4.0.1 (#2950)

Bumps [actions/setup-node](https://github.com/actions/setup-node) from 4.0.0 to 4.0.1.
- [Release notes](https://github.com/actions/setup-node/releases)
- [Commits](actions/setup-node@v4.0.0...v4.0.1)

---
updated-dependencies:
- dependency-name: actions/setup-node
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore(deps): bump SonarSource/sonarcloud-github-action from 2.0.0 to 2.1.1 (#2931)

chore(deps): bump SonarSource/sonarcloud-github-action

Bumps [SonarSource/sonarcloud-github-action](https://github.com/sonarsource/sonarcloud-github-action) from 2.0.0 to 2.1.1.
- [Release notes](https://github.com/sonarsource/sonarcloud-github-action/releases)
- [Commits](SonarSource/sonarcloud-github-action@v2.0.0...v2.1.1)

---
updated-dependencies:
- dependency-name: SonarSource/sonarcloud-github-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: upgrade @aws-sdk/client-personalize from 3.414.0 to 3.470.0 (#2962)

fix: upgrade @aws-sdk/client-personalize from 3.414.0 to 3.470.0

Snyk has created this PR to upgrade @aws-sdk/client-personalize from 3.414.0 to 3.470.0.

See this package in npm:
https://www.npmjs.com/package/@aws-sdk/client-personalize

See this project in Snyk:
https://app.snyk.io/org/sandeep-L8FvsjCG7mBBqonjSmN48c/project/4b0f037e-0dab-4719-aeee-b46d2ae82119?utm_source=github&utm_medium=referral&page=upgrade-pr

Co-authored-by: snyk-bot <[email protected]>

* chore: upgrade @aws-sdk/lib-storage from 3.417.0 to 3.456.0 (#2911)

fix: upgrade @aws-sdk/lib-storage from 3.417.0 to 3.456.0

Snyk has created this PR to upgrade @aws-sdk/lib-storage from 3.417.0 to 3.456.0.

See this package in npm:
https://www.npmjs.com/package/@aws-sdk/lib-storage

See this project in Snyk:
https://app.snyk.io/org/sandeep-L8FvsjCG7mBBqonjSmN48c/project/4b0f037e-0dab-4719-aeee-b46d2ae82119?utm_source=github&utm_medium=referral&page=upgrade-pr

Co-authored-by: snyk-bot <[email protected]>

* chore(release): 1.53.0

* revert: fix: tiktok add missing field brand (#2977)

* Revert "fix: tiktok add missing field brand (#2942)"

This reverts commit ca76297.

* chore: update changelog

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: AASHISH MALIK <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Krishna Chaitanya <[email protected]>
Co-authored-by: nidhilashkari17 <[email protected]>
Co-authored-by: snyk-bot <[email protected]>
Co-authored-by: Sai Kumar Battinoju <[email protected]>
Co-authored-by: chandumlg <[email protected]>
Co-authored-by: Dilip Kola <[email protected]>
Co-authored-by: Sudip Paul <[email protected]>
Co-authored-by: Anant Jain <[email protected]>
Co-authored-by: Yashasvi Bajpai <[email protected]>
Co-authored-by: Sandeep Digumarty <[email protected]>
Co-authored-by: Mihir Bhalala <[email protected]>
Co-authored-by: Sankeerth <[email protected]>
Co-authored-by: Akash Chetty <[email protected]>
Co-authored-by: shrouti1507 <[email protected]>
Co-authored-by: Gauravudia <[email protected]>
Co-authored-by: GitHub Actions <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants