Skip to content

Commit

Permalink
Merge branch 'develop' into refactor.shopify
Browse files Browse the repository at this point in the history
  • Loading branch information
anantjain45823 authored Sep 8, 2023
2 parents 7db623a + b3c328e commit db09237
Show file tree
Hide file tree
Showing 28 changed files with 1,215 additions and 697 deletions.
41 changes: 41 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,47 @@

All notable changes to this project will be documented in this file. See [standard-version](https://github.com/conventional-changelog/standard-version) for commit guidelines.

### [1.40.2](https://github.com/rudderlabs/rudder-transformer/compare/v1.40.1...v1.40.2) (2023-09-06)


### Bug Fixes

* marketo bulk upload import issue ([#2559](https://github.com/rudderlabs/rudder-transformer/issues/2559)) ([752f351](https://github.com/rudderlabs/rudder-transformer/commit/752f351f02b7f7611c702d7dbcb4804972bb0970))

### [1.40.1](https://github.com/rudderlabs/rudder-transformer/compare/v1.40.0...v1.40.1) (2023-09-06)


### Bug Fixes

* **google_ads_offline_conversions:** partial failure status code issue ([abfce44](https://github.com/rudderlabs/rudder-transformer/commit/abfce44067dbcefe7f2db90a5bd8e2895fd49ea9))
* **google_ads_offline_conversions:** partial failure status code issue ([#2552](https://github.com/rudderlabs/rudder-transformer/issues/2552)) ([ae90087](https://github.com/rudderlabs/rudder-transformer/commit/ae900872680fd258dbb7cf10d5bfe6f02def94a5))
* type issue in lookup via externalid, row lock error as retryable ([#2553](https://github.com/rudderlabs/rudder-transformer/issues/2553)) ([319ff90](https://github.com/rudderlabs/rudder-transformer/commit/319ff903059f21f8b11df3e984547a82f35e7ceb))
* update datafile lookup error message ([#2555](https://github.com/rudderlabs/rudder-transformer/issues/2555)) ([c4aff36](https://github.com/rudderlabs/rudder-transformer/commit/c4aff3626a1f75059bd6a09edff1e38b4e6fc4e4))

## [1.40.0](https://github.com/rudderlabs/rudder-transformer/compare/v1.39.1...v1.40.0) (2023-09-04)


### Features

* add eu instance support to kustomer destination ([#2513](https://github.com/rudderlabs/rudder-transformer/issues/2513)) ([34dbabf](https://github.com/rudderlabs/rudder-transformer/commit/34dbabfcec610b87a0a1512743528bef2e4b69ae))
* blank audience support in google ads ([#2526](https://github.com/rudderlabs/rudder-transformer/issues/2526)) ([54d3704](https://github.com/rudderlabs/rudder-transformer/commit/54d3704a9dea612b98735f7191351e8195af205a))
* eloqua new destination cdkv2 ([#2501](https://github.com/rudderlabs/rudder-transformer/issues/2501)) ([1db0573](https://github.com/rudderlabs/rudder-transformer/commit/1db0573eff0a0091248ffa2107fd8064a03ce2dd))
* **ga4:** added support of campaign_details event ([#2542](https://github.com/rudderlabs/rudder-transformer/issues/2542)) ([95920b8](https://github.com/rudderlabs/rudder-transformer/commit/95920b8a851e1e78a7154dae222033c7f34b3c09))
* **posthog:** support timestamp mapping from properties ([#2507](https://github.com/rudderlabs/rudder-transformer/issues/2507)) ([88392d7](https://github.com/rudderlabs/rudder-transformer/commit/88392d70b73525a15933e5a83a25df7d6c9417ee))
* retl audience support google ads ([#2530](https://github.com/rudderlabs/rudder-transformer/issues/2530)) ([804aa79](https://github.com/rudderlabs/rudder-transformer/commit/804aa79113ed628d4c4dc92ad5dd4aa347aabe5a))
* support for profiles event in redis ([#2497](https://github.com/rudderlabs/rudder-transformer/issues/2497)) ([f0c0a21](https://github.com/rudderlabs/rudder-transformer/commit/f0c0a211d167be2393c92db0a37dd517b1dbd1c4))


### Bug Fixes

* **braze:** enable merge behaviour to stitch user data ([#2508](https://github.com/rudderlabs/rudder-transformer/issues/2508)) ([8a2cf93](https://github.com/rudderlabs/rudder-transformer/commit/8a2cf93d9e83954edf1878390c254fb88a6c83c7))
* **gaoc:** custom variables issue ([#2545](https://github.com/rudderlabs/rudder-transformer/issues/2545)) ([3afee53](https://github.com/rudderlabs/rudder-transformer/commit/3afee53759e19765c4a284910cfd86e774dc0a24))
* **INT-512:** removed personal information from test cases ([#2517](https://github.com/rudderlabs/rudder-transformer/issues/2517)) ([9582e31](https://github.com/rudderlabs/rudder-transformer/commit/9582e31b9398f8d9bb01c431fd573fc54dbf7b3d))
* **iterable:** squadcast alert ([#2535](https://github.com/rudderlabs/rudder-transformer/issues/2535)) ([5a2194b](https://github.com/rudderlabs/rudder-transformer/commit/5a2194baa2c07d5b0fbe7bd7f4cfdec9117661ba))
* missing type for page and group calls ([#2512](https://github.com/rudderlabs/rudder-transformer/issues/2512)) ([bf08b9e](https://github.com/rudderlabs/rudder-transformer/commit/bf08b9e7177dbe7920e50e014484189a0c336b75))
* remove secure environment for datafile call ([#2544](https://github.com/rudderlabs/rudder-transformer/issues/2544)) ([b069e26](https://github.com/rudderlabs/rudder-transformer/commit/b069e262e9864a60611ee1b1e8e6c91dad76b7f4))
* fix: marketo bulk upload bugs and refactor ([#2414](https://github.com/rudderlabs/rudder-transformer/issues/2414)) ([9e3ace1](https://github.com/rudderlabs/rudder-transformer/pull/2546/commits/9e3ace17012f8fae3db35608367d98840037d1c0))

### [1.39.1](https://github.com/rudderlabs/rudder-transformer/compare/v1.39.0...v1.39.1) (2023-08-28)


Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "rudder-transformer",
"version": "1.39.1",
"version": "1.40.2",
"description": "",
"homepage": "https://github.com/rudderlabs/rudder-transformer#readme",
"bugs": {
Expand Down
10 changes: 5 additions & 5 deletions src/cdk/v2/bindings/default.js
Original file line number Diff line number Diff line change
Expand Up @@ -34,15 +34,15 @@ function assertConfig(val, message) {
}
}

function assertHttpResp(response, message) {
if (!isHttpStatusSuccess(response.status)) {
function assertHttpResp(processedResponse, message) {
if (!isHttpStatusSuccess(processedResponse.status)) {
throw new NetworkError(
message,
message.status,
processedResponse.status,
{
[tags.TAG_NAMES.ERROR_TYPE]: getDynamicErrorType(response.status),
[tags.TAG_NAMES.ERROR_TYPE]: getDynamicErrorType(processedResponse.status),
},
response,
processedResponse.response,
);
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,8 @@ steps:
const dataFileUrl = .destination.Config.dataFileUrl;
const rawResponse = await $.handleHttpRequest("get", dataFileUrl);
const processedResponse = rawResponse.processedResponse;
$.assertHttpResp(processedResponse, "Data File Lookup Failed");
processedResponse.response
$.assertHttpResp(processedResponse, "Data File Lookup Failed due to " + JSON.stringify(processedResponse.response));
processedResponse.response;
- name: validateDateFileForTrackPageAndScreen
condition: $.outputs.messageType in [ {{$.EventType.TRACK}}, {{$.EventType.PAGE}}, {{$.EventType.SCREEN}}]
Expand Down
4 changes: 2 additions & 2 deletions src/controllers/bulkUpload.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ export const fileUpload = async (ctx) => {
response = await destFileUploadHandler.processFileData(ctx.request.body);
} catch (error: any) {
response = {
statusCode: error.response ? error.response.status : 400,
statusCode: error?.response?.status || error?.status || 400,
error: error.message || 'Error occurred while processing payload.',
metadata: error.response ? error.response.metadata : null,
};
Expand Down Expand Up @@ -89,7 +89,7 @@ export const pollStatus = async (ctx) => {
response = await destFileUploadHandler.processPolling(ctx.request.body);
} catch (error: any) {
response = {
statusCode: error.response?.status || 400,
statusCode: error.response?.status || error?.status || 400,
error: error.message || 'Error occurred while processing payload.',
};
errNotificationClient.notify(error, 'Poll Status', {
Expand Down
51 changes: 50 additions & 1 deletion src/util/prometheus.js
Original file line number Diff line number Diff line change
Expand Up @@ -424,7 +424,7 @@ class Prometheus {
name: 'marketo_bulk_upload_polling',
help: 'marketo_bulk_upload_polling',
type: 'counter',
labelNames: ['status', 'state'],
labelNames: ['status', 'state', 'requestTime'],
},
{
name: 'marketo_fetch_token',
Expand Down Expand Up @@ -789,6 +789,18 @@ class Prometheus {
type: 'counter',
labelNames: ['http_status', 'destination_id'],
},
{
name: 'marketo_bulk_upload_upload_file_succJobs',
help: 'marketo_bulk_upload_upload_file_succJobs',
type: 'counter',
labelNames: [],
},
{
name: 'marketo_bulk_upload_upload_file_unsuccJobs',
help: 'marketo_bulk_upload_upload_file_unsuccJobs',
type: 'counter',
labelNames: [],
},
{
name: 'braze_lookup_time',
help: 'braze look-up time',
Expand Down Expand Up @@ -874,6 +886,43 @@ class Prometheus {
labelNames: ['processSessions'],
buckets: [10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 150, 200],
},
{
name: 'marketo_bulk_upload_create_header_time',
help: 'marketo_bulk_upload_create_header_time',
type: 'histogram',
labelNames: [],
},
{
name: 'marketo_bulk_upload_fetch_job_time',
help: 'marketo_bulk_upload_fetch_job_time',
type: 'histogram',
labelNames: [],
},
{
name: 'marketo_bulk_upload_fetch_job_create_response_time',
help: 'marketo_bulk_upload_fetch_job_create_response_time',
type: 'histogram',
labelNames: [],
},
{
name: 'marketo_bulk_upload_create_file_time',
help: 'marketo_bulk_upload_create_file_time',
type: 'histogram',
labelNames: [],
},
{
name: 'marketo_bulk_upload_upload_file_time',
help: 'marketo_bulk_upload_upload_file_time',
type: 'histogram',
labelNames: [],
},
{
name: 'marketo_bulk_upload_create_csvloop_time',
help: 'marketo_bulk_upload_create_csvloop_time',
type: 'histogram',
labelNames: [],
},

];

metrics.forEach((metric) => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -270,9 +270,9 @@ const responseHandler = (destinationResponse) => {
if (partialFailureError && partialFailureError.code !== 0) {
throw new NetworkError(
`[Google Ads Offline Conversions]:: partialFailureError - ${partialFailureError?.message}`,
status,
400,
{
[tags.TAG_NAMES.ERROR_TYPE]: getDynamicErrorType(status),
[tags.TAG_NAMES.ERROR_TYPE]: getDynamicErrorType(400),
},
partialFailureError,
);
Expand Down
37 changes: 37 additions & 0 deletions src/v0/destinations/intercom/networkHandler.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
const { proxyRequest, prepareProxyRequest } = require('../../../adapters/network');
const { processAxiosResponse } = require('../../../adapters/utils/networkUtils');

const { RetryableError } = require('../../util/errorTypes');

const errorResponseHandler = (destinationResponse, dest) => {
const { status } = destinationResponse;
if (status === 408) {
throw new RetryableError(
`[Intercom Response Handler] Request failed for destination ${dest} with status: ${status}`,
500,
destinationResponse,
);
}
};

const destResponseHandler = (destinationResponse, dest) => {
errorResponseHandler(destinationResponse, dest);
return {
destinationResponse: destinationResponse.response,
message: 'Request Processed Successfully',
status: destinationResponse.status,
};
};

class networkHandler {
constructor() {
this.responseHandler = destResponseHandler;
this.proxy = proxyRequest;
this.prepareProxy = prepareProxyRequest;
this.processAxiosResponse = processAxiosResponse;
}
}

module.exports = {
networkHandler,
};
36 changes: 36 additions & 0 deletions src/v0/destinations/marketo_bulk_upload/config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
const ABORTABLE_CODES = ['601', '603', '605', '609', '610'];
const RETRYABLE_CODES = ['713', '602', '604', '611'];
const THROTTLED_CODES = ['502', '606', '607', '608', '615'];

const MARKETO_FILE_SIZE = 10485760;
const MARKETO_FILE_PATH = `${__dirname}/uploadFile/marketo_bulkupload.csv`;

const FETCH_ACCESS_TOKEN = 'marketo_bulk_upload_access_token_fetching';

const POLL_ACTIVITY = 'marketo_bulk_upload_polling';
const POLL_STATUS_ERR_MSG = 'Could not poll status';

const UPLOAD_FILE = 'marketo_bulk_upload_upload_file';
const FILE_UPLOAD_ERR_MSG = 'Could not upload file';

const JOB_STATUS_ACTIVITY = 'marketo_bulk_upload_get_job_status';
const FETCH_FAILURE_JOB_STATUS_ERR_MSG = 'Could not fetch failure job status';
const FETCH_WARNING_JOB_STATUS_ERR_MSG = 'Could not fetch warning job status';
const ACCESS_TOKEN_FETCH_ERR_MSG = 'Error during fetching access token';

module.exports = {
ABORTABLE_CODES,
RETRYABLE_CODES,
THROTTLED_CODES,
MARKETO_FILE_SIZE,
POLL_ACTIVITY,
UPLOAD_FILE,
JOB_STATUS_ACTIVITY,
MARKETO_FILE_PATH,
FETCH_ACCESS_TOKEN,
POLL_STATUS_ERR_MSG,
FILE_UPLOAD_ERR_MSG,
FETCH_FAILURE_JOB_STATUS_ERR_MSG,
FETCH_WARNING_JOB_STATUS_ERR_MSG,
ACCESS_TOKEN_FETCH_ERR_MSG,
};
Loading

0 comments on commit db09237

Please sign in to comment.