Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: logger upgrade in services, dest, source #3228

Merged
merged 6 commits into from
Apr 9, 2024
Merged

feat: logger upgrade in services, dest, source #3228

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
6c34e8c
feat: logger upgrade in services, dest, source
aashishmalik Apr 1, 2024
cebad75
feat: logger upgrade in services, dest, source
aashishmalik Apr 1, 2024
0a4f059
fix: linting error
aashishmalik Apr 1, 2024
d80da63
fix: fixed import in destination/controller
aashishmalik Apr 1, 2024
152c374
chore: upgrade integrations-lib version
aashishmalik Apr 1, 2024
ed32176
feat: added child logging
aashishmalik Apr 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .eslintrc.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
"off",
{ "cases": { "camelCase": true, "pascalCase": true, "kebabCase": true } }
],
"import/no-import-module-exports": "off",
"unicorn/no-instanceof-array": "error",
"unicorn/no-static-only-class": "error",
"unicorn/consistent-destructuring": "error",
Expand Down
43 changes: 4 additions & 39 deletions package-lock.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@
"@koa/router": "^12.0.0",
"@ndhoule/extend": "^2.0.0",
"@pyroscope/nodejs": "^0.2.9",
"@rudderstack/integrations-lib": "^0.2.7",
"@rudderstack/integrations-lib": "^0.2.8",
"@rudderstack/workflow-engine": "^0.7.5",
"@shopify/jest-koa-mocks": "^5.1.1",
"ajv": "^8.12.0",
Expand Down
10 changes: 6 additions & 4 deletions src/cdk/v2/handler.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
import {
WorkflowEngine,
WorkflowEngineFactory,
TemplateType,
ExecutionBindings,
StepOutput,
TemplateType,
WorkflowEngine,
WorkflowEngineFactory,
} from '@rudderstack/workflow-engine';
import { FixMe } from '../../util/types';

import tags from '../../v0/util/tags';

import {
getErrorInfo,
getPlatformBindingsPaths,
getRootPathForDestination,
getWorkflowPath,
getPlatformBindingsPaths,
isCdkV2Destination,
} from './utils';

Expand Down Expand Up @@ -82,10 +82,12 @@ export async function processCdkV2Workflow(
destType: string,
parsedEvent: FixMe,
feature: string,
logger: FixMe,
requestMetadata: NonNullable<unknown> = {},
bindings: Record<string, FixMe> = {},
) {
try {
logger.debug(`Processing cdkV2 workflow`);
const workflowEngine = await getCachedWorkflowEngine(destType, feature, bindings);
return await executeWorkflow(workflowEngine, parsedEvent, requestMetadata);
} catch (error) {
Expand Down
38 changes: 10 additions & 28 deletions src/controllers/bulkUpload.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
/* eslint-disable global-require, import/no-dynamic-require, @typescript-eslint/no-unused-vars */
import { structuredLogger as logger } from '@rudderstack/integrations-lib';
import { client as errNotificationClient } from '../util/errorNotifier';
import logger from '../logger';
import {
getDestFileUploadHandler,
getJobStatusHandler,
getPollStatusHandler,
getDestFileUploadHandler,
} from '../util/fetchDestinationHandlers';
import { CatchErr, ContextBodySimple } from '../util/types';
// TODO: To be refactored and redisgned
Expand All @@ -31,10 +31,7 @@
};

export const fileUpload = async (ctx) => {
logger.debug(
'Native(Bulk-Upload): Request to transformer:: /fileUpload route',
JSON.stringify(ctx.request.body),
);
logger.debug('Native(Bulk-Upload): Request to transformer:: /fileUpload route', ctx.request.body);

Check warning on line 34 in src/controllers/bulkUpload.ts

View check run for this annotation

Codecov / codecov/patch

src/controllers/bulkUpload.ts#L34 

Added line #L34 was not covered by tests
const getReqMetadataFileUpload = () => {
try {
const reqBody = ctx.request.body;
Expand Down Expand Up @@ -69,18 +66,12 @@
});
}
ctx.body = response;
logger.debug(
'Native(Bulk-Upload): Response from transformer:: /fileUpload route',
JSON.stringify(ctx.body),
);
logger.debug('Native(Bulk-Upload): Response from transformer:: /fileUpload route', ctx.body);

Check warning on line 69 in src/controllers/bulkUpload.ts

View check run for this annotation

Codecov / codecov/patch

src/controllers/bulkUpload.ts#L69 

Added line #L69 was not covered by tests
return ctx.body;
};

export const pollStatus = async (ctx) => {
logger.debug(
'Native(Bulk-Upload): Request to transformer:: /pollStatus route',
JSON.stringify(ctx.request.body),
);
logger.debug('Native(Bulk-Upload): Request to transformer:: /pollStatus route', ctx.request.body);

Check warning on line 74 in src/controllers/bulkUpload.ts

View check run for this annotation

Codecov / codecov/patch

src/controllers/bulkUpload.ts#L74 

Added line #L74 was not covered by tests

const { destType }: ContextBodySimple = ctx.request.body;
const destFileUploadHandler = getPollStatusHandler('v0', destType.toLowerCase());
Expand All @@ -104,17 +95,14 @@
});
}
ctx.body = response;
logger.debug(
'Native(Bulk-Upload): Request from transformer:: /pollStatus route',
JSON.stringify(ctx.body),
);
logger.debug('Native(Bulk-Upload): Request from transformer:: /pollStatus route', ctx.body);

Check warning on line 98 in src/controllers/bulkUpload.ts

View check run for this annotation

Codecov / codecov/patch

src/controllers/bulkUpload.ts#L98 

Added line #L98 was not covered by tests
return ctx.body;
};

export const getWarnJobStatus = async (ctx) => {
logger.debug(
'Native(Bulk-Upload): Request to transformer:: /getWarningJobs route',
JSON.stringify(ctx.request.body),
ctx.request.body,
);

const { destType }: ContextBodySimple = ctx.request.body;
Expand All @@ -140,17 +128,14 @@
});
}
ctx.body = response;
logger.debug(
'Native(Bulk-Upload): Request from transformer:: /getWarningJobs route',
JSON.stringify(ctx.body),
);
logger.debug('Native(Bulk-Upload): Request from transformer:: /getWarningJobs route', ctx.body);

Check warning on line 131 in src/controllers/bulkUpload.ts

View check run for this annotation

Codecov / codecov/patch

src/controllers/bulkUpload.ts#L131 

Added line #L131 was not covered by tests
return ctx.body;
};

export const getFailedJobStatus = async (ctx) => {
logger.debug(
'Native(Bulk-Upload): Request to transformer:: /getFailedJobs route',
JSON.stringify(ctx.request.body),
ctx.request.body,
);

const { destType }: ContextBodySimple = ctx.request.body;
Expand All @@ -176,9 +161,6 @@
});
}
ctx.body = response;
logger.debug(
'Native(Bulk-Upload): Request from transformer:: /getFailedJobs route',
JSON.stringify(ctx.body),
);
logger.debug('Native(Bulk-Upload): Request from transformer:: /getFailedJobs route', ctx.body);

Check warning on line 164 in src/controllers/bulkUpload.ts

View check run for this annotation

Codecov / codecov/patch

src/controllers/bulkUpload.ts#L164 

Added line #L164 was not covered by tests
return ctx.body;
};
33 changes: 16 additions & 17 deletions src/controllers/delivery.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,28 +1,30 @@
/* eslint-disable prefer-destructuring */
/* eslint-disable sonarjs/no-duplicate-string */
import {
isDefinedAndNotNullAndNotEmpty,
structuredLogger as logger,
} from '@rudderstack/integrations-lib';
import { Context } from 'koa';
import { isDefinedAndNotNullAndNotEmpty } from '@rudderstack/integrations-lib';
import { ServiceSelector } from '../helpers/serviceSelector';
import { DeliveryTestService } from '../services/delivertTest/deliveryTest';
import { DestinationPostTransformationService } from '../services/destination/postTransformation';
import { MiscService } from '../services/misc';
import {
DeliveryV1Response,
DeliveryV0Response,
DeliveryV1Response,
ProcessorTransformationOutput,
ProxyV0Request,
ProxyV1Request,
} from '../types/index';
import { ServiceSelector } from '../helpers/serviceSelector';
import { DeliveryTestService } from '../services/delivertTest/deliveryTest';
import { ControllerUtility } from './util';
import logger from '../logger';
import { DestinationPostTransformationService } from '../services/destination/postTransformation';
import tags from '../v0/util/tags';
import { FixMe } from '../util/types';
import tags from '../v0/util/tags';
import { ControllerUtility } from './util';

const NON_DETERMINABLE = 'Non-determinable';

export class DeliveryController {
public static async deliverToDestination(ctx: Context) {
logger.debug('Native(Delivery):: Request to transformer::', JSON.stringify(ctx.request.body));
logger.info('Native(Delivery):: Request to transformer::', ctx.request.body);
let deliveryResponse: DeliveryV0Response;
const requestMetadata = MiscService.getRequestMetadata(ctx);
const deliveryRequest = ctx.request.body as ProxyV0Request;
Expand Down Expand Up @@ -52,12 +54,12 @@
ctx.body = { output: deliveryResponse };
ControllerUtility.deliveryPostProcess(ctx, deliveryResponse.status);

logger.debug('Native(Delivery):: Response from transformer::', JSON.stringify(ctx.body));
logger.debug('Native(Delivery):: Response from transformer::', ctx.body);
return ctx;
}

public static async deliverToDestinationV1(ctx: Context) {
logger.debug('Native(Delivery):: Request to transformer::', JSON.stringify(ctx.request.body));
logger.debug('Native(Delivery):: Request to transformer::', ctx.request.body);
let deliveryResponse: DeliveryV1Response;
const requestMetadata = MiscService.getRequestMetadata(ctx);
const deliveryRequest = ctx.request.body as ProxyV1Request;
Expand Down Expand Up @@ -91,15 +93,12 @@
ControllerUtility.deliveryPostProcess(ctx);
}

logger.debug('Native(Delivery):: Response from transformer::', JSON.stringify(ctx.body));
logger.debug('Native(Delivery):: Response from transformer::', ctx.body);
return ctx;
}

public static async testDestinationDelivery(ctx: Context) {
logger.debug(
'Native(Delivery-Test):: Request to transformer::',
JSON.stringify(ctx.request.body),
);
logger.debug('Native(Delivery-Test):: Request to transformer::', ctx.request.body);

Check warning on line 101 in src/controllers/delivery.ts

View check run for this annotation

Codecov / codecov/patch

src/controllers/delivery.ts#L101 

Added line #L101 was not covered by tests
const { destination }: { destination: string } = ctx.params;
const { version }: { version: string } = ctx.params;
const {
Expand All @@ -117,7 +116,7 @@
);
ctx.body = { output: response };
ControllerUtility.postProcess(ctx);
logger.debug('Native(Delivery-Test):: Response from transformer::', JSON.stringify(ctx.body));
logger.debug('Native(Delivery-Test):: Response from transformer::', ctx.body);

Check warning on line 119 in src/controllers/delivery.ts

View check run for this annotation

Codecov / codecov/patch

src/controllers/delivery.ts#L119 

Added line #L119 was not covered by tests
return ctx;
}
}
Loading
Loading