Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: added credentials to javascript transformations #3408

Merged
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
732fbf4
chore: added credentials to javascript transformations
kanishkkatara May 23, 2024
e6a3dbb
chore: added tests
kanishkkatara May 23, 2024
a7e4ba4
chore: change in credentials structure
kanishkkatara May 24, 2024
5111ce9
chore: changes in test transform API
kanishkkatara May 30, 2024
c280d81
fix: credential param
kanishkkatara May 30, 2024
3137964
fix: test
kanishkkatara May 31, 2024
7eb537c
fix: credential error handling
kanishkkatara Jun 3, 2024
a57da89
chore: added tests
kanishkkatara Jun 3, 2024
1fd068b
chore: added errors in credentials
kanishkkatara Jun 3, 2024
8808508
chore: added credentials at event level for python transformations
kanishkkatara Jun 4, 2024
ce2d62d
Merge branch 'develop' into feature/dat-1207-javascript-transformatio…
kanishkkatara Jun 11, 2024
546a937
chore: added check for multiple arguments
kanishkkatara Jun 12, 2024
bb94c81
chore: change in lodash import
kanishkkatara Jun 12, 2024
a80d403
chore: changes in transformation error handling
kanishkkatara Jun 17, 2024
67e9277
fix: fixes in javascript transformation
kanishkkatara Jun 18, 2024
e7a59f9
chore: added tests for transform batch and code version 0
kanishkkatara Jun 18, 2024
46cd341
fix: pr comments
kanishkkatara Jun 18, 2024
819f508
fix: undefined tests
kanishkkatara Jun 18, 2024
fa3619c
Merge branch 'develop' into feature/dat-1207-javascript-transformatio…
kanishkkatara Jun 18, 2024
e6c5098
fix: credentials payload fix
kanishkkatara Jun 18, 2024
3db810c
fix: tests names, credentialsMap refactor
kanishkkatara Jun 19, 2024
7b0ea80
refactor: change in credentials function validation code
kanishkkatara Jun 19, 2024
2a17453
Merge branch 'develop' into feature/dat-1207-javascript-transformatio…
kanishkkatara Jun 19, 2024
d655fd1
Merge branch 'develop' into feature/dat-1207-javascript-transformatio…
kanishkkatara Jun 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions src/types/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -137,12 +137,19 @@ type UserTransformationLibrary = {
VersionID: string;
};

type Credential = {
Key: string;
kanishkkatara marked this conversation as resolved.
Show resolved Hide resolved
Value: string;
isPublic: boolean;
};

type ProcessorTransformationRequest = {
request?: object;
message: object;
metadata: Metadata;
destination: Destination;
libraries?: UserTransformationLibrary[];
credentials?: Credential[];
};

type RouterTransformationRequestData = {
Expand Down
2 changes: 2 additions & 0 deletions src/util/customTransformer-v1.js
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ function calculateMsFromIvmTime(value) {
async function userTransformHandlerV1(
events,
userTransformation,
credentials,
libraryVersionIds,
testMode = false,
) {
Expand All @@ -62,6 +63,7 @@ async function userTransformHandlerV1(
userTransformation.code,
libraryVersionIds,
userTransformation.versionId,
credentials,
userTransformation.secrets || {},
testMode,
);
Expand Down
2 changes: 2 additions & 0 deletions src/util/customTransformer.js
Original file line number Diff line number Diff line change
Expand Up @@ -285,12 +285,14 @@ async function userTransformHandler(
events.forEach((ev) => {
eventsMetadata[ev.message.messageId] = ev.metadata;
});
const credentials = events[0].credentials;
kanishkkatara marked this conversation as resolved.
Show resolved Hide resolved

let userTransformedEvents = [];
let result;
if (res.codeVersion && res.codeVersion === '1') {
result = await UserTransformHandlerFactory(res).runUserTransfrom(
events,
credentials,
testMode,
libraryVersionIDs,
);
Expand Down
10 changes: 8 additions & 2 deletions src/util/customTransformerFactory.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,20 @@ const UserTransformHandlerFactory = (userTransformation) => {
}
},

runUserTransfrom: async (events, testMode, libraryVersionIds) => {
runUserTransfrom: async (events, credentials, testMode, libraryVersionIds) => {
switch (userTransformation.language) {
case 'pythonfaas':
case 'python':
return runOpenFaasUserTransform(events, userTransformation, libraryVersionIds, testMode);

default:
return userTransformHandlerV1(events, userTransformation, libraryVersionIds, testMode);
return userTransformHandlerV1(
events,
userTransformation,
credentials,
libraryVersionIds,
testMode,
);
}
},
};
Expand Down
19 changes: 16 additions & 3 deletions src/util/ivmFactory.js
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
return module;
}

async function createIvm(code, libraryVersionIds, versionId, secrets, testMode) {
async function createIvm(code, libraryVersionIds, versionId, credentials, secrets, testMode) {
const createIvmStartTime = new Date();
const logs = [];
const libraries = await Promise.all(
Expand Down Expand Up @@ -243,6 +243,11 @@
}),
);

await jail.set('_credentials', function (...args) {
if (args.length == 0 || !credentials || !credentials[args[0]]) return 'ERROR';
kanishkkatara marked this conversation as resolved.
Show resolved Hide resolved
kanishkkatara marked this conversation as resolved.
Show resolved Hide resolved
return credentials[args[0]];

Check warning on line 248 in src/util/ivmFactory.js

View check run for this annotation

Codecov / codecov/patch

src/util/ivmFactory.js#L248

Added line #L248 was not covered by tests
});

await jail.set('_rsSecrets', function (...args) {
if (args.length == 0 || !secrets || !secrets[args[0]]) return 'ERROR';
return secrets[args[0]];
Expand Down Expand Up @@ -321,6 +326,14 @@
]);
};

let credentials = _credentials;
delete _credentials;
global.credentials = function(...args) {
return credentials([
...args.map(arg => new ivm.ExternalCopy(arg).copyInto())
]);
};

return new ivm.Reference(function forwardMainPromise(
fnRef,
resolve,
Expand Down Expand Up @@ -411,10 +424,10 @@
return evaluateModule(isolate, context, code);
}

async function getFactory(code, libraryVersionIds, versionId, secrets, testMode) {
async function getFactory(code, libraryVersionIds, versionId, credentials, secrets, testMode) {
const factory = {
create: async () => {
return createIvm(code, libraryVersionIds, versionId, secrets, testMode);
return createIvm(code, libraryVersionIds, versionId, credentials, secrets, testMode);
},
destroy: async (client) => {
client.fnRef.release();
Expand Down
Loading