feat!: fx and interscheme implementation #1058

kleyow · 2024-07-04T18:39:46Z

Changes:

Currency Conversion (FX)
- Database changes (knex migrations) for FX
- Updated prepare and position handlers to support FX kafka messages
- Updated timeout handler to timeout dependent transfers incase of FX
- Implemented batch mode for the timeout and abort position messages along with FX actions
Interscheme
- Proxy cache implementation using redis
- Parameterize switch id
- Extend admin api to support proxy participants
- Added new transfer internal state (RESERVED_FORWARDED) to support interscheme
- Adjusted position handler implementation for proxied participants
- Adjusted validations in prepare handler to support proxied participants
Performance optimisations

* feat(3574): updated prepare handler for FX * feat(3574): added fxTransfer related tables; updated validator for FX * feat(3574): added fxTransfer related tables; updated validator for FX * feat(3574): comments/todos * feat(3574): added fx roleTypes; added content.context * feat(3574): added cyril; updated unit-tests

* chore: updated central service shared lib * feat: added some changes for fx flow * feat: added changes for position prepare handler * chore: refactor cyril functions * feat: position-commit working * feat: upgraded central shared * chore(snapshot): 17.4.0-snapshot.0 * chore(snapshot): 17.4.0-snapshot.1 * chore(snapshot): 17.4.0-snapshot.2

* feat: add FX quotes endpointType and kafka topics * chore: upgrade cs-shared * chore: fix audit

…feat/fx-impl

… into feat/fx-impl

* fix: cluster performance testing issues (#996) * test: fix disconnect errors (#998) * chore(release): 17.6.1 [skip ci] * chore: fix current tests * boolean * chore: add endpoints to test data * fix endpoint import * chore: improve validator coverage * chore: move prepare tests into file to match src structure --------- Co-authored-by: Kalin Krustev <[email protected]> Co-authored-by: mojaloopci <[email protected]>

* chore: more coverage * coverage

* fix: call commit and rollback * test: fix coverage * fix: call commit and rollback * fix: remove unnecessary commit and rollback * fix: remove unnecessary commit and rollback

* fix: int tests * fix: int tests * fix: audit and lint fix * fix: spelling * chore: skipped an int test * chore(snapshot): 17.8.0-snapshot.17 * chore(snapshot): 17.8.0-snapshot.18 * chore(snapshot): 17.8.0-snapshot.19 * chore(snapshot): 17.8.0-snapshot.20 * fix: add duplication logic and test for fxTransfers * conversionState * alter int test for message key 0 * alter test --------- Co-authored-by: Kevin Leyow <[email protected]>

… transfer/facade (#1099) * feat(csi-318): added externalParticipants table * refactor(csi-631): added calculateProxyObligation fn * refactor(csi-631): added forwardPrepare fn * refactor(csi-631): added forwardPrepare fn * refactor(csi-631): improved logging in transfer facade * chore(csi-632): added migrations to create externalParticipant table * chore(csi-632): added migration to add externalParticipantId FK to fxTransferParticipant * chore(csi-632): added migration to add externalParticipantId FK to fxTransferParticipant * feat(csi-633): added externalParticipant model; added JSDocs; updated transfer/facade * feat(csi-633): added externalParticipantId field to fxTransferParticipant table * feat(csi-633): added externalParticipantId field to fxTransferParticipant table

… updated transfer/facade" (#1100) Revert "feat(csi-633): added externalParticipant model; added JSDocs; updated…" This reverts commit eb54f67.

* fix: int tests * fix: int tests * fix: audit and lint fix * fix: spelling * chore: skipped an int test * chore(snapshot): 17.8.0-snapshot.17 * chore(snapshot): 17.8.0-snapshot.18 * chore(snapshot): 17.8.0-snapshot.19 * chore(snapshot): 17.8.0-snapshot.20 * fix: get fx transfers * fix: refactor * fix: fx fulfilment * fix: unit tests * fix: tests

fix: from argument in kafka notification for abort

Revert "fix: fix abort callback (#1106)" This reverts commit b6e9e2b.

* fix: check participant.isActive in prepare * chore(snapshot): 17.8.0-snapshot.16 * chore(snapshot): 17.8.0-snapshot.17 * fix: check position account is active in prepare * chore(snapshot): 17.8.0-snapshot.18 * test: temporarily disable coverage for proxy * chore(snapshot): 17.8.0-snapshot.19 * chore(snapshot): 17.8.0-snapshot.20 * ci: temporarily disable int tests for snapshots * chore(snapshot): 17.8.0-snapshot.21 * fix: fix typos * refactor: reactor getFSPProxy * chore(snapshot): 17.8.0-snapshot.22 * doc: update comment * chore(snapshot): 17.8.0-snapshot.23 * fix(csi-603): fix getTransferParticipant query join * ci: re-enable integration tests for snapshots * chore(snapshot): 17.8.0-snapshot.24 * chore(snapshot): 17.8.0-snapshot.25 * fix: fix query * chore(snapshot): 17.8.0-snapshot.26 * refactor: refactor * refactor: refactor * fix(csi-610): fix hub responding with RESERVED instead of COMMITED for v1.1 reserved fulfil * chore(snapshot): 17.8.0-snapshot.27 --------- Co-authored-by: Vijay <[email protected]>

…ss (#1105) * feat(csi/643): add fx-notify publishing on payer init fxTranfer success * loop * deps * tests * list

* fix: from argument in kafka notification for abort * fix: position changes * fix: to number * fix: position change in timeout * fix: related fxtransfer check * fix: unit tests * fix: timeout * chore: deps * fix fx-abort tests * fix fx-timeout tests * chore: added a comment * fix more tests * fix: invalid fulfilment * fix: unit test * chore(snapshot): 17.8.0-snapshot.28 * chore(snapshot): 17.8.0-snapshot.29 * fix: lint * chore(snapshot): 17.8.0-snapshot.30 --------- Co-authored-by: Kevin Leyow <[email protected]>

* feat(csi-318): added externalParticipants table * refactor(csi-631): added calculateProxyObligation fn (#1093) * refactor(csi-631): added calculateProxyObligation fn * refactor(csi-631): added forwardPrepare fn * refactor(csi-631): added forwardPrepare fn * refactor(csi-631): improved logging in transfer facade * chore(csi-632): added migrations to create externalParticipant table (#1094) * refactor(csi-631): added calculateProxyObligation fn * refactor(csi-631): added forwardPrepare fn * refactor(csi-631): added forwardPrepare fn * refactor(csi-631): improved logging in transfer facade * chore(csi-632): added migrations to create externalParticipant table * chore(csi-632): added migration to add externalParticipantId FK to fxTransferParticipant * chore(csi-632): added migration to add externalParticipantId FK to fxTransferParticipant * feat(csi-633): added externalParticipant model; updated transfer/facade; added JSDocs; (#1101) * refactor(csi-631): added calculateProxyObligation fn * refactor(csi-631): added forwardPrepare fn * refactor(csi-631): added forwardPrepare fn * refactor(csi-631): improved logging in transfer facade * chore(csi-632): added migrations to create externalParticipant table * chore(csi-632): added migration to add externalParticipantId FK to fxTransferParticipant * chore(csi-632): added migration to add externalParticipantId FK to fxTransferParticipant * feat(csi-633): added externalParticipant model; added JSDocs; updated transfer/facade * feat(csi-633): added externalParticipantId field to fxTransferParticipant table * feat(csi-633): added externalParticipantId field to fxTransferParticipant table * feat(csi-633): updated from feat/fx-impl * feat(csi-650): updated transferTimeout handler to take into account externalParticipant (#1107) * feat(csi-650): updated transferTimeout handler to take into account externalParticipant * feat(csi-650): fixed ep1.externalParticipantId field * feat(csi-650): used leftJoin for externalParticipant table * feat(csi-650): added externalPayeeName as source to timeout handler * feat(csi-650): updated fxTimeout logic to take into account externalParticipant info * feat(csi-650): code cleaning up * feat(csi-650): code cleaning up * feat(csi-651): updated fxAbort handling to use externalParticipant info (#1111) * feat(csi-650): updated transferTimeout handler to take into account externalParticipant * feat(csi-650): fixed ep1.externalParticipantId field * feat(csi-650): used leftJoin for externalParticipant table * feat(csi-650): added externalPayeeName as source to timeout handler * feat(csi-650): updated fxTimeout logic to take into account externalParticipant info * feat(csi-650): code cleaning up * feat(csi-650): code cleaning up * feat(csi-651): updated fxAbort handling to use externalParticipant info * feat(csi-651): updated fxValidation handling * feat(csi-651): fixed one leftJoin clause * feat(csi-651): updated getExternalParticipantIdByNameOrCreate * feat(csi-651): updated getExternalParticipantIdByNameOrCreate * feat(csi-651): added externalParticipantCached model * feat(csi-651): fixed prepare-internals tests * feat(csi-651): added more tests * feat(csi-651): reverted changes back to feat/fx-impl * feat(csi-651): reverted unneeded changes back to feat/fx-impl version * feat(csi-651): excluded some files from test coverage check * chore(snapshot): 17.8.0-snapshot.32 * chore(snapshot): 17.8.0-snapshot.33

src/api/participants/routes.js

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

sonarcloud · 2024-10-21T10:24:57Z

Quality Gate failed

Failed conditions
26.8% Duplication on New Code (required ≤ 3%)
B Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

shashi165 · 2024-11-11T22:45:37Z

src/shared/fspiopErrorFactory.js

+    return Factory.createFSPIOPError(
+      Enums.FSPIOPErrorCodes.TRANSFER_EXPIRED,
+      ERROR_MESSAGES.fxTransferExpired,
+      cause = null, replyTo = ''


Suggested change

cause = null, replyTo = ''

cause, replyTo

.circleci/config.yml

elnyry-sam-k · 2024-11-13T15:37:14Z

documentation/fx-implementation/README.md

@@ -0,0 +1,48 @@
+# FX Implementation
+
+## Proof of Concept (PoC) Implementation for Payer-Side Currency Conversion (Happy Path Only)


Does this still hold?

shashi165 · 2024-11-13T22:48:55Z

package.json

    "@hapi/inert": "7.1.0",
    "@hapi/joi": "17.1.1",
    "@hapi/vision": "7.0.3",
-    "@hapi/catbox-memory": "6.0.2",
-    "@mojaloop/database-lib": "11.0.5",
    "@mojaloop/central-services-error-handling": "13.0.1",
    "@mojaloop/central-services-health": "15.0.0",


is it missing dependency @mojaloop/sdk-standard-components required by @mojaloop/central-services-error-handling

Why do you think it is missing a dependency? Build is successful https://app.circleci.com/pipelines/github/mojaloop/central-ledger/2809/workflows/b9859b2a-253d-483b-80b8-3e395393bd11

shashi165 · 2024-11-13T23:12:20Z

Need to fix the sonarcloud issues and run the tests. integration tests are failing locally

vijayg10 and others added 30 commits November 8, 2023 18:34

feat: added db migration scripts for fx

0b62904

feat(3574): added FX endpoints to seeds (#989)

07f2888

chore(snapshot): 17.4.0-snapshot.3

2dc6a1f

chore(snapshot): 17.4.0-snapshot.4

fbc3252

chore(snapshot): 17.4.0-snapshot.5

e9d0ba2

chore(snapshot): 17.4.0-snapshot.6

853c90a

chore(snapshot): 17.4.0-snapshot.7

745802f

chore(snapshot): 17.4.0-snapshot.8

55d56f5

fix: positions

7f5c80d

fix: disable unit tests for snapshot

98f7fdc

chore(snapshot): 17.4.0-snapshot.9

1fb4904

chore: added docs

70d0732

chore: updated doc

29136ea

fix: normal fulfil

a883a33

fix: normal flow

8be45cd

fix: updated fx diagram

cbfe0dd

chore: dep update

c3e8af8

chore(snapshot): 17.4.0-snapshot.10

a0e5ae6

feat(mojaloop/#3689): fx quotes changes (#995)

77e7ff1

* feat: add FX quotes endpointType and kafka topics * chore: upgrade cs-shared * chore: fix audit

chore(snapshot): 17.4.0-snapshot.12

224a3d3

ci: disable unit tests and test coverage runs for snapshots

c2b64c7

chore(snapshot): 17.4.0-snapshot.13

8155ba0

Merge branch main of https://github.com/mojaloop/central-ledger into …

1dc7032

…feat/fx-impl

Merge branch feat/fx-impl of https://github.com/mojaloop/central-ledger…

831da00

… into feat/fx-impl

fix: manual changes from upstream commits

0979ae5

chore(snapshot): 17.7.0-snapshot.0

6df1384

test(mojaloop/#3819): harden fx prepare flow (#1002)

b4ad846

* chore: more coverage * coverage

vijayg10 changed the title ~~feat: add fx and interscheme functionality~~ feat: fx and interscheme implementation Sep 4, 2024

kalinkrustev and others added 17 commits September 4, 2024 18:30

fix: remove misleading commit and rollback (#1089)

3cae6f2

* fix: call commit and rollback * test: fix coverage * fix: call commit and rollback * fix: remove unnecessary commit and rollback * fix: remove unnecessary commit and rollback

Revert "feat(csi-633): added externalParticipant model; added JSDocs;…

8151468

… updated transfer/facade" (#1100) Revert "feat(csi-633): added externalParticipant model; added JSDocs; updated…" This reverts commit eb54f67.

fix: fx transfer extension (#1102)

bcc6fa9

fix: retify int tests (#1104)

3182803

fix: fix abort callback (#1106)

b6e9e2b

fix: from argument in kafka notification for abort

Revert "fix: fix abort callback" (#1109)

99d94df

Revert "fix: fix abort callback (#1106)" This reverts commit b6e9e2b.

chore(snapshot): 17.8.0-snapshot.22

c1a1e17

feat(csi/643): add fx-notify publishing on payer init fxTranfer succe…

60ea20b

…ss (#1105) * feat(csi/643): add fx-notify publishing on payer init fxTranfer success * loop * deps * tests * list

chore(snapshot): 17.8.0-snapshot.28

4bf39b7

feat(csi-634): added mock-knex lib to mock mysql in unit-tests (#1113)

afa1828

feat: add ULID support (#1114)

cc1d482

github-advanced-security bot found potential problems Sep 25, 2024

View reviewed changes

src/api/participants/routes.js Fixed Show fixed Hide fixed

src/api/participants/routes.js Fixed Show fixed Hide fixed

kleyow changed the title ~~feat: fx and interscheme implementation~~ feat!: fx and interscheme implementation Sep 26, 2024

kalinkrustev and others added 4 commits September 30, 2024 11:12

Fix code scanning alert no. 9: Missing regular expression anchor

56055c3

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

fix: uuid/ulid regex

5661049

test: start using mojaloop/build orb (#1115)

0e4fc9e

fix: sonar security hot spots

03acfb7

shashi165 reviewed Nov 11, 2024

View reviewed changes

elnyry-sam-k reviewed Nov 13, 2024

View reviewed changes

.circleci/config.yml Show resolved Hide resolved

elnyry-sam-k reviewed Nov 13, 2024

View reviewed changes

shashi165 reviewed Nov 13, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat!: fx and interscheme implementation #1058

feat!: fx and interscheme implementation #1058

kleyow commented Jul 4, 2024 •

edited by vijayg10

Loading

sonarcloud bot commented Oct 21, 2024 •

edited

Loading

shashi165 Nov 11, 2024

elnyry-sam-k Nov 13, 2024

shashi165 Nov 13, 2024

kalinkrustev Nov 25, 2024

shashi165 commented Nov 13, 2024

		@@ -0,0 +1,48 @@
		# FX Implementation

		## Proof of Concept (PoC) Implementation for Payer-Side Currency Conversion (Happy Path Only)

feat!: fx and interscheme implementation #1058

Are you sure you want to change the base?

feat!: fx and interscheme implementation #1058

Conversation

kleyow commented Jul 4, 2024 • edited by vijayg10 Loading

sonarcloud bot commented Oct 21, 2024 • edited Loading

Quality Gate failed

shashi165 Nov 11, 2024

Choose a reason for hiding this comment

elnyry-sam-k Nov 13, 2024

Choose a reason for hiding this comment

shashi165 Nov 13, 2024

Choose a reason for hiding this comment

kalinkrustev Nov 25, 2024

Choose a reason for hiding this comment

shashi165 commented Nov 13, 2024

kleyow commented Jul 4, 2024 •

edited by vijayg10

Loading

sonarcloud bot commented Oct 21, 2024 •

edited

Loading