Several refactoring suggestions for the `@solana/[email protected]` implementation #10

steveluscher · 2024-08-08T18:11:07Z

No description provided.

steveluscher

Hey @WilfredAlmeida! Some refactoring suggestions for the new ping thing client.

steveluscher · 2024-08-08T18:11:24Z

package.json

    "axios": "^1.7.3",
    "bs58": "^6.0.0",
    "dotenv": "^16.4.5",
    "js-yaml": "^4.1.0",
    "undici": "^6.19.5"
  },
+  "overrides": {


This presumes that you're using npm.

steveluscher · 2024-08-08T18:12:14Z

utils/rpc.mjs

+export const rpc = createSolanaRpc(
+    process.env.RPC_ENDPOINT,
+);
+export const rpcSubscriptions = createSolanaRpcSubscriptions_UNSTABLE(
+    process.env.WS_ENDPOINT,
+);


In general, it's good to share these as widely as possible so that they can maximally coalesce requests and subscriptions together.

steveluscher · 2024-08-08T18:13:38Z

utils/blockhash.mjs

+  for await (const notification of recentBlockhashesNotifications) {
+    latestBlockhash = getLatestBlockhashFromNotification(
+      notification,
+      differenceBetweenSlotHeightAndBlockHeight
+    );
+    yield latestBlockhash;
+  }


You essentially want a read-through cache that has the freshest blockhash at all times. Watching the recent blockhashes sysvar is a fun way to do this, so long as you don't mind the sort of gross computation of lastValidBlockHeight.

steveluscher · 2024-08-08T18:14:19Z

ping-thing-client.mjs

@@ -100,15 +91,13 @@ async function pingThing() {
    }
    try {
      try {
+        const latestBlockhash = await getLatestBlockhash();


Pull the latest blockhash out of the read-through cache here and jam it into the transaction message.

steveluscher · 2024-08-08T18:14:54Z

ping-thing-client.mjs

+            [slotSent] = await Promise.all([
+              getNextSlot(),
+              mSendTransaction(transactionSignedWithFeePayer, {
+                commitment: "confirmed",
+                maxRetries: 0n,
+                skipPreflight: true,
+              }),
+            ]);


Fetch the sent slot as near to the actual send as possible. I don't know if you want the slot on the leading edge (ie. you don't mind the actual send being one slot later) or the trailing edge (ie. you don't mind it being earlier). I fetch them in parallel here, and the getNextSlot() function really does wait for the next slot (it's not a read-through cache like the blockhash iterator).

Know what, I updated this PR because I think I see what's going on here. You want the sentSlot to live outside this loop and be computed:

Once.

Before even trying to send.

steveluscher · 2024-08-08T18:19:13Z

ping-thing-client.mjs

@@ -68,7 +67,7 @@ async function pingThing() {
  const signer = await createSignerFromKeyPair(USER_KEYPAIR);
  const BASE_TRANSACTION_MESSAGE = pipe(
    createTransactionMessage({ version: 0 }),
-    (tx) => setTransactionMessageFeePayer(signer.address, tx),
+    (tx) => setTransactionMessageFeePayerSigner(signer, tx),


The signers API works by jamming TransactionSigner objects into your message…

steveluscher · 2024-08-08T18:19:24Z

ping-thing-client.mjs

+        const transactionSignedWithFeePayer =
+          await signTransactionMessageWithSigners(transactionMessage);


…which now knows how to self-sign!

steveluscher · 2024-08-08T18:19:52Z

ping-thing-client.mjs

+const mSendTransaction = sendTransactionWithoutConfirmingFactory({
+  rpc,
+});
+
+const getRecentSignatureConfirmationPromise =
+  createRecentSignatureConfirmationPromiseFactory({
+    rpc,
+    rpcSubscriptions,
+  });


In general, we want to create senders/confirmers once and then use them for the lifetime of the rpc/rpcSubscriptions.

steveluscher · 2024-08-08T18:20:40Z

ping-thing-client.mjs

+const mSendAndConfirmTransaction = sendAndConfirmTransactionFactory({
  rpc,
+  rpcSubscriptions,


This factory does all of the things you were doing with the pieces.

steveluscher · 2024-08-08T18:21:08Z

ping-thing-client.mjs

+                mSendAndConfirmTransaction(transactionSignedWithFeePayer, {
+                  commitment: "confirmed",
+                }),
+                timeout(TX_RETRY_INTERVAL * txSendAttempts),


You can still achieve the timeout behavior by racing the all-up send-and-confirmer with a timeout promise.

steveluscher · 2024-08-08T19:18:24Z

ping-thing-client.mjs

+                  maxRetries: 0n,
+                  skipPreflight: true,


Accidentally dropped these.

steveluscher · 2024-08-25T05:23:40Z

Converting this to draft. Some parts of this refactoring are bad, actually. I'll update this later.

steveluscher · 2024-10-06T21:17:56Z

utils/blockhash.mjs

+  const latestUnusedBlockhash = latestBlockhashes.find(
+    ({ blockhash }) => !usedBlockhashes.has(blockhash),
+  );


This handles the extremely rare case that you're:

Sending an identical transaction

Asked for a latest blockhash quickly enough to get the same one as last time (eg. maybe the blockhash fetcher has fallen behind)

Wouldn't the usedBlockhashes list increase the memory consumption over time given the ever running nature of the ping thing?

An identical transaction should be fine since, and, if transactions need not be identical then every time a blockhash is read from the watcher, it can have a value like lastServedBlockhash and if both the previously served blockhash and current one are same then handle it by waiting or polling for a new blockhash

Wouldn't the usedBlockhashes list increase the memory consumption over time given the ever running nature of the ping thing?

Nope! The code above copies over only the used hashes that are still valid. The maximum size of usedBlockhashes is the number of valid blockhashes.

steveluscher · 2024-10-06T21:20:39Z

ping-thing-client.mjs

+        slotSent = await getNextSlot();
+        const sendRetryInterval = setInterval(() => {
+          sendAbortController.abort();
+          console.log(`Tx not confirmed after ${TX_RETRY_INTERVAL * txSendAttempts++}ms, resending`);
+          sendTransaction();
+        }, TX_RETRY_INTERVAL);
+        txStart = Date.now();
+        sendTransaction();
+        await mConfirmRecentSignature({
+          abortSignal: AbortSignal.any([]),
+          commitment: COMMITMENT_LEVEL,
+          signature,
+        });


OK, so I get what your original implementation was angling at now @WilfredAlmeida. You want to:

Get the slot.

Fire off the transaction

Start one confirmer

Retry the transaction send if you haven't gotten confirmation in x milliseconds.

Yes that's correct

… `Signer#publicKey`

…nused_ one.

…loop on the side

… of the leading edge (firstShredReceived)

steveluscher commented Aug 8, 2024

View reviewed changes

steveluscher marked this pull request as ready for review August 8, 2024 18:21

steveluscher force-pushed the web-3-new-suggestions branch from 949da54 to a727a51 Compare August 8, 2024 18:25

steveluscher commented Aug 8, 2024

View reviewed changes

ping-thing-client.mjs Outdated

Comment on lines 124 to 125

maxRetries: 0n,

skipPreflight: true,

Copy link

Author

steveluscher Aug 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accidentally dropped these.

steveluscher force-pushed the web-3-new-suggestions branch 2 times, most recently from bcfbe15 to 625773e Compare August 8, 2024 21:16

steveluscher marked this pull request as draft August 25, 2024 05:23

steveluscher force-pushed the web-3-new-suggestions branch 3 times, most recently from 3893ad5 to ef28626 Compare October 6, 2024 05:57

steveluscher marked this pull request as ready for review October 6, 2024 05:58

steveluscher force-pushed the web-3-new-suggestions branch from ef28626 to 5047a05 Compare October 6, 2024 06:09

steveluscher commented Oct 6, 2024

View reviewed changes

steveluscher force-pushed the web-3-new-suggestions branch from ba8d65f to d4fed25 Compare October 6, 2024 21:19

steveluscher commented Oct 6, 2024

View reviewed changes

steveluscher force-pushed the web-3-new-suggestions branch 5 times, most recently from b11a72b to f9da5a7 Compare October 12, 2024 06:06

steveluscher force-pushed the web-3-new-suggestions branch from f9da5a7 to 56509bd Compare October 15, 2024 17:45

steveluscher added 8 commits October 23, 2024 14:25

Force all package resolutions to suitable canary release

660d590

Make the RPC global to the application

646d1e6

Repair missing import of isSolanaError

bdac4b9

Implement a blockhash fetcher as an AsyncIterator

5a7cb98

Implement a slot fetcher as an AsyncIterator

5c80154

Use TransactionSigner#address instead of computing the address from…

c3c9aed

… `Signer#publicKey`

Only create the base transaction once

13e34ce

Use signers API

a94ee1c

steveluscher added 10 commits October 23, 2024 14:26

Hoist factory creation up to global scope

83fe4c0

Replace transaction send-and-confirm with default implementation

4e627bd

Sprinking dotenv configs where they need to go

bd1e906

Move the stopwatch starter _after_ the slot fetcher

e425521

JSON.stringify no longer fatals when it encounters a bigint

c5e69b3

Keep the entire list of blockhashes in memory, and vend the latest _u…

9d72a30

…nused_ one.

Start exactly _one_ transaction confirmer, and a separate send retry …

d00892b

…loop on the side

Fetch the next slot by watching the trailing edge (completed) instead…

f98870b

… of the leading edge (firstShredReceived)

Prevent uncaught errors in the send-retry loop from bubbling up uncaught

e8d53b4

Fail the transaction loop when the blockhash expires

4acf40b

steveluscher force-pushed the web-3-new-suggestions branch from 2087094 to 4acf40b Compare October 23, 2024 14:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Several refactoring suggestions for the `@solana/[email protected]` implementation #10

Several refactoring suggestions for the `@solana/[email protected]` implementation #10

steveluscher commented Aug 8, 2024

steveluscher left a comment

steveluscher Aug 8, 2024

steveluscher Aug 8, 2024

steveluscher Aug 8, 2024

steveluscher Aug 8, 2024

steveluscher Aug 8, 2024

steveluscher Aug 8, 2024

steveluscher Aug 8, 2024

steveluscher Aug 8, 2024

steveluscher Aug 8, 2024

steveluscher Aug 8, 2024

steveluscher Aug 8, 2024

steveluscher Aug 8, 2024

steveluscher commented Aug 25, 2024

steveluscher Oct 6, 2024

WilfredAlmeida Oct 7, 2024

WilfredAlmeida Oct 7, 2024

steveluscher Oct 7, 2024

steveluscher Oct 6, 2024

WilfredAlmeida Oct 7, 2024

		const transactionSignedWithFeePayer =
		await signTransactionMessageWithSigners(transactionMessage);

Several refactoring suggestions for the @solana/[email protected] implementation #10

Are you sure you want to change the base?

Several refactoring suggestions for the @solana/[email protected] implementation #10

Conversation

steveluscher commented Aug 8, 2024

steveluscher left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

steveluscher commented Aug 25, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Several refactoring suggestions for the `@solana/[email protected]` implementation #10

Several refactoring suggestions for the `@solana/[email protected]` implementation #10