Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: tx list function name fuzzy search #2102

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions migrations/1728654359950_idx-contract-call-function-name-trgm.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
/** @param { import("node-pg-migrate").MigrationBuilder } pgm */
exports.up = pgm => {
pgm.sql(`
DO $$
BEGIN
IF EXISTS (
SELECT 1
FROM pg_available_extensions
WHERE name = 'pg_trgm'
) THEN
CREATE EXTENSION IF NOT EXISTS pg_trgm;

CREATE INDEX IF NOT EXISTS idx_contract_call_function_name_trgm
ON txs
USING gin (contract_call_function_name gin_trgm_ops);
END IF;
END
$$;
`);
};

/** @param { import("node-pg-migrate").MigrationBuilder } pgm */
exports.down = pgm => {
pgm.sql(`
DROP INDEX IF EXISTS idx_contract_call_function_name_trgm;
`);

pgm.sql('DROP EXTENSION IF EXISTS pg_trgm;');
};
12 changes: 12 additions & 0 deletions src/api/routes/tx.ts
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,12 @@ export const TxRoutes: FastifyPluginAsync<
examples: [1706745599],
})
),
search_term: Type.Optional(
Type.String({
description: 'Option to search for transactions by a search term',
examples: ['swap'],
})
),
Comment on lines +116 to +121
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this also be paired with something like a search_field_name, which current supports contract_call_function_name? Seems like there will be use cases were a search will not be interested in every single supported field.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea is to have the search_term as a broad search filter that looks into multiple fields. My plan so far is to incrementally add more fields to the same index and test performance, e.g.:

CREATE INDEX idx_search_term_trgm
ON public.txs
USING gin (
  contract_call_function_name gin_trgm_ops,
  contract_name gin_trgm_ops
  memo gin_trgm_ops
  ....
);

Open to suggestions ofc.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That seems fine to me. In the future we could always add another query param that lets users specify a given field(s) to search. And for now it can default to "all".

contract_id: Type.Optional(
Type.String({
description: 'Option to filter results by contract ID',
Expand Down Expand Up @@ -178,6 +184,11 @@ export const TxRoutes: FastifyPluginAsync<
contractId = req.query.contract_id;
}

let searchTerm: string | undefined;
if (typeof req.query.search_term === 'string') {
searchTerm = req.query.search_term;
}

const { results: txResults, total } = await fastify.db.getTxList({
offset,
limit,
Expand All @@ -188,6 +199,7 @@ export const TxRoutes: FastifyPluginAsync<
startTime: req.query.start_time,
endTime: req.query.end_time,
contractId,
searchTerm,
functionName: req.query.function_name,
nonce: req.query.nonce,
order: req.query.order,
Expand Down
24 changes: 24 additions & 0 deletions src/datastore/pg-store.ts
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,8 @@ import { parseBlockParam } from '../api/routes/v2/schemas';

export const MIGRATIONS_DIR = path.join(REPO_DIR, 'migrations');

const TRGM_SIMILARITY_THRESHOLD = 0.3;
rafaelcr marked this conversation as resolved.
Show resolved Hide resolved

/**
* This is the main interface between the API and the Postgres database. It contains all methods that
* query the DB in search for blockchain data to be returned via endpoints or WebSockets/Socket.IO.
Expand Down Expand Up @@ -1416,6 +1418,7 @@ export class PgStore extends BasePgStore {
startTime,
endTime,
contractId,
searchTerm,
functionName,
nonce,
order,
Expand All @@ -1430,6 +1433,7 @@ export class PgStore extends BasePgStore {
startTime?: number;
endTime?: number;
contractId?: string;
searchTerm?: string;
functionName?: string;
nonce?: number;
order?: 'desc' | 'asc';
Expand Down Expand Up @@ -1468,6 +1472,23 @@ export class PgStore extends BasePgStore {
const contractIdFilterSql = contractId
? sql`AND contract_call_contract_id = ${contractId}`
: sql``;

const searchTermFilterSql = searchTerm
? sql`
AND (
CASE
WHEN EXISTS (
SELECT 1
FROM pg_extension
WHERE extname = 'pg_trgm'
)
THEN similarity(contract_call_function_name, ${searchTerm}) > ${TRGM_SIMILARITY_THRESHOLD}
ELSE contract_call_function_name ILIKE '%' || ${searchTerm} || '%'
END
)
`
: sql``;

const contractFuncFilterSql = functionName
? sql`AND contract_call_function_name = ${functionName}`
: sql``;
Expand All @@ -1479,6 +1500,7 @@ export class PgStore extends BasePgStore {
!startTime &&
!endTime &&
!contractId &&
!searchTerm &&
!functionName &&
!nonce;

Expand All @@ -1497,6 +1519,7 @@ export class PgStore extends BasePgStore {
${startTimeFilterSql}
${endTimeFilterSql}
${contractIdFilterSql}
${searchTermFilterSql}
${contractFuncFilterSql}
${nonceFilterSql}
`;
Expand All @@ -1511,6 +1534,7 @@ export class PgStore extends BasePgStore {
${startTimeFilterSql}
${endTimeFilterSql}
${contractIdFilterSql}
${searchTermFilterSql}
${contractFuncFilterSql}
${nonceFilterSql}
${orderBySql}
Expand Down
70 changes: 70 additions & 0 deletions tests/api/tx.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2326,6 +2326,76 @@ describe('tx tests', () => {
);
});

test('tx list - filter by searchTerm using trigram', async () => {
const transferTokenTx = {
tx_id: '0x1111',
contract_call_function_name: 'transferToken',
};

const stakeTokenTx = {
tx_id: '0x2222',
contract_call_function_name: 'stakeToken',
};

const burnTokenTx = {
tx_id: '0x3333',
contract_call_function_name: 'burnToken',
};

const block1 = new TestBlockBuilder({ block_height: 1, index_block_hash: '0x01' })
.addTx(transferTokenTx)
.build();
await db.update(block1);

const block2 = new TestBlockBuilder({
block_height: 2,
index_block_hash: '0x02',
parent_block_hash: block1.block.block_hash,
parent_index_block_hash: block1.block.index_block_hash,
})
.addTx(stakeTokenTx)
.addTx(burnTokenTx)
.build();
await db.update(block2);

const searchTerm = 'transfer';

const txsReq = await supertest(api.server).get(`/extended/v1/tx?search_term=${searchTerm}`);
expect(txsReq.status).toBe(200);
expect(txsReq.body).toEqual(
expect.objectContaining({
results: [
expect.objectContaining({
tx_id: transferTokenTx.tx_id,
}),
],
})
);

const broadSearchTerm = 'token';

const txsReqBroad = await supertest(api.server).get(
`/extended/v1/tx?search_term=${broadSearchTerm}`
);
expect(txsReqBroad.status).toBe(200);

expect(txsReqBroad.body).toEqual(
expect.objectContaining({
results: [
expect.objectContaining({
tx_id: burnTokenTx.tx_id,
}),
expect.objectContaining({
tx_id: stakeTokenTx.tx_id,
}),
expect.objectContaining({
tx_id: transferTokenTx.tx_id,
}),
],
})
);
});

test('tx list - filter by contract id/name', async () => {
const testContractAddr = 'ST27W5M8BRKA7C5MZE2R1S1F4XTPHFWFRNHA9M04Y.hello-world';
const testContractFnName = 'test-contract-fn';
Expand Down
Loading