fix: indexes to optimize principal-based etag db lookups

[zone117x]

Possible fix for #2147

In version v8.1.0, several endpoints (in particular the balance endpoints) were updated to use new etag caching handling. This change was actually an optimization to reduce db load, however, it performs a sql query that is very CPU intensive for postgres deployments without very large memory cache configurations.

This PR adds composite indexes which should be less CPU intensive than the bitmap scans required by the previously used indexes.

I was able to reproduce the slow query perf on a small-ish vm, and verify that this PR significantly improves the query:

// before PR
Planning Time: 4.330 ms
Execution Time: 774.455 ms

// after PR
Planning Time: 0.851 ms
Execution Time: 0.399 ms

Using this sample query:

    WITH activity AS (
      (
        SELECT '0x' || encode(tx_id, 'hex') AS tx_id
        FROM principal_stx_txs
        WHERE principal = 'SP2ADT4TAV92SPBNP3WE9GPJ12F6CKGWMEJ1H1BB9' AND canonical = true AND microblock_canonical = true
        ORDER BY block_height DESC, microblock_sequence DESC, tx_index DESC
        LIMIT 1
      )
      UNION
      (
        SELECT '0x' || encode(tx_id, 'hex') AS tx_id
        FROM (
          (
            SELECT tx_id, block_height, microblock_sequence, tx_index, event_index
            FROM ft_events
            WHERE sender = 'SP2ADT4TAV92SPBNP3WE9GPJ12F6CKGWMEJ1H1BB9'
              AND canonical = true
              AND microblock_canonical = true
            ORDER BY block_height DESC, microblock_sequence DESC, tx_index DESC, event_index DESC
            LIMIT 1
          )
          UNION ALL
          (
            SELECT tx_id, block_height, microblock_sequence, tx_index, event_index
            FROM ft_events
            WHERE recipient = 'SP2ADT4TAV92SPBNP3WE9GPJ12F6CKGWMEJ1H1BB9'
              AND canonical = true
              AND microblock_canonical = true
            ORDER BY block_height DESC, microblock_sequence DESC, tx_index DESC, event_index DESC
            LIMIT 1
          )
        ) AS combined
        ORDER BY block_height DESC, microblock_sequence DESC, tx_index DESC, event_index DESC
        LIMIT 1
      )
      UNION
      (
        SELECT '0x' || encode(tx_id, 'hex') AS tx_id
        FROM nft_events
        WHERE (sender = 'SP2ADT4TAV92SPBNP3WE9GPJ12F6CKGWMEJ1H1BB9' OR recipient = 'SP2ADT4TAV92SPBNP3WE9GPJ12F6CKGWMEJ1H1BB9')
          AND canonical = true
          AND microblock_canonical = true
        ORDER BY block_height DESC, microblock_sequence DESC, tx_index DESC, event_index DESC
        LIMIT 1
      )
     UNION
      (
        SELECT 'mempool-' || '0x' || encode(tx_id, 'hex') AS tx_id
        FROM mempool_txs
        WHERE pruned = false AND
          (sender_address = 'SP2ADT4TAV92SPBNP3WE9GPJ12F6CKGWMEJ1H1BB9'
          OR sponsor_address = 'SP2ADT4TAV92SPBNP3WE9GPJ12F6CKGWMEJ1H1BB9'
          OR token_transfer_recipient_address = 'SP2ADT4TAV92SPBNP3WE9GPJ12F6CKGWMEJ1H1BB9')
        ORDER BY receipt_time DESC, sender_address DESC, nonce DESC
        LIMIT 1
      )
    )
    SELECT tx_id FROM activity WHERE tx_id IS NOT NULL;

[github-actions]

Vercel deployment URL: https://stacks-blockchain-2iy3g7ror-hirosystems.vercel.app 🚀

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

📢 Thoughts on this report? Let us know!

[blockstack-devops]

🎉 This PR is included in version 8.2.1-beta.1 🎉

The release is available on:

• npm package (@beta dist-tag)
• GitHub release

Your semantic-release bot 📦🚀