CAT in Tapscript (BIP-347)

[0xBEEFCAF3]

CAT in Tapscript

This PR provides the necessary code to enable the opcode OP_CAT in Tapscript as specified in BIP-347: OP_CAT in Tapscript and BIN-2024-0001,

Important: This PR does not include miner activation functionality. This means that merging this PR into bitcoin-core will not make OP_CAT functional in Bitcoin.
If this PR is merged it is not a signal of community consensus around activating CAT nor be read as a portent about the activation process or timeline.
The PR is not a stand-in for consensus around such decisions.

This PR includes:

• An implementation of the Tapscript opcode OP_CAT along with the flags SCRIPT_VERIFY_OP_CAT and SCRIPT_VERIFY_DISCOURAGE_OP_CAT.
• Integration and unit tests which ensure that our CAT implementation works as expected. This includes ensuring that CAT never uses more that 520 bytes of stack memory. A full description of what these tests cover is given below.
• An improvement to src/test/script_tests.cpp JSON script format enabling the rapid creation of new Tapscript unit tests.

Activation on Bitcoin-inquisition (signet)

Bitcoin-inquisition PR 37 which contained our implementation of OP_CAT was merged into bitcoin-inquisition (signet) on Apr 25 2024, released as bitcoin-inquisition release 25.2 on Apr 26 2024. and activated in bitcoin-inquisition Apr 30 2024. The bitcoin-inquisition PR was reviewed in the PR review club (transcript of discussion). Since Apr 30 2024 there have been many OP_CAT transactions created and spent on signet.

The code merged into bitcoin-inquisition differs from this PR, as we have removed the consensus logic which was used to activate it on signet.

OP_CAT Tapscript Implementation

We implement OP_CAT as a new Tapscript op code by redefining the opcode OP_SUCCESS126 (126 in decimal and 0x7e in hexadecimal). This is the same opcode value used by the original OP_CAT.

When evaluated, the OP_CAT instruction:

1. Pops the top two values off the stack,
2. concatenates the popped values together in stack order,
3. and then pushes the concatenated value on the top of the stack.

OP_CAT fails if there are fewer than two values on the stack or if a concatenated value would have a combined size greater than the maximum script element size of 520 bytes.

See BIP 347 for a deeper description.

Errors thrown

If evaluated OP_CAT can throw the following errors:

• If at time of evaluation the stack has fewer than two elements we throw the error: SCRIPT_ERR_INVALID_STACK_OPERATION
• If at time of evaluation the top two stack elements have a combined size greater than MAX_SCRIPT_ELEMENT_SIZE (520 bytes) we throw the error: SCRIPT_ERR_PUSH_SIZE.

Script verification flags

While this PR does not contain any miner signaling and activation logic and can not activate OP_CAT, it does contain two flags which future activation logic could set to control activation of OP_CAT.

• SCRIPT_VERIFY_OP_CAT IF a bitcoin node has this set to true, then it treat OP_CAT enabled for Tapscript. That is, OP_SUCCESS126 will be redefining to OP_CAT in Tapscript. If this was set to true at the consensus level this would cause a soft fork.

SCRIPT_VERIFY_DISCOURAGE_OP_CAT When set to true, a node receiving any Tapscript transaction containing the opcode OP_CAT or OP_SUCCESS126 will reject the transaction throwing the error SCRIPT_ERR_DISCOURAGE_OP_CAT but not banning the node which relayed the transaction. This prevents nodes from relaying transactions with OP_CAT. This is equivalent to the behavior of SCRIPT_ERR_DISCOURAGE_OP_CAT = true when SCRIPT_VERIFY_OP_CAT = false as OP_CAT is an OP_SUCCESS (OP_SUCCESS126).

This is how these two flags are intended to be used to ensure a smooth soft fork.

Stage	SCRIPT_VERIFY OP_CAT	SCRIPT_VERIFY DISCOURAGE_OP_CAT	Status
1.Default	False	True	This does not represent any change in behavior of the bitcoin-core node.
2.Upgrading	True	True	OP_CAT is activating.
3.Upgraded	True	False	It is clear that OP_CAT has been activated and the network has been upgraded.

The flag SCRIPT_ERR_DISCOURAGE_OP_CAT provides a window of time for the network to fully activate, before nodes will relay or accept transactions containing OP_CAT in their mem pools.

Tests

This PR contains a suite of script tests to ensure that OP_CAT functions as expected. In the JSON script tests (script_tests.json), we test:

• That if OP_CAT is not activated that there are no changes to bitcoin consensus.
• That regardless of the activation or non-activation of OP_CAT in Tapscript, pre-Tapscript scripts, i.e. Bitcoin scripts, have no changes in behavior.
• That OP_CAT when evaluated throws the expected error when there are less than two elements on the stack.
• That OP_CAT when evaluated in a variety of circumstances and edge cases, successfully concatenates elements of the stack. This includes:
  • multiple calls to OP_CAT in a row,
  • evaluating OP_CAT inside of a IF conditional,
  • evaluating OP_CAT on zero size stack elements, random and large stack elements,
  • evaluating OP_CAT on values being moved to and from the alt stack,
  • and checking that when evaluated OP_CAT concatenates the elements in the expected order.

All of these tests are designed to cover the happy path of OP_CAT, the various errors which OP_CAT can throw and all the corner cases between those two outcomes.

Additionally we include three tests outside of the JSON script tests.

• cat_simple and cat_empty_stack are designed to test OP_CAT outside of the JSON serialization regime. Ensure that we catch bugs that we might miss in the JSON script tests due to a bug introduced at the JSON serialization layer.
• cat_dup_test enumerates all stack element sizes from 1 to 522 bytes and then enumerates up to 10 repetitions of OP\_DUP OP\_CAT. It then tests if the stack element would exceed 520 bytes and if so did OP_CAT throw the error SCRIPT_ERR_PUSH_SIZE. This allows us to be certain that OP_CAT will not introduce any OP\_DUP OP\_CAT memory exhaustion attacks.

Better Tapscript tests in JSON script tests

While writing these JSON script tests (script_tests.json) we ran into the following problem. The JSON script tests are simple and easy to write for pre-Tapscript scripts, but adding or changing a Tapscript test requires substantial work per test.
Consider the following pre-tapscript test:

["'aa' 'bb'", "CAT 0x4c 0x02 0xaabb EQUAL", "P2SH,STRICTENC", "DISABLED_OPCODE", "CAT disabled"]

whereas a Tapscript test for the same script (annotated with comments for better readability) would look like:

[
    [
        "aa",
        "bb",
        "7e4c02aabb87", // output script
        "c0d6889cb081036e0faefa3a35157ad71086b123b2b144b649798b494c300a961d", // control block
        0.00000001
    ],
    "",
    "0x51 0x20 0x15048ed3a65748549c27b671936987093cf73a4c9cb18522a74fb9553060ca99", // Tapscript output
    "P2SH,WITNESS,TAPROOT",
    "OK",
    "TAPSCRIPT CATs aa and bb together and checks if EQUAL to aabb"
]

Computing the Tapscript output, such as 0x51 0x20 0x15048ed3a65748549c27b671936987093cf73a4c9cb18522a74fb9553060ca99, requires writing custom code and running it for each test. The same is true for the Tapscript control block, such as c0d6889cb081036e0faefa3a35157ad71086b123b2b144b649798b494c300a961d. If a test is changed or updated new outputs and control blocks must be computed. The complexity of doing this is likely the reason that no one has added any Tapscript tests to JSON script tests until this PR.

In this PR we address this issue by adding the following improvements to JSON script tests:

• Adding simple macros ("#SCRIPT# and #CONTROLBLOCK#) that allow the script test parser to automatically generate and inject a valid Tapscript output and control block to be computed automatically from the JSON script.
• Allowing Tapscript scripts to use the human readable strings like pre-script scripts by marking the location of the script in the witness stack using #SCRIPT#. This transforms the unreadable script 7e4c02aabb87 into #SCRIPT# CAT 0x4c 0x02 0xaabb EQUAL.

This results in the following JSON script test which is far easier to write and easier to read.

[
    [
        "aa",
        "bb",
        "#SCRIPT# CAT",
        "#CONTROLBLOCK#",
        0.00000001
    ],
    "",
    "0x51 0x20 #TAPROOTOUTPUT#",
    "P2SH,WITNESS,TAPROOT,OP_CAT",
    "OK",
    "TAPSCRIPT Test of OP_CAT flag by calling CAT on two elements. TAPSCRIPT_OP_CAT flag is set so CAT is executed."
],

[DrahtBot]

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Code Coverage & Benchmarks

For details see: https://corecheck.dev/bitcoin/bitcoin/pulls/29247.

Reviews

See the guideline for information on the review process.
A summary of reviews will appear here.

Conflicts

Reviewers, this pull request conflicts with the following ones:

• #29270 (Implement OP_CHECKSIGFROMSTACK(VERIFY) by reardencode)
• #29269 (Add OP_INTERNALKEY for Tapscript by reardencode)

If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first.

[DrahtBot]

🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the
documentation.

Possibly this is due to a silent merge conflict (the changes in this pull request being
incompatible with the current code in the target branch). If so, make sure to rebase on the latest
commit of the target branch.

Leave a comment here, if you need help tracking down a confusing failure.

_{Debug: https://github.com/bitcoin/bitcoin/runs/20473822722}

[0xBEEFCAF3]

Drafting until I get the inquisition PR approved and I can get the builds passing.

[moonsettler]

Some of us have been playing with the idea of neutered CAT: instead of the MAX_SCRIPT_ELEMENT_SIZE (520 bytes) the maximum output size would be 80 bytes.

Rationale:
It explicitly disables open ended second order effects by removing the ability to assemble a SIGHASH or CTV template on the stack for more detailed introspection than intended. If such introspection behavior is desired in the future it can be explicitly enabled by specialized and more efficient opcodes.

Combined with CSFS it can be used as a signed datacarrier, for which the 'standard' limit is 80 bytes, it also has to be smaller than 84 bytes which is required for building CTV templates on the stack, and larger than 64/65/72 bytes which are respectively needed for:

• Merkle inclusion: 2x 20/32 byte hashes
• LN-symmetry (LNhance): 2x 32 byte hashes
• Separate sig: 64/72 bytes (0-conf bonds, staking contracts)

Could be a livable compromise between the conservatives that want to preserve certain characteristics of bitcoin and the prometheans who want to give the developers more practical and useful tools to build with.

[bigspider]

Rationale: It explicitly disables open ended second order effects by removing the ability to assemble a SIGHASH or CTV template on the stack for more detailed introspection than intended. If such introspection behavior is desired in the future it can be explicitly enabled by specialized and more efficient opcodes.

Script is already expressive enough to (awkwardly, expensively) compute arbitrary SHA256 hashes on the stack, except that the result would be broken in smaller pieces of at most 4 bytes. With CAT, those can be concatenated to get a single 32-byte result.

Therefore, neutering CATs does not achieve the desired result of preventing the CHECKSIG tricks, unless you limit the length of the result to less than 32 bytes - which would also neuter most of the utility of the opcode.

[DrahtBot]

🚧 At least one of the CI tasks failed.
_{Debug: https://github.com/bitcoin/bitcoin/runs/29308305455}

Hints

Make sure to run all tests locally, according to the documentation.

The failure may happen due to a number of reasons, for example:

• Possibly due to a silent merge conflict (the changes in this pull request being
  incompatible with the current code in the target branch). If so, make sure to rebase on the latest
  commit of the target branch.

• A sanitizer issue, which can only be found by compiling with the sanitizer and running the
  affected test.

• An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

[rot13maxi]

SCRIPT_VERIFY_DISCOURAGE_OP_CAT is acting as a CAT-specific SCRIPT_VERIFY_DISCOURAGE_OP_SUCCESS that can be turned off while still not relaying transasctions with other OP_SUCCESS opcodes. Since this is the first time we're carving out an OP_SUCCESS and doing this, it would be great to have a test that exercises the combinations of SCRIPT_VERIFY_DISCOURAGE_OP_CAT and SCRIPT_VERIFY_OP_CAT

[0xBEEFCAF3]

Great suggestion! Added some additional unit tests to script_tests.json testing different combinations of scripts with and without (SCRIPT_VERIFY_DISCOURAGE_OP_CAT, SCRIPT_VERIFY_OP_CAT and SCRIPT_VERIFY_DISCOURAGE_OP_SUCCESS)

[ajtowns]

Added to the inquisition PR in bitcoin-inquisition#68

[DrahtBot]

🚧 At least one of the CI tasks failed.
_{Debug: https://github.com/bitcoin/bitcoin/runs/30231832503}

Hints

Make sure to run all tests locally, according to the documentation.

The failure may happen due to a number of reasons, for example:

• Possibly due to a silent merge conflict (the changes in this pull request being
  incompatible with the current code in the target branch). If so, make sure to rebase on the latest
  commit of the target branch.

• A sanitizer issue, which can only be found by compiling with the sanitizer and running the
  affected test.

• An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

[achow101]

Converting to draft while waiting for consensus on this.

[EthanHeilman]

@achow101 What does draft vs non-draft mean in the bitcoin-core github? My understanding was moving from draft to non-draft means code is ready for review. Similar to moving out of WIP.

Is draft/non-draft is being used to signal community consensus or bitcoin-core consensus? If so, I agree this should be a draft.

[achow101]

In general, draft means that a PR is not in a state where it could be merged. It indicates to reviewers that they may not want to do any in depth code review of the PR yet.

In the context of this PR, there does not appear to be consensus for the concept of OP_CAT in TapScript yet, so this PR cannot be merged even if the code is okay. Hence marking it as a draft.

Note that this is a result of a Kill/Shill/Merge session that occurred at the most recent CoreDev.

[EthanHeilman]

@achow101 Thanks, that's helpful for understanding the context. If consensus forms around OP_CAT should I or @0xBEEFCAF3 mark this as no longer as draft? I can't speak for @0xBEEFCAF3 but I would feel slightly uncomfortable making that decision as it would open me up to criticism that I was attempting to manufacture consensus, even if there was consensus. Then again if that is the expectation I suppose we'd have to do that and take the licks. What is the process here?

[achow101]

If consensus forms around OP_CAT should I or @0xBEEFCAF3 mark this as no longer as draft?

Yes. A maintainer may also do so as well.

[DrahtBot]

🚧 At least one of the CI tasks failed.
_{Debug: https://github.com/bitcoin/bitcoin/runs/32250870897}

Hints

Try to run the tests locally, according to the documentation. However, a CI failure may still
happen due to a number of reasons, for example:

• Possibly due to a silent merge conflict (the changes in this pull request being
  incompatible with the current code in the target branch). If so, make sure to rebase on the latest
  commit of the target branch.

• A sanitizer issue, which can only be found by compiling with the sanitizer and running the
  affected test.

• An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.