make a functional testing package

[turadg]

What is the Problem Being Solved?

The tests in /a3p-integration tests the functionality of a running chain. It happens to be synthetic but the essential aspect is it performs operations through RPC.

There are two key problems.

Most of the utilities are in another repo
https://github.com/Agoric/agoric-3-proposals/tree/main/packages/synthetic-chain#design has the tools to support testing with a synthetic chain, but the synthetic part really only has to do with how the chain gets started and managed. Interacting with the chain is a concern of agoric-sdk. We see this over and over writing tests in a3p-integration that some improvement is devised and it has to make a cumbersome round-trip to the a3p repo to get back into the test through the @agoric/synthetic-chain library. (example)

Many of the calls go through agoric-cli

Functions like follow and run are implemented in JS by the agoric package but it doesn't export the functionality as JS modules. It only provides a CLI, which has required that something like follow which could be a function call has a complex CLI presentation within the tests,

For example:

      agoric
        .follow(
          '-lF',
          `:published.wallet.${longLivingBidderAddr}`,
          '-o',
          'text',
        )
        .then(res => marshaller.fromCapData(JSON.parse(res))),

Description of the Design

Make a new package, perhaps @agoric/functional-testing.

Move much of the utilities in @agoric/synthetic-chain into it.

Make JS interfaces to the functionality in agoric-cli. E.g.
Be able to follow with something like,

const published = makeFollower(agoricConfig, marshaller);

const stream = published.follow(`wallet.${longLivingBidderAddr}`;

related issues

• publish a JS library that provides many of the functions of agoric-cli #10168

That one is about abstractions that can be used in agoric-cli and this is about packaging things for A3P testing. There will likely be PRs that make progress on both at once but their "done" criteria are different.

• package up utilities for dapps #9109

Abstractions that can also be used in dapps.

• testing API to extend ava functional tests to end-to-end context #8963

That one is about testing abstractions that can be re-used across functional tests and bootstrap tests. We might want to design the solution to this issue in a way that would satisfy 8963 but that would be a bigger lift. We could deliver this issue and come back to 8963 later.

Security Considerations

Scaling Considerations

Test Plan

Everything in https://github.com/Agoric/agoric-3-proposals/blob/main/packages/synthetic-chain/src/lib/cliHelper.ts should be available without synthetic-chain or spawning

Upgrade Considerations

[dckc]

related issues / conversations...

• testing API to extend ava functional tests to end-to-end context #8963
• publish a JS library that provides many of the functions of agoric-cli #10168

[turadg]

@dckc thanks for linking those. I thought I had made something like 10168 but couldn't find it.

I'll update the description to clarify how these each relate.