Tests are failing

[sirkitree]

Tried out running the tests today (npm run test) and there are a lot of failures currently.

We should

1. evaluate which tests are still necessary
2. get them working
3. setup automated github action to run test for every pull request

Node.js v22.8.0
Test Suites: 16 failed, 16 total
Tests:       58 failed, 58 total
Snapshots:   0 total
Time:        14.737 s
Ran all test suites.

@lalalune could you advise here?
I think we can work on the tests but the first task of identifying which ones are necessary would be great if you could help with that more quickly than others.

[sirkitree]

from @lalalune

there are some, but they need a little love
i had tests in the old version: https://github.com/jointhealliance/bgent
https://github.com/JoinTheAlliance/bgent/tree/main/src/lib/__tests__
https://github.com/JoinTheAlliance/bgent/tree/main/src/lib/actions/__tests__
https://github.com/JoinTheAlliance/bgent/tree/main/src/lib/evaluators/__tests__
https://github.com/JoinTheAlliance/bgent/tree/main/src/lib/providers/__tests__
these are the old ones
i moved most of these into tests
if you wanna get the test automation working with a hello world test, i can assign this

[sirkitree]

After the above is merged, we need to quantify existing tests and decide which are still relevant.

I'll analyze the test files and describe their purposes.
Here are the test suites found in the codebase:

Basic Tests

• Located in core/src/test_resources/basic.test.ts
• Simple test suite that verifies:
  • Basic test functionality works
  • Environment variables are accessible
  • Confirms NODE_ENV is 'test' and TEST_DATABASE_CLIENT is 'sqlite'

Memory Tests

• Located in core/tests/memory.test.ts
• Tests memory-related functionality:
  • Search memories by embedding similarity
  • Validates memory ranking based on similarity scores
  • Tests memory lifecycle (creation, search, and removal)
  • Verifies embedding caching functionality

Browser Service Tests

• Located in core/tests/browser.test.ts
• Tests browser automation functionality:
  • Browser initialization and cleanup
  • Content fetching from various websites
  • CAPTCHA handling
  • Error handling for 404s and network issues
  • Ad blocking functionality
  • Handling of reCAPTCHA and hCAPTCHA

Token Provider Tests

• Located in core/tests/token.test.ts
• Tests token-related functionality:
  • Fetching token security data
  • Token holder information
  • Trade data retrieval
  • Formatted token reports

Goals Tests

• Located in core/tests/goals.test.ts and core/tests/goal.test.ts
• Tests goal management functionality:
  • Goal creation
  • Goal updates
  • Status management
  • Objective tracking

Runtime Tests

• Located in core/tests/runtime.test.ts
• Tests core runtime functionality:
  • Runtime instance creation
  • Memory lifecycle within runtime
  • State management
  • Basic runtime operations

Actions Tests

• Located in core/tests/actions.test.ts
• Tests action system:
  • Action loading
  • Action validation
  • Action handler execution
  • Test actions (TEST_ACTION and TEST_ACTION_FAIL)

Continue Action Tests

• Located in core/tests/continue.test.ts
• Tests the continue action:
  • Validation function responses
  • Message handling
  • Database interactions
  • Action state management

Messages Tests

• Located in core/tests/messages.test.ts
• Tests message formatting and handling:
  • Actor formatting
  • Message formatting
  • Facts formatting
  • Message content validation

Evaluation Tests

• Located in core/tests/evaluation.test.ts
• Tests evaluation system:
  • Evaluator format validation
  • Context loading
  • Example validation
  • Evaluation process execution

The test suite uses Jest as the testing framework and includes features like:

• Before/After hooks for setup and cleanup
• Environment variable loading from .dev.vars
• Timeout configurations for long-running tests
• Database-specific test configurations
• Test reporting functionality

Most tests include proper setup and teardown procedures to ensure a clean testing environment for each test case.

[sirkitree]

Now that we have a basic hello world type of test in place and running on ci, we can start lookin at these other ones. I'm going to split each one of these into their own task so they can be handled piece by piece and possibly spread out amongst the collective.

• Memory Tests #183
• Browser Service Tests #184
• Token Provider Tests #185
• Goals Tests #186
• Runtime Tests #187
• Actions Tests #188
• Continue Action Tests #189
• Messages Tests #190
• Evaluation Tests #191