Build QueryService to handle complex queries

[gregbrowndev]

Currently, the repositories are used for both querying and write-only domain-logic. As we've added queries such as Find-Jobs, which include complex logic such as filtering, pagination, and eventually sorting, this creates an extra burden to test, as the query logic must be duplicated in the fake in-memory adapter. Moreover, not only do we need to integration and e2e test the live query logic thoroughly, but we only need to write unit tests for the core service, which is fairly pointless.

To solve this problem, leverage CQRS to separate the read-path from the write-path. The repositories should only be concerned with data access functions required for the domain logic, e.g. get and perhaps a few special case find by condition functions when performing actions in bulk. The domain logic likely never needs to paginate or sort the results from the DB. To solve this, introduce a QueryService to handle user queries with filtering, pagination, sorting, auth, and permission-checking.

The QueryService can either share the same database as the other applications or go full CQRS with a separate read-only database model. However, this will require us to handle eventual consistency in the architecture.

To handle eventual consistency, we can either go full distributed data and have the read model in a same database and kept in sync by another transaction using either events or orchestration, or keep the read model in the same database and include the read-model updates in the same write-transaction.

Another consideration is where to put the QueryService. The hexagonal architecture of the server emphasises the unit, integration, and e2e test strategy ideal for the write-path. We can simplify the architecture a lot for the read-path and have direct access to the DB in the QueryService. However, this poses challenges for bootstrapping the application in the unit test environment. Even if we had a modulith architecture where the QueryService is implemented in a separate module to the domain logic and thus can use an appropriate architectural style, it would still be a problem. Maybe the only solution for this is to take it to its final endpoint of having a fully distributed query microservice.

We need to decide what solution is best for this application.

Requirements:

• Decide whether to adopt eventual-consistency or keep the read-path strongly consistent with the write-path.
• Decide whether to build a Query microservice.
• Implement the Query service and refactor tests.