Test should be structured based on adapters and a core

[amv]

What do you think about the following goals:

1. The core unit tests should be run with a mocked adapter (I think this is at least mostly, if not fully, the case currently).
2. Adapters should ship individually with their own adapter-specific unit tests (apart from maybe the fs-adapter, which would be shipped with the core).
3. The core should ship with system integration tests that are adapter agnostic and they should be easy to run with any adapter the user chooses to test.

[amv]

Also it would be cool to have the unit and e2e tests structured by component so that if you change something in the ejector, you could easily run both with mocha test/unit/ejector/ test/e2e/ejector/

[jarnoharno]

There should probably also be test suites for adapter / controller APIs to test any custom adapter or controller against the spec.

[amv]

Is there a spec for the controller?

I had the impression that we will build a default controller that operates in a specified way (and provide tests for that specific way), but I don't really see what could be seen to necessarily belong in a "generic controller spec".

I think the controller would be the place that regulates for example uniqueness and concurrency constraints, which would in some cases mean that the task is simply not run at all, and not ejected either (for example because some other controller has started the same job before). It would be really hard to build a general test that would test both the default retry controller and also this kind of a controller with the same tests.

But adapter agnostic integration tests (which was my point number 3) that can easily be run with an adapter of the users choice would definitely be a nice addition.

To enable this kind of generic adapter tests, it might be a good idea to add some kind of an interface to the adapter to destroy the whole state of the adapter backend, which would then be run between tests. Do you have any idea of other good locations to implement this "state clearing" than the adapter interface itself?

[jarnoharno]

I was thinking along the lines of: "Create a controller. Call it with a task. Advance fake timers. If the controller calls worker, make sure it has the correct payload. Respond. Advance fake timers. If the controller ejects task, make sure it contains the correct id." Add to this the possible future requirement that the controller needs to eject with the provided ejector function.

Oops, I misunderstood the third point :p

The state clearing could be another function in the adapter API, resetTasks() maybe?