Test concurrency (interaction of two processes with a database (f.x. an event sourced content repository))

[mhsdesign]

I know para tests is probably not the right tool for that but what i thought i want to use ;)

so i got one test with two methods:

one method is the one user doing some thing while the other method reflects what the other user does.
Also there is a setup method (creating db schema and dummy data) and a file locking so the setup is only run once throughout processes.

This works well for one test at a time. And looks like this:

WorkspaceWritingDuringRebaseTest: 52816: ------ process started ------
WorkspaceWritingDuringRebaseTest: 52815: ------ process started ------
WorkspaceWritingDuringRebaseTest: 52816: setup started
WorkspaceWritingDuringRebaseTest: 52815: waiting for setup
WorkspaceWritingDuringRebaseTest: 52816: setup finished
WorkspaceWritingDuringRebaseTest: 52815: wait for setup finished
# both processes start the actual test after one of the processes handles the setup

# that is what i need to test, this concurrency case will throw an exception and i assert that things are correctly handled and not broken afterwards
WorkspaceWritingDuringRebaseTest: 52815: rebase started
WorkspaceWritingDuringRebaseTest: 52816: write during rebase
WorkspaceWritingDuringRebaseTest: 52815: rebase finished

Now im the procedure of adding more of these tests, and i have to ensure that only one test file is run in parallel at a time before the next one starts.

If not all parallel cases of each test (in -f mode) are finished, paratest will go directly to the next test case and start that, while the previous test case is not finished in all processes.

I thought i could leverage @groups for that - to say only methods of a group can be parallel at once - but found out that they dont work that way :)

Now i will go ahead to write a for loop to boot paratest n times, but i could probably stop using paratest at this point :D

(just for reference this is one of the tests im talking about https://github.com/neos/neos-development-collection/blob/85b6a4dfe30306775ad142da355d5af9158d5edf/Neos.ContentRepository.BehavioralTests/Tests/Parallel/WorkspaceWritingDuringRebase/WorkspaceWritingDuringRebase.php)

[Slamdunk]

I honestly haven't understand what you are doing nor what your goal is 😕

[Jean85]

If I understood this correctly, you have interdependent tests, so a test is dependent on the result of another one; this is an antipattern, especially with a parallel executor like Paratest. Each test should be independent and it's execution should provide the same outcome, with no regard to the order of execution of other tests.

[mhsdesign]

Thanks for your comments so far, maybe i have not expressed myself the best.

But yes i want to test that my system handles concurrent write operations correctly and that a user attempting to do X while another user started already doing Y will get a concurrency exception.

Each of those interactions is encapsulated in a single test file. With two methods each (like one is called whileAWorkspaceIsBeingRebased and the second one is thenConcurrentCommandsLeadToAnException), and im using functional mode and spawn two processes so they are run in parallel.

As said earlier ill probably not use paratest to run all the tests but only file per file to guarantee that the test runs dont conflict with another

for f in Neos.ContentRepository.BehavioralTests/Tests/Parallel/**/*Test.php;
  do ../../bin/paratest --debug -v --functional --processes 2 --colors --stop-on-failure -c ../../Build/BuildEssentials/PhpUnit/FunctionalTests.xml $f
done

This is very likely a x-y problem and im aware that paratest is NOT build for that, still i thought it would be cool to experiment with that and find better solutions.

[Slamdunk]

Each of those interactions is encapsulated in a single test file. With two methods each (like one is called whileAWorkspaceIsBeingRebased and the second one is thenConcurrentCommandsLeadToAnException), and im using functional mode and spawn two processes so they are run in parallel.

Ok now I get it, and indeed you are using the wrong tool.

My suggestion is: write a single @test and leverage 2 subprocesses created by you.
Once done, you can safely and indipendently use Paratest, given you use the TEST_TOKEN env variable for the shared resources like the filesystem (i.e. self::REBASE_IS_RUNNING_FLAG_PATH . getenv('TEST_TOKEN'))