Multithreaded access error when validating fields

[AlexanderWells-diamond]

It seems that part(s) of epicsdbbuilder are not threadsafe. Inside of ValidateDbField is a single DBEntry item, which is used as a cursor into a database. However, inside the ValidateFieldValue method, which can be called from multiple threads simultaneously, the cursor gets re-used for multiple invocations.

This has the end result of creating an error message like this:

AssertionError: Can't write "@PV-FROM-CM" to field OUT: fldDes not found

Which comes from this line.

This was originally spotted in PythonSoftIOC's scheduled testing, where this error message very occasionally appears during scheduled testing. See this issue.

[Araneidae]

Do we have a nice performance regression test for pythonIoc? I remember in the past having to be quite clever about locking to avoid a serious performance bottleneck in the startup of a large soft IOC (there's a slightly cryptic comment about this in one of the code bases, but I can't remember where right now). I can think of a couple of fixes for the cursor problem, but it would be good to be confident that they won't cause performance regression.

The fixes are one of:

1. Add a mutex
2. Remove the shared cursor and regenerate it on every use (I think this is compatible with how the function is used, but needs checking).

Both of these have potential performance impact.

[AlexanderWells-diamond]

We have no performance testing for PythonSoftIOC. It's quite hard to set one up, especially one to run in CI, due to the sheer number of variables to control.