You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Under certain conditions, ReadWriteFileConnection threads will accumulate after recompile. Affects StringSerializableDataSources (YML, JSON, INI, CSV, XML) in PersistenceNetwork. On recompile, by default all DataSources in the DataSourceFactory cache are cleared. There's a secondary cache in ThreadsafeDataSource using a WeakHashMap. If it's cleared from there by the time of next get/store_value() call, a new DataSource is created. This causes the StringSerializableDataSource to create a new ReadWriteFileConnection ConnectionMixin, which initializes a new ThreadPoolExecutor with a single core thread. This ExecutorService won't shut down until there are no core threads and all references are lost. Thus we now have two (and counting) ReadWriteFileConnection threads.
Recreation steps:
Setup JSON (or other StringSerializableDataSource) connection in persistence.ini
get|store a value in this DataSource to create core thread in ReadWriteFileConnection
Recompile to clear data sources from DataSourceFactory static map
Force a gc to clear data sources from ThreadsafeDataSource static map
get|store same StringSerializableDataSource again
Solution depends on intended/desired behavior of these systems, but race conditions should probably be considered. Here's some ideas I explored before posting this:
Make ThreadPoolExecutor static in ReadWriteFileConnection (only ever one thread for these connection types)
Shutdown ThreadPoolExecutor in a new method that's called in StringSerializableDataSource.disconnect() and then await termination to ensure all tasks are complete
Set core threads to zero (simple, but doesn't help with race conditions)
Do not clear DataSources from DataSourceFactory (would keep around DataSources for removed connections, but this is rare; would need to make sure disconnect() doesn't break any DataSources)
This won't always affect users that use these connection types, but in the mean time users can workaround this leak by using /recompile -r to avoid reloading the Persistence Network.
The text was updated successfully, but these errors were encountered:
This behavior wasn't immediately reproduceable on a clean installation. I suspect something was triggering gc in their ms scripts, and with multiple StringSerializableDataSources, this caused these waiting threads to accumulate quickly after recompiling many times ("100+").
Under certain conditions, ReadWriteFileConnection threads will accumulate after recompile. Affects StringSerializableDataSources (YML, JSON, INI, CSV, XML) in PersistenceNetwork. On recompile, by default all DataSources in the DataSourceFactory cache are cleared. There's a secondary cache in ThreadsafeDataSource using a WeakHashMap. If it's cleared from there by the time of next get/store_value() call, a new DataSource is created. This causes the StringSerializableDataSource to create a new ReadWriteFileConnection ConnectionMixin, which initializes a new ThreadPoolExecutor with a single core thread. This ExecutorService won't shut down until there are no core threads and all references are lost. Thus we now have two (and counting) ReadWriteFileConnection threads.
Recreation steps:
Solution depends on intended/desired behavior of these systems, but race conditions should probably be considered. Here's some ideas I explored before posting this:
This won't always affect users that use these connection types, but in the mean time users can workaround this leak by using
/recompile -r
to avoid reloading the Persistence Network.The text was updated successfully, but these errors were encountered: