Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Writing egg files from psyllid fails #3

Open
laroque opened this issue Jun 6, 2019 · 1 comment
Open

Writing egg files from psyllid fails #3

laroque opened this issue Jun 6, 2019 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@laroque
Copy link
Member

laroque commented Jun 6, 2019

To reproduce, run the attached config files (remove the extra .txt extension required by github for upload), with a local egg file (match the egg file's name to the path in the config files).

psyllid_egg_reader_mask_only.yaml.txt
psyllid_egg_reader_trigger_only.yaml.txt

@laroque laroque added the bug Something isn't working label Jun 6, 2019
@laroque laroque self-assigned this Jun 6, 2019
@laroque
Copy link
Member Author

laroque commented Jul 23, 2019

We've determined that the problem in the second file is that libhdf5 is inherently not thread safe (even when working with different files). This configuration has two threads, one ready an input egg and another writing an output egg. To deal with this:

  • we can add the --enable-threadsafe flag, but this is an unsupported configuration (combining with --enable-cxx). There's a flag to override that, with it on it appears to work but we haven't tested very far.
  • we discussed adding a global mutex in monarch to provide threadsafety around all hdf5 interactions
  • the most extreme option would be to rewrite monarch to use only the lower-level C interface (instead of the much more convenient C++ interface); this seems like significant work and we don't currently have a plan to do it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant