Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Parsing of magres files #193

Merged
merged 35 commits into from
Feb 7, 2024
Merged

Parsing of magres files #193

merged 35 commits into from
Feb 7, 2024

Conversation

JosePizarro3
Copy link
Collaborator

@JosePizarro3 JosePizarro3 commented Jan 10, 2024 •
edited
Loading

I still need to do:

  • Parse spin-spin coupling parsing (isc in the code).
  • Add functionality connecting magres with CASTEP as a NMR workflow.

Related with the magres MR in Gitlab.

@JosePizarro3 JosePizarro3 self-assigned this Jan 10, 2024
@JosePizarro3 JosePizarro3 linked an issue Jan 10, 2024 that may be closed by this pull request
Parsing of .magres files #171
Open
@JosePizarro3
Copy link
Collaborator Author

@ladinesa here you have the changes. Bear in mind I defined magres_workflow.py in the utils, so it can be used (if I define it in the metainfo folder of magres I got a circular import problem).

Let me know what you think, I need still to parse the SpinSpinCoupling section, but I wanted to show you this first.

Thanks!

@JosePizarro3 JosePizarro3 requested a review from ladinesa January 17, 2024 13:56
@ladinesa
Copy link
Collaborator

(if I define it in the metainfo folder of magres I got a circular import problem)

can you paste the error here?

@JosePizarro3
Copy link
Collaborator Author

(if I define it in the metainfo folder of magres I got a circular import problem)

can you paste the error here?

This is it (again, only when defining in /magres/metainfo/magres.py and importing them in the utils.py using ..magres.metainfo.magres import NMRMagRes, NMRMagResMethod, NMRMagResResults:

ERROR    nomad.client         2024-01-17T14:05:29 parsing was not successful
  - exception: Traceback (most recent call last):
      File "/home/josepizarro/nomad/nomad/parsing/parsers.py", line 215, in run_parser
        parser.parse(mainfile_path, entry_archive, logger=logger, **kwargs)
      File "/home/josepizarro/nomad/nomad/parsing/parser.py", line 460, in parse
        self.mainfile_parser.parse(mainfile, archive, logger)
      File "/home/josepizarro/nomad/nomad/parsing/parser.py", line 451, in mainfile_parser
        raise e
      File "/home/josepizarro/nomad/nomad/parsing/parser.py", line 446, in mainfile_parser
        Parser = self.import_parser_class()
      File "/home/josepizarro/nomad/nomad/parsing/parser.py", line 463, in import_parser_class
        return import_class(self._parser_class_name, 'parser')
      File "/home/josepizarro/nomad/nomad/parsing/parser.py", line 419, in import_class
        raise e
      File "/home/josepizarro/nomad/nomad/parsing/parser.py", line 412, in import_class
        module = importlib.import_module(module_path)
      File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
      File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
      File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
      File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
      File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
      File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
      File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 850, in exec_module
      File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
      File "/home/josepizarro/nomad/dependencies/parsers/electronic/electronicparsers/fhiaims/__init__.py", line 19, in <module>
        from .parser import FHIAimsParser
      File "/home/josepizarro/nomad/dependencies/parsers/electronic/electronicparsers/fhiaims/parser.py", line 59, in <module>
        from ..utils import BeyondDFTWorkflowsParser
      File "/home/josepizarro/nomad/dependencies/parsers/electronic/electronicparsers/utils/__init__.py", line 19, in <module>
        from .utils import get_files, numpy_type_to_json_serializable, BeyondDFTWorkflowsParser
      File "/home/josepizarro/nomad/dependencies/parsers/electronic/electronicparsers/utils/utils.py", line 51, in <module>
        from ..magres.metainfo.magres import (
      File "/home/josepizarro/nomad/dependencies/parsers/electronic/electronicparsers/magres/__init__.py", line 19, in <module>
        from .parser import MagresParser
      File "/home/josepizarro/nomad/dependencies/parsers/electronic/electronicparsers/magres/parser.py", line 48, in <module>
        from ..utils import BeyondDFTWorkflowsParser
    ImportError: cannot import name 'BeyondDFTWorkflowsParser' from partially initialized module 'electronicparsers.utils' (most likely due to a circular import) (/home/josepizarro/nomad/dependencies/parsers/electronic/electronicparsers/utils/__init__.py)
  - exception_hash: 1zAJbjhQvtS4kVzlN_6ggqkidb9j
  - nomad.client.parser: parsers/fhi-aims
  - nomad.commit: 
  - nomad.deployment: devel
  - nomad.service: cli
  - nomad.version: 1.2.2.dev304+g663ab7d9d.d20240117

@ladinesa
Copy link
Collaborator

NMRMagResResults

This is expected and implies that we perhaps need to put the

(if I define it in the metainfo folder of magres I got a circular import problem)

can you paste the error here?

This is it (again, only when defining in /magres/metainfo/magres.py and importing them in the utils.py using ..magres.metainfo.magres import NMRMagRes, NMRMagResMethod, NMRMagResResults:

ERROR    nomad.client         2024-01-17T14:05:29 parsing was not successful
  - exception: Traceback (most recent call last):
      File "/home/josepizarro/nomad/nomad/parsing/parsers.py", line 215, in run_parser
        parser.parse(mainfile_path, entry_archive, logger=logger, **kwargs)
      File "/home/josepizarro/nomad/nomad/parsing/parser.py", line 460, in parse
        self.mainfile_parser.parse(mainfile, archive, logger)
      File "/home/josepizarro/nomad/nomad/parsing/parser.py", line 451, in mainfile_parser
        raise e
      File "/home/josepizarro/nomad/nomad/parsing/parser.py", line 446, in mainfile_parser
        Parser = self.import_parser_class()
      File "/home/josepizarro/nomad/nomad/parsing/parser.py", line 463, in import_parser_class
        return import_class(self._parser_class_name, 'parser')
      File "/home/josepizarro/nomad/nomad/parsing/parser.py", line 419, in import_class
        raise e
      File "/home/josepizarro/nomad/nomad/parsing/parser.py", line 412, in import_class
        module = importlib.import_module(module_path)
      File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
      File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
      File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
      File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
      File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
      File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
      File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 850, in exec_module
      File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
      File "/home/josepizarro/nomad/dependencies/parsers/electronic/electronicparsers/fhiaims/__init__.py", line 19, in <module>
        from .parser import FHIAimsParser
      File "/home/josepizarro/nomad/dependencies/parsers/electronic/electronicparsers/fhiaims/parser.py", line 59, in <module>
        from ..utils import BeyondDFTWorkflowsParser
      File "/home/josepizarro/nomad/dependencies/parsers/electronic/electronicparsers/utils/__init__.py", line 19, in <module>
        from .utils import get_files, numpy_type_to_json_serializable, BeyondDFTWorkflowsParser
      File "/home/josepizarro/nomad/dependencies/parsers/electronic/electronicparsers/utils/utils.py", line 51, in <module>
        from ..magres.metainfo.magres import (
      File "/home/josepizarro/nomad/dependencies/parsers/electronic/electronicparsers/magres/__init__.py", line 19, in <module>
        from .parser import MagresParser
      File "/home/josepizarro/nomad/dependencies/parsers/electronic/electronicparsers/magres/parser.py", line 48, in <module>
        from ..utils import BeyondDFTWorkflowsParser
    ImportError: cannot import name 'BeyondDFTWorkflowsParser' from partially initialized module 'electronicparsers.utils' (most likely due to a circular import) (/home/josepizarro/nomad/dependencies/parsers/electronic/electronicparsers/utils/__init__.py)
  - exception_hash: 1zAJbjhQvtS4kVzlN_6ggqkidb9j
  - nomad.client.parser: parsers/fhi-aims
  - nomad.commit: 
  - nomad.deployment: devel
  - nomad.service: cli
  - nomad.version: 1.2.2.dev304+g663ab7d9d.d20240117

can you remove all under utils/init.py and use the absolute path.

@JosePizarro3
Copy link
Collaborator Author

can you remove all under utils/init.py and use the absolute path.

How do you mean? Do you mean:

  1. Add NMRMagRes workflow in magres/metainfo/magres.py.
  2. Delete ALL imports in utils/__init__.py?
  3. Import NMRMagRes in utils/utils.py with ..magres.metainfo.magres?

@ladinesa
Copy link
Collaborator

can you remove all under utils/init.py and use the absolute path.

How do you mean? Do you mean:

  1. Add NMRMagRes workflow in magres/metainfo/magres.py.
  2. Delete ALL imports in utils/__init__.py?
  3. Import NMRMagRes in utils/utils.py with ..magres.metainfo.magres?

Do number 2 and do the utils imports as from electronicparsers.utils import. Then perhaps you can have NMRMagRes in magres/metainfo

@JosePizarro3
Copy link
Collaborator Author

can you remove all under utils/init.py and use the absolute path.

How do you mean? Do you mean:

  1. Add NMRMagRes workflow in magres/metainfo/magres.py.
  2. Delete ALL imports in utils/__init__.py?
  3. Import NMRMagRes in utils/utils.py with ..magres.metainfo.magres?

Do number 2 and do the utils imports as from electronicparsers.utils import. Then perhaps you can have NMRMagRes in magres/metainfo

Ok, I managed to make it work. You will see a lot of files changed so now I am using the abs path for the utils in other parsers. Let me know what you think.

If this is ready then, I will focus on finishing today / tomorrow the Gitlab side.

@JosePizarro3
Copy link
Collaborator Author

Ok, it seems that, even tho the pathing is working in Github, it is completely broken from the central nomad (you can see the testing here).

I guess one has to write the whole path, but the it brokes the local github testing.

I will leave it to how it was: define utils/magres_workflow.py and import it in utils/__init__.py.

@JosePizarro3 JosePizarro3 force-pushed the 171-parsing-of-magres-files branch 2 times, most recently from fc62ade to 06568de Compare February 5, 2024 14:33
ladinesa and others added 27 commits February 7, 2024 09:28
@ladinesa @JosePizarro3
Add metainfo import
66cee6c
@JosePizarro3
Added initial version of magres parser
89cb080
@JosePizarro3
Added NMR magres flag to CASTEP in section run
8d14c8b
@JosePizarro3
Added method to magres
78bb81b 
Added Method.label to magres and CASTEP
@JosePizarro3
Fixing bugs in magres
a572ed1
@JosePizarro3
Added more parsing to magres
b3a401b 
Added test for magres
@JosePizarro3
Fix naming test solid_dmft
7422efd 
Added condition for DMFT workflow in w2dynamics
@JosePizarro3
Deleting x_castep_nmr_flag
c39a4b6
@JosePizarro3
Applying auto ruff to magres parser
510ddb4
@JosePizarro3
Added Ruff formatting to utils.py
8c6efbd
@JosePizarro3
Added parsing workflow for magres
5f98c18
@JosePizarro3
Fixed input
8a226fb
@JosePizarro3
Fixed parse_tb_workflow in utils
b5b030c
@JosePizarro3
Defined NMRMagRes workflow in magres parser
b423f04 
Fixed utils workflow extraction to new naming of workflow classes
@JosePizarro3
Clean up NMRMagRes file format parsing
41a6119
@JosePizarro3
Added magres_workflow to utils due to circular import problem in util…
6585af0 
…s.py
@JosePizarro3
New naming quantities for SpinSpinCoupling
49fcfe0
@JosePizarro3
Deleted m_create
0c3b950 
Removed variables in warnings
@JosePizarro3
Added parsing of indirect spin spin coupling
0cc9ccd
@JosePizarro3
Better conditions for automatic connection with CASTEP
6602502 
Fixing bugs
@JosePizarro3
Fix parser
d729d7a
@JosePizarro3
Fix paths
daebee7
@JosePizarro3
Forget utils in edmft
e5553f9
@JosePizarro3
Fixed mypy
27dedac
@JosePizarro3
Deleted indirect from SpinSpinCoupling quantities
c756bb5
@JosePizarro3
Reverting import utils changes
aca114d
@JosePizarro3
Add new imports
bb5eb98
@JosePizarro3 JosePizarro3 force-pushed the 171-parsing-of-magres-files branch from 06568de to bb5eb98 Compare February 7, 2024 08:28
@JosePizarro3 JosePizarro3 merged commit 5ab4822 into develop Feb 7, 2024
2 checks passed
@JosePizarro3 JosePizarro3 deleted the 171-parsing-of-magres-files branch February 7, 2024 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ladinesa ladinesa ladinesa left review comments

Assignees

@JosePizarro3 JosePizarro3

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Parsing of .magres files
2 participants
@JosePizarro3 @ladinesa