Executor improvements Part 01 #1924
Conversation
| from covalent.executor import _executor_manager | ||
|
|
||
| logger = logging.getLogger() | ||
| logger.setLevel(logging.ERROR) | ||
| handler = logging.StreamHandler(sys.stderr) | ||
| logger.addHandler(handler) | ||
| logger.propagate = False | ||
| sdk_constants = get_default_sdk_config() |
There was a problem hiding this comment.
We should use get_config instead of directly importing the defaults since they might've been changed by the user in the config file directly. Using get_config with specifically what is needed (instead of the entire sdk config) will take that into consideration as well.
| @@ -389,7 +397,24 @@ def _get_tf_statefile_path(self) -> str: | |||
|
|
|||
| """ | |||
| # Saving in a directory which doesn't get deleted on purge | |||
| return str(Path(self.executor_tf_path) / "terraform.tfstate") | |||
| return str(Path(self.executor_tfstate_path) / "terraform.tfstate") | |||
There was a problem hiding this comment.
Saving in a directory which doesn't get deleted on purge
self.executor_tfstate_path seems to point to ~/.config/covalent/executor_plugins/*, this WILL get deleted by purge. We need to add an exception in the purge command to prevent deletion of these state files.
| # Saving in a directory which doesn't get deleted on purge | ||
| state_path = os.path.join(parent_path, self.executor_name) | ||
| if not os.path.exists(state_path): | ||
| os.makedirs(state_path) |
There was a problem hiding this comment.
Same comment as above. One minor thing is that if possible let's try to use pathlib's Path instead of os.path to join/checking existence/checking whether its a directory/etc.
Changed