Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose programmatic equivalent of CLI commands #1854

Merged
merged 48 commits into from
Nov 24, 2023
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
d030306
export `covalent_start` and `covalent_stop`
araghukas Nov 20, 2023
290481f
check server stopped
araghukas Nov 20, 2023
fab67cd
update changelog
araghukas Nov 20, 2023
46b395c
move commands to main namespace
araghukas Nov 20, 2023
f706575
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 20, 2023
afb9aad
improve docstrings
araghukas Nov 21, 2023
9eba72a
fix `covalent_is_running` to return bool
araghukas Nov 21, 2023
a1004e3
reorder `covalent_is_running` conditions
araghukas Nov 21, 2023
8622ac3
`quiet` mode to suppress stdout; more docstrings
araghukas Nov 21, 2023
7e4117b
use poll function instead of while loop
araghukas Nov 21, 2023
58a48b2
explain package
araghukas Nov 21, 2023
bd367cb
add api docs entry
araghukas Nov 21, 2023
c784193
update api docs
araghukas Nov 21, 2023
e430db5
restore import from `._programmatic`
araghukas Nov 21, 2023
10cd66e
update api docs
araghukas Nov 21, 2023
4cbe7d3
Merge branch 'develop' into 1853-programmatic-cli-commands
araghukas Nov 21, 2023
e18852d
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 21, 2023
8ea150a
add test for new functions
araghukas Nov 21, 2023
e023bde
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 21, 2023
60f9ca2
add test for `covalent_is_running`
araghukas Nov 21, 2023
112c5f4
removing covalent's dependency on dispatcher
kessler-frost Nov 21, 2023
d7c6c05
ignore pip reqs in new package
araghukas Nov 21, 2023
2b4d947
refactor docstrings
araghukas Nov 21, 2023
bca8bc0
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 21, 2023
7d18fa4
update docs
araghukas Nov 21, 2023
f6e0e3a
revert api docs
araghukas Nov 21, 2023
d54ee56
include 'Returns' in docstrings so maybe docs will render 🤷 pls
araghukas Nov 21, 2023
ea1c6a6
remove useless 'Returns' from docstrings 🤦‍♂️
araghukas Nov 21, 2023
d7ef463
try autofunction refs to main namespace instead
araghukas Nov 21, 2023
8b107eb
revert using main namespace refs
araghukas Nov 21, 2023
de284c1
add more logging and edit messages
araghukas Nov 22, 2023
13bfdb6
refactor hanging tests
araghukas Nov 22, 2023
f308f8e
refactor tests into functional tests
araghukas Nov 22, 2023
e7709ea
Revert "refactor tests into functional tests"
araghukas Nov 23, 2023
cf6f286
create global var for timeout
araghukas Nov 23, 2023
07fa34e
use mock start and stop commands
araghukas Nov 23, 2023
11247fc
Merge branch 'develop' into 1853-programmatic-cli-commands
kessler-frost Nov 23, 2023
73ac729
Merge branch 'develop' into 1853-programmatic-cli-commands
kessler-frost Nov 23, 2023
3294979
Merge branch 'develop' into 1853-programmatic-cli-commands
kessler-frost Nov 23, 2023
f7220c6
Merge branch 'develop' into 1853-programmatic-cli-commands
kessler-frost Nov 23, 2023
61c27b8
renamed server check function and added import error check tests
kessler-frost Nov 24, 2023
243982b
None wasn't an acceptable value to redirect_stdout
kessler-frost Nov 24, 2023
0e1e07c
refactor to use subprocess
araghukas Nov 24, 2023
ea53935
refactor as multiple tests w/ patched start/stop
araghukas Nov 24, 2023
cf6e48a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 24, 2023
0491ab1
add nopycln inside new tests
araghukas Nov 24, 2023
2b7f126
renaming things a bit
kessler-frost Nov 24, 2023
1daecca
Merge branch 'develop' into 1853-programmatic-cli-commands
kessler-frost Nov 24, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [UNRELEASED]

### Added

- Programmatic equivalents of CLI commands `covalent start` and `covalent stop`

### Fixed

- Lattice-default metadata attributes are now applied correctly
Expand Down
15 changes: 15 additions & 0 deletions covalent/_programmatic/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Copyright 2021 Agnostiq Inc.
#
# This file is part of Covalent.
#
# Licensed under the Apache License 2.0 (the "License"). A copy of the
# License may be obtained with this software package or at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Use of this file is prohibited except in compliance with the License.
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
96 changes: 96 additions & 0 deletions covalent/_programmatic/commands.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
# Copyright 2021 Agnostiq Inc.
#
# This file is part of Covalent.
#
# Licensed under the Apache License 2.0 (the "License"). A copy of the
# License may be obtained with this software package or at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Use of this file is prohibited except in compliance with the License.
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""Functions providing programmatic access to Covalent CLI commands."""
import time
from typing import Any, Dict, Optional

Check warning on line 19 in covalent/_programmatic/commands.py

View check run for this annotation

Codecov / codecov/patch

covalent/_programmatic/commands.py#L17-L19

Added lines #L17 - L19 were not covered by tests

import click
import psutil

Check warning on line 22 in covalent/_programmatic/commands.py

View check run for this annotation

Codecov / codecov/patch

covalent/_programmatic/commands.py#L21-L22

Added lines #L21 - L22 were not covered by tests

from covalent_dispatcher._cli.service import _read_pid, start, stop

Check warning on line 24 in covalent/_programmatic/commands.py

View check run for this annotation

Codecov / codecov/patch

covalent/_programmatic/commands.py#L24

Added line #L24 was not covered by tests

from .._shared_files import logger
from .._shared_files.config import get_config

Check warning on line 27 in covalent/_programmatic/commands.py

View check run for this annotation

Codecov / codecov/patch

covalent/_programmatic/commands.py#L26-L27

Added lines #L26 - L27 were not covered by tests

app_log = logger.app_log

Check warning on line 29 in covalent/_programmatic/commands.py

View check run for this annotation

Codecov / codecov/patch

covalent/_programmatic/commands.py#L29

Added line #L29 was not covered by tests


def _call_cli_command(func: click.Command, **kwargs: Dict[str, Any]) -> Any:

Check warning on line 32 in covalent/_programmatic/commands.py

View check run for this annotation

Codecov / codecov/patch

covalent/_programmatic/commands.py#L32

Added line #L32 was not covered by tests
"""Call the CLI command ``func`` with the specified kwargs."""
ctx = click.Context(func)
ctx.invoke(func, **kwargs)

Check warning on line 35 in covalent/_programmatic/commands.py

View check run for this annotation

Codecov / codecov/patch

covalent/_programmatic/commands.py#L34-L35

Added lines #L34 - L35 were not covered by tests


def covalent_is_running() -> bool:

Check warning on line 38 in covalent/_programmatic/commands.py

View check run for this annotation

Codecov / codecov/patch

covalent/_programmatic/commands.py#L38

Added line #L38 was not covered by tests
"""Return True if the Covalent server is in a ready state."""
pid = _read_pid(get_config("dispatcher.cache_dir") + "/ui.pid")
return (

Check warning on line 41 in covalent/_programmatic/commands.py

View check run for this annotation

Codecov / codecov/patch

covalent/_programmatic/commands.py#L40-L41

Added lines #L40 - L41 were not covered by tests
psutil.pid_exists(pid)
and pid != -1
and get_config("dispatcher.address")
and get_config("dispatcher.port")
)


def covalent_start(

Check warning on line 49 in covalent/_programmatic/commands.py

View check run for this annotation

Codecov / codecov/patch

covalent/_programmatic/commands.py#L49

Added line #L49 was not covered by tests
develop: bool = False,
port: Optional[str] = None,
mem_per_worker: Optional[str] = None,
workers: Optional[int] = None,
threads_per_worker: Optional[int] = None,
ignore_migrations: bool = False,
no_cluster: bool = False,
no_triggers: bool = False,
triggers_only: bool = False,
):
"""
santoshkumarradha marked this conversation as resolved.
Show resolved Hide resolved
Start the Covalent server. Wrapper for the `covalent start` CLI command.
"""
if covalent_is_running():
return

Check warning on line 64 in covalent/_programmatic/commands.py

View check run for this annotation

Codecov / codecov/patch

covalent/_programmatic/commands.py#L63-L64

Added lines #L63 - L64 were not covered by tests

kwargs = {

Check warning on line 66 in covalent/_programmatic/commands.py

View check run for this annotation

Codecov / codecov/patch

covalent/_programmatic/commands.py#L66

Added line #L66 was not covered by tests
"develop": develop,
"port": port or get_config("dispatcher.port"),
"mem_per_worker": mem_per_worker or get_config("dask.mem_per_worker"),
"workers": workers or get_config("dask.num_workers"),
"threads_per_worker": threads_per_worker or get_config("dask.threads_per_worker"),
"ignore_migrations": ignore_migrations,
"no_cluster": no_cluster,
"no_triggers": no_triggers,
"triggers_only": triggers_only,
}

_call_cli_command(start, **kwargs)

Check warning on line 78 in covalent/_programmatic/commands.py

View check run for this annotation

Codecov / codecov/patch

covalent/_programmatic/commands.py#L78

Added line #L78 was not covered by tests

while not covalent_is_running():
app_log.debug("Waiting for Covalent Server to be to dispatch-ready...")
time.sleep(1)

Check warning on line 82 in covalent/_programmatic/commands.py

View check run for this annotation

Codecov / codecov/patch

covalent/_programmatic/commands.py#L80-L82

Added lines #L80 - L82 were not covered by tests


def covalent_stop():

Check warning on line 85 in covalent/_programmatic/commands.py

View check run for this annotation

Codecov / codecov/patch

covalent/_programmatic/commands.py#L85

Added line #L85 was not covered by tests
"""
Stop the Covalent server. Wrapper for the `covalent stop` CLI command.
"""
if not covalent_is_running():
santoshkumarradha marked this conversation as resolved.
Show resolved Hide resolved
return

Check warning on line 90 in covalent/_programmatic/commands.py

View check run for this annotation

Codecov / codecov/patch

covalent/_programmatic/commands.py#L89-L90

Added lines #L89 - L90 were not covered by tests

_call_cli_command(stop)

Check warning on line 92 in covalent/_programmatic/commands.py

View check run for this annotation

Codecov / codecov/patch

covalent/_programmatic/commands.py#L92

Added line #L92 was not covered by tests

while covalent_is_running():
app_log.debug("Waiting for Covalent Server to stop...")
time.sleep(1)

Check warning on line 96 in covalent/_programmatic/commands.py

View check run for this annotation

Codecov / codecov/patch

covalent/_programmatic/commands.py#L94-L96

Added lines #L94 - L96 were not covered by tests
Loading