backend: Deployments refactor; Add deployment service and fix deployment config setting #831

malexw · 2024-11-06T17:50:17Z

The motivation for this change was a request to improve how deployment configs are set in Coral.

Deployments are set up by creating subclasses of BaseDeployment and usually these have the configuration values we want to use when calling models. However, we also have the ability to make changes to these models, which are then stored in the DB. This refactor introduces a deployment_service module to hide the differences between models that are purely defined in code and models that have updates stored in the DB.

This change also finishes off the partially-completed Coral work that allows for changing model config values.

There is a new potential security issue now introduced. We are assuming that anyone who is running Toolkit should have permission to view and change global deployment configurations, which includes API keys. If that's not true, we'll need to add a concept of admin users in the future and prevent non-admins from viewing and changing these values.

AI Description

This PR introduces a new DeploymentNotFoundError exception to handle cases where a deployment is not found. It also refactors the get_deployment function to use a new deployment_service module, which provides methods for retrieving deployments by name or ID. The get_deployment function now returns a BaseDeployment instance, which is a base class for all model deployment options.

The PR also adds a new DeploymentInfo class, which represents the information required to interact with an LLM. This class is used in various parts of the codebase, including the create_deployment, update_deployment, and get_deployment functions.

Additionally, the PR updates the create_deployment_by_config and create_model_by_config functions to use the new DeploymentInfo class. It also updates the validate_deployment_config function to use the new deployment_service module.

The PR also introduces a new get_installed_deployments function, which returns a list of installed deployments. This function is used in the get_available_deployments function, which has been updated to use the new deployment_service module.

The PR also updates the get_deployments_info function to use the new deployment_service module. This function is used in the list_deployments function, which has been updated to use the new DeploymentInfo class.

The PR also introduces a new update_config function, which updates the configuration of a deployment. This function is used in the set_env_vars function, which has been updated to use the new deployment_service module.

The PR also updates the validate_env_vars function to use the new deployment_service module. This function is used in the create_agent function, which has been updated to use the new DeploymentInfo class.

The PR also introduces a new get_default_deployment function, which returns the default deployment. This function is used in the get_deployment function, which has been updated to use the new deployment_service module.

The PR also updates the get_available_deployments function to use the new deployment_service module. This function is used in the get_deployments_info function, which has been updated to use the new DeploymentInfo class.

The PR also updates the get_deployment_info function to use the new deployment_service module. This function is used in the get_deployment function, which has been updated to use the new DeploymentInfo class.

The PR also updates the get_deployment_info_by_name function to use the new deployment_service module. This function is used in the get_deployments_info function, which has been updated to use the new DeploymentInfo class.