AKS Extension Enable KAITO AI Toolchain operator add on in AKS cluster.

[Tatsinnit]

Enable Implementation of Kaito AI Toolchain Operator in AKS VSCode Extension (In TS)

1. Overview

We would like to request the integration of implementing Kaito AI Toolchain operators within the AKS Visual Studio Code extension. This will streamline the development process for those using VsCode with AKS doing all the heavy lifting upfront, and enhance the functionality of the Kaito AI Toolchain when working with Kubernetes operators in Azure.

2. Problem Statement

Currently, the AKS Visual Studio Code extension does not natively support the creation of Kaito AI Toolchain operators Installation. AI/ML-Developers working with VsCode makes engineering life easier when we integrate (KAITO) AI toolchain operators, which adds simplification to the entry point for KAITO and eases the complexity and overhead to the development AI lifecycle. One such document reside here: https://learn.microsoft.com/en-us/azure/aks/ai-toolchain-operator

3. Proposed Solution

We propose the inclusion of One-Click install of KAITO and generation of workflow as a first-class citizen for Kaito AI Toolchain operator development within the AKS VSCode extension. This can be achieved by:

• Many Moving pieces under one-click Ease x Integration of Azure TypeScript SDKs: Incorporating innovative development kits into the extension to allow the creation, validation, and deployment of Kaito AI Toolchain operators directly in TypeScript.

• TypeScript Operator Templates: Offering pre-configured templates for Kaito AI Toolchain operators that are written in VsCode eco-system, enabling developers to quickly get started without the need for extensive configuration.

4. Benefits

• Increased Developer Productivity.

• Wider Adoption.

• Improved Code Maintainability.

5. Use Case Example

A data scientist is building a machine learning operator using the Kaito AI Toolchain in AKS. By providing one-click feature this will help in quicker engineering adoption for KAITO into AKS.

6. Impact

This feature will significantly improve the user experience for KAITO enablement for AKS cluster along with ease to entry for AI developers and enhance the overall usability of the AKS extension for Kaito AI Toolchain integration. By aligning with modern development practices, this feature will ensure the extension remains competitive and relevant to AI developers.

7. Conclusion

Adding One-Click install for KAITO support for Kaito AI Toolchain operators in the AKS VSCode extension will address existing limitations, increase productivity, and attract a broader developer base. We believe this feature will contribute greatly to the growth and success of the Kaito AI Toolchain ecosystem.

Development Idea AKS SDK Usage

1: SDK Ways to approach and less to get lost in dense doc mentioned above.

Sample code

https://learn.microsoft.com/en-us/rest/api/resources/features/register?view=rest-resources-2021-07-01&tabs=JavaScript

        const { FeatureClient } = require("@azure/arm-features");
        const { DefaultAzureCredential } = require("@azure/identity");

        async function registerFeature() {
          const resourceProviderNamespace = "Resource Provider Namespace"; // <-- --namespace "Microsoft.ContainerService" 
          const featureName = "feature"; // <--  --name "AIToolchainOperatorPreview"
          const credential = new DefaultAzureCredential(); 
          const client = new FeatureClient(credential, subscriptionId);
          const result = await client.features.register(resourceProviderNamespace, featureName);
          console.log(result);
        }
        
        registerFeature().catch(console.error);

2: Observation Part

3: CRD Generation

[hsubramanianaks]

@Tatsinnit Thanks for the details, here is the basic flow, of course there will be improvements to this , ex: what if Kaito is installed already etc..?

1. Install Kaito - Register the preview feature, install kaito as addon and verify installation
2. select LLM models - we can either have config files of supported models by Kaito or filter it using hugging face API. seems like static config list is quick win.
3. generate workspace CRD yaml and show the CRD yaml.
4. deploy the CRD yaml.

For installing Kaito below is the javascript sdk - https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/containerservice/arm-containerservice
version: 21.1.0-beta.1 - this version only contains the support for Kaito
model: https://github.com/Azure/azure-sdk-for-js/blob/6dd8b582e2f5a3257befab368bab0e64499f4d57/sdk/containerservice/arm-containerservice/src/models/index.ts#L2473

Sample code for installing Kaito via managedcluster sdk

const managedClusterSpec: ManagedCluster = {
            location: cluster.location,
            aiToolchainOperatorProfile: { enabled: true },
            oidcIssuerProfile: { enabled: true },
        };

const poller = await this.containerServiceClient.managedClusters.beginCreateOrUpdate(
               resourceGrpName,
                clusterName,
                managedClusterSpec,
            );

[Tatsinnit]

Nice, and thanks for above Hari, Indeed, regarding what is Kaito is already install, so user owns the cluster but we have this indicator

• User can check that from the run ubectl command page about the Kaito pods
• Or, we can pre-check that for the user before starting the install.
• Also, third thing will be the CRD feature will only be enabled if the cluster has the Katio installed, we should check feasibility of it because that flag enablement is new and if it work for enabling if not we can always pre-check on generate CRD command and inform use off that is no the case.

[Tatsinnit]

Note: Currently supported model families in Kaito are documented here:

• https://github.com/Azure/kaito/tree/main/presets