Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Supporting deployments on zos4 nodes #3497

Open
wants to merge 35 commits into
base: development
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
aa5ae9b
Supporting deployments on zos4 nodes
maayarosama Oct 3, 2024
2503bf6
Fixing tests, adding vmlight primitive and adding a match method
maayarosama Oct 13, 2024
2f61d6b
Adding networklight class in primitives
maayarosama Oct 15, 2024
de4bab1
Merge branch 'development' into development_zos4
maayarosama Oct 23, 2024
a9cdb25
Updating networklight class in primitives
maayarosama Oct 23, 2024
84002ac
Adjusting FilterOptions to support features in automatic and manual n…
maayarosama Nov 4, 2024
70c99cb
Merge branch 'development' into development_zos4
maayarosama Nov 4, 2024
212441e
Adding wiregaurd to filters
maayarosama Nov 4, 2024
8f3f19b
Merge branch 'development' into development_zos4
maayarosama Nov 4, 2024
880c0fd
Adjusting getFeaturesFromFilters method and adding planetary, myceliu…
maayarosama Nov 4, 2024
8419d9b
Updating networklight primitive class, create deployment and create m…
maayarosama Nov 6, 2024
90a353d
Adjusting load, NodeExists, addNode and getFreeSubnet methods in netw…
maayarosama Nov 10, 2024
391fedf
Merge branch 'development' into development_zos4
maayarosama Nov 12, 2024
496aa5a
Updating getFeaturesFromFilters method
maayarosama Nov 13, 2024
0b31535
Fixing reservation network error, updating k8s to support zos4 deplo…
maayarosama Nov 17, 2024
b25d63b
Merge branch 'development' into development_zos4
maayarosama Nov 17, 2024
bab3ca6
Added a script to deploy on a zos4 node
maayarosama Nov 17, 2024
23d742a
Adding deployment flow doc
maayarosama Nov 17, 2024
17c3ec9
Resolving comments: removing unnecessary comments, adding wiregaurd i…
maayarosama Nov 20, 2024
14ab9b9
Updating network light primitive class
maayarosama Nov 25, 2024
e8e9cab
Revert "Resolving comments: removing unnecessary comments, adding wir…
maayarosama Nov 26, 2024
386b9d2
Revert "Updating network light primitive class"
maayarosama Nov 26, 2024
734dea1
Added some changes in networklight primitive and added some more adju…
maayarosama Nov 27, 2024
65d443f
Fixing _deleteMachineNetwork function
maayarosama Nov 28, 2024
1d7da56
Adjusting manual selection feature error, fixing the public_ip rando…
maayarosama Dec 1, 2024
0215009
Adjusting manual selection feature error, fixing the public_ip rando…
maayarosama Dec 1, 2024
061df42
Merge branch 'development' into development_zos4
maayarosama Dec 8, 2024
a80242c
Added the required features in all scripts' filters
maayarosama Dec 8, 2024
4dc4508
Adjusting Supported interface in manage domains in case of zmachine-l…
maayarosama Dec 9, 2024
af39e90
Adjust zos4 script and merging functionalites of delete zmachine and …
maayarosama Dec 11, 2024
b4778f9
Merge branch 'development' into development_zos4
maayarosama Dec 15, 2024
af97202
Resolving comments: Addinf Features enum, using WorkloadTypes in Zmac…
maayarosama Dec 15, 2024
68f0c7b
Merge branch 'development' into development_zos4
maayarosama Dec 16, 2024
c553325
Refactoring vm primitive file, adjusting features type in filtersOpti…
maayarosama Dec 16, 2024
0609f41
Fixing typo in deployment flow readme file
maayarosama Dec 18, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
117 changes: 117 additions & 0 deletions packages/grid_client/docs/deployment_flow.md
maayarosama marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# Deployment Flow

This document outlines the process for determining whether to deploy on a zos3 or zos4 node.
maayarosama marked this conversation as resolved.
Show resolved Hide resolved

## Machine Model Initialization

The deployment process begins with initializing the machine model as follows:

```ts
const vms: MachinesModel = {
name: "newMY",
network: {
name: "hellotest",
ip_range: "10.249.0.0/16",
myceliumSeeds: [
{
nodeId: 168,
seed: "050d109829d8492d48bfb33b711056080571c69e46bfde6b4294c4c5bf468a76", //(HexSeed of length 32)
},
],
},
machines: [
{
name: "testvmMY",
node_id: 168,
disks: [
{
name: "wedDisk",
size: 8,
mountpoint: "/testdisk",
},
],
public_ip: false,
public_ip6: false,
planetary: true,
mycelium: true,
myceliumSeed: "1e1404279b3d", //(HexSeed of length 6)
cpu: 1,
memory: 1024 * 2,
rootfs_size: 0,
flist: "https://hub.grid.tf/tf-official-apps/base:latest.flist",
entrypoint: "/sbin/zinit init",
env: {
SSH_KEY: config.ssh_key,
},
},
],
metadata: "",
description: "test deploying single VM with mycelium via ts grid3 client",
};
```

## Deployment Execution

- The next step is invoking the `deploy` function:
- Takes the `MachinesModel` object as a parameter
- Checks if a machine with the same name already exists and if so throws an error
- If not it calls the `_createDeployment` function along some othe functions
- Finally it returns the created contracts and the wiregaurd configuration

```ts
await client.machines.deploy(vms);
```

- The `_createDeployment` function:
- Takes the `MachinesModel` object as a parameter
- Retrieves the features of the node using: `await this.rmb.request([nodeTwinId], "zos.system.node_features_get", "", 20, 3);`
- Examines the retrieved features to determine the network's primitive type (`Network` or `ZNetworkLight`) and initializes it accordingly.
- Sets the contractMetadata based on the network type.
- Invokes the `create` function

```ts
await this._createDeployment(options);
```

- The `create` function
- Validates or assigns IP addresses based on the network type.
- Determines network type (`network` or `network-light`) based on node features.
- Adds access points and updates network configurations as necessary.
- Initialize the VM primitive (`VMPrimitive` or `VMLightPrimitive`) based on the network type
- Configures the VM with networking, storage, and environment variables.
- Generates a Mycelium seed if not provided.
- Generate the deployments
- Returns the deployments and wiregaurd configurations

```ts
await this.vm.create(
machine.name,
machine.node_id,
machine.flist,
machine.cpu,
machine.memory,
machine.rootfs_size,
machine.disks!,
machine.public_ip,
machine.public_ip6!,
machine.planetary,
machine.mycelium,
machine.myceliumSeed!,
network,
options.network.myceliumSeeds!,
machine.entrypoint,
machine.env,
contractMetadata,
options.metadata,
options.description,
machine.qsfs_disks,
this.config.projectName,
options.network.addAccess,
options.network.accessNodeId,
machine.ip,
machine.corex,
machine.solutionProviderId!,
machine.zlogsOutput,
machine.gpus,
);
```
108 changes: 108 additions & 0 deletions packages/grid_client/scripts/single_vm_zos4.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
import { generateRandomHexSeed, GridClient, MachinesDeleteModel, MachinesModel } from "../src";
import { config, getClient } from "./client_loader";
import { log } from "./utils";

async function deploy(client: GridClient, vms: MachinesModel) {
const res = await client.machines.deploy(vms);
log("================= Deploying VM =================");
log(res);
log("================= Deploying VM =================");
}

async function getDeployment(client: GridClient, name: string) {
const res = await client.machines.getObj(name);
log("================= Getting deployment information =================");
log(res);
log("================= Getting deployment information =================");
}

async function cancel(client: GridClient, options: MachinesDeleteModel) {
const res = await client.machines.delete(options);
log("================= Canceling the deployment =================");
log(res);
log("================= Canceling the deployment =================");
}

async function main() {
const name = "newMY";
const grid3 = await getClient(`vm/${name}`);

const vms: MachinesModel = {
name,
network: {
name: "hellotest",
ip_range: "10.249.0.0/16",
myceliumSeeds: [
{
nodeId: 255,
/**
* ### Mycelium Network Seed:
* - The `seed` is an optional field used to provide a specific seed for the Mycelium network.
* - If not provided, the `GridClient` will generate a seed automatically when the `mycelium` flag is enabled.
* - **Use Case:** If you need the new machine to have the same IP address as a previously deleted machine, set the `seed` field to the old seed value.
*/
seed: generateRandomHexSeed(32),
},
],
},
machines: [
{
name: "testvmMY",
node_id: 255,
maayarosama marked this conversation as resolved.
Show resolved Hide resolved
disks: [
{
name: "wedDisk",
size: 8,
mountpoint: "/testdisk",
},
],
public_ip: false,
public_ip6: false,
planetary: true,
/**
* ### Mycelium Flag Behavior:
* - When the `mycelium` flag is enabled, there’s no need to manually provide the `myceliumSeed` flag.
* - The `GridClient` will automatically generate the necessary seed for you.
* - **However**, if you have **an existing seed** from a previously deleted machine and wish to deploy a new machine that retains the same IP address,
* - **you can simply pass in the old seed during deployment instead of calling the `generateRandomHexSeed()` function**.
*/
mycelium: true,
/**
* ### Mycelium Seed:
* - The `myceliumSeed` is an optional field used to provide a specific seed for the Mycelium network.
* - If not provided, the `GridClient` will generate a seed automatically when the `mycelium` flag is enabled.
* - **Use Case:** If you need the new machine to have the same IP address as a previously deleted machine, set the `seed` field to the old seed value. */
myceliumSeed: generateRandomHexSeed(6), // (HexSeed of length 6)
cpu: 1,
memory: 265,
rootfs_size: 0,
flist: "https://hub.grid.tf/tf-official-apps/base:latest.flist",
entrypoint: "/sbin/zinit init",
env: {
SSH_KEY: config.ssh_key,
},
/**
* ### Features:
* - The `features` is an optional field used to provide a specific feature to the node.
* - If not provided, The deploy method will used the get feature zos call to get node's features and deploy according to that.
*/
features: ["zmachine-light"],
},
],
metadata: "",
description: "test deploying single ZOS4 VM with mycelium via ts grid3 client",
};

//Deploy VMs
await deploy(grid3, vms);

//Get the deployment
await getDeployment(grid3, name);

//Uncomment the line below to cancel the deployment
// await cancel(grid3, { name });

await grid3.disconnect();
}

main();
5 changes: 3 additions & 2 deletions packages/grid_client/src/high_level/kubernetes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { events } from "../helpers/events";
import { VMHL } from "../high_level//machine";
import { MyceliumNetworkModel, QSFSDiskModel } from "../modules/models";
import { Network } from "../primitives/network";
import { ZNetworkLight } from "../primitives/networklight";
import { Deployment } from "../zos/deployment";
import { WorkloadTypes } from "../zos/workload";
import { HighLevelBase } from "./base";
Expand All @@ -22,7 +23,7 @@ class KubernetesHL extends HighLevelBase {
planetary: boolean,
mycelium: boolean,
myceliumSeed: string,
network: Network,
network: Network | ZNetworkLight,
myceliumNetworkSeeds: MyceliumNetworkModel[] = [],
sshKey: string,
contractMetadata: string,
Expand Down Expand Up @@ -100,7 +101,7 @@ class KubernetesHL extends HighLevelBase {
planetary: boolean,
mycelium: boolean,
myceliumSeed: string,
network: Network,
network: Network | ZNetworkLight,
myceliumNetworkSeeds: MyceliumNetworkModel[] = [],
sshKey: string,
contractMetadata,
Expand Down
Loading
Loading