diff --git a/doc/content/toolstack/features/MVD/index.md b/doc/content/toolstack/features/MVD/index.md new file mode 100644 index 00000000000..c8a696c191b --- /dev/null +++ b/doc/content/toolstack/features/MVD/index.md @@ -0,0 +1,347 @@ +'++ +title = "Multi-version drivers" ++++ + +Linux loads device drivers on boot and every device driver exists in one +version. XAPI extends this scheme such that device drivers may exist in +multiple variants plus a mechanism to select the variant being loaded on +boot. Such a driver is called a multi-version driver and we expect only +a small subset of drivers, built and distributed by XenServer, to have +this property. The following covers the background, API, and CLI for +multi-version drivers in XAPI. + +## Variant vs. Version + +A driver comes in several variants, each of which has a version. A +variant may be updated to a later version while retaining its identity. +This makes variants and versions somewhat synonymous and is admittedly +confusing. + +## Device Drivers in Linux and XAPI + +Drivers that are not compiled into the kernel are loaded dynamically +from the file system. They are loaded from the hierarchy + +* `/lib/modules//` + +and we are particularly interested in the hierarchy + +* `/lib/modules//updates/` + +where vendor-supplied ("driver disk") drivers are located and where we +want to support multiple versions. A driver has typically file extension +`.ko` (kernel object). + +A presence in the file system does not mean that a driver is loaded as +this happens only on demand. The actually loaded drivers +(or modules, in Linux parlance) can be observed from + +* `/proc/modules` + +``` +netlink_diag 16384 0 - Live 0x0000000000000000 +udp_diag 16384 0 - Live 0x0000000000000000 +tcp_diag 16384 0 - Live 0x0000000000000000 +``` + +which includes dependencies between modules (the `-` means no dependencies). + +## Driver Properties + +* A driver name is unique and a driver can be loaded only once. The fact + that kernel object files are located in a file system hierarchy means + that a driver may exist multiple times and in different version in the + file system. From the kernel's perspective a driver has a unique name + and is loaded at most once. We thus can talk about a driver using its + name and acknowledge it may exist in different versions in the file + system. + +* A driver that is loaded by the kernel we call *active*. + +* A driver file (`name.ko`) that is in a hierarchy searched by the + kernel is called *selected*. If the kernel needs the driver of that + name, it would load this object file. + +For a driver (`name.ko`) selection and activation are independent +properties: + +* *inactive*, *deselected*: not loaded now and won't be loaded on next + boot. +* *active*, *deselected*: currently loaded but won't be loaded on next + boot. +* *inactive*, *selected*: not loaded now but will be loaded on demand. +* *active*, *selected*: currently loaded and will be loaded on demand + after a reboot. + +For a driver to be selected it needs to be in the hierarchy searched by +the kernel. By removing a driver from the hierarchy it can be +de-selected. This is possible even for drivers that are already loaded. +Hence, activation and selection are independent. + +## Multi-Version Drivers + +To support multi-version drivers, XenServer introduces a new +hierarchy in Dom0. This is mostly technical background because a +lower-level tool deals with this and not XAPI directly. + +* `/lib/modules//updates/` is searched by the kernel for + drivers. +* The hierarchy is expected to contain symbolic links to the file + actually containing the driver: + `/lib/modules//xenserver///.ko` + +The `xenserver` hierarchy provides drivers in several versions. To +select a particular version, we expect a symbolic link from +`updates/.ko` to `//.ko`. At the next boot, +the kernel will search the `updates/` entries and load the linked +driver, which will become active. + +Example filesystem hierarchy: +``` +/lib/ +└── modules + └── 4.19.0+1 -> + ├── updates + │ ├── aacraid.ko + │ ├── bnx2fc.ko -> ../xenserver/bnx2fc/2.12.13/bnx2fc.ko + │ ├── bnx2i.ko + │ ├── cxgb4i.ko + │ ├── cxgb4.ko + │ ├── dell_laptop.ko -> ../xenserver/dell_laptop/1.2.3/dell_laptop.ko + │ ├── e1000e.ko + │ ├── i40e.ko + │ ├── ice.ko -> ../xenserver/intel-ice/1.11.17.1/ice.ko + │ ├── igb.ko + │ ├── smartpqi.ko + │ └── tcm_qla2xxx.ko + └── xenserver + ├── bnx2fc + │ ├── 2.12.13 + │ │ └── bnx2fc.ko + │ └── 2.12.20-dell + │ └── bnx2fc.ko + ├── dell_laptop + │ └── 1.2.3 + │ └── dell_laptop.ko + └── intel-ice + ├── 1.11.17.1 + │ └── ice.ko + └── 1.6.4 + └── ice.ko + +``` + +Selection of a driver is synonymous with creating a symbolic link to the +desired version. + +## Versions + +The version of a driver is encoded in the path to its object file but +not in the name itself: for `xenserver/intel-ice/1.11.17.1/ice.ko` the +driver name is `ice` and only its location hints at the version. + +The kernel does not reveal the location from where it loaded an active +driver. Hence the name is not sufficient to observe the currently active +version. For this, we use [ELF notes]. + +The driver file (`name.ko`) is in ELF linker format and may contain +custom [ELF notes]. These are binary annotations that can be compiled +into the file. The kernel reveals these details for loaded drivers +(i.e., modules) in: + +* `/sys/module//notes/` + +The directory contains files like + +* `/sys/module/xfs/notes/.note.gnu.build-id` + +with a specific name (`.note.xenserver`) for our purpose. Such a file contains +in binary encoding a sequence of records, each containing: + +* A null-terminated name (string) +* A type (integer) +* A desc (see below) + +The format of the description is vendor specific and is used for +a null-terminated string holding the version. The name is fixed to +"XenServer". The exact format is described in [ELF notes]. + +A note with the name "XenServer" and a particular type then has the version +as a null-terminated string the `desc` field. Additional "XenServer" notes +of a different type may be present. + +[ELF notes]: https://www.netbsd.org/docs/kernel/elf-notes.html + +## API + +XAPI has capabilities to inspect and select multi-version drivers. + +The API uses the terminology introduced above: + +* A driver is specific to a host. +* A driver has a unique name; however, for API purposes a driver is + identified by a UUID (on the CLI) and reference (programmatically). +* A driver has multiple variants; each variant has a version. + Programatically, variants are represented as objects (referenced by + UUID and a reference) but this is mostly hidden in the CLI for + convenience. +* A driver variant is active if it is currently used by the kernel + (loaded). +* A driver variant is selected if it will be considered by the kernel + (on next boot or when loading on demand). +* Only one variant can be active, and only one variants can be selected. + +Inspection and selection of drivers is facilitated by a tool +("drivertool") that is called by xapi. Hence, XAPI does not by itself +manipulate the file system that implements driver selection. + +An example interaction with the API through xe: + +``` +[root@lcy2-dt110 log]# xe hostdriver-list uuid=c0fe459d-5f8a-3fb1-3fe5-3c602fafecc0 params=all +uuid ( RO) : c0fe459d-5f8a-3fb1-3fe5-3c602fafecc0 + name ( RO): cisco-fnic + type ( RO): network + description ( RO): cisco-fnic + info ( RO): cisco-fnic + host-uuid ( RO): 6de288e7-0f82-4563-b071-bcdc083b0ffd + active-variant ( RO): + selected-variant ( RO): + variants ( RO): generic/1.2 + variants-dev-status ( RO): generic=beta + variants-uuid ( RO): generic=abf5997b-f2ad-c0ef-b27f-3f8a37bf58a6 + variants-hw-present ( RO): +``` + +Selection of a variant by name (which is unique per driver); this +variant would become active after reboot. + +``` +[root@lcy2-dt110 log]# xe hostdriver-select variant-name=generic uuid=c0fe459d-5f8a-3fb1-3fe5-3c602fafecc0 +[root@lcy2-dt110 log]# xe hostdriver-list uuid=c0fe459d-5f8a-3fb1-3fe5-3c602fafecc0 params=all +uuid ( RO) : c0fe459d-5f8a-3fb1-3fe5-3c602fafecc0 + name ( RO): cisco-fnic + type ( RO): network + description ( RO): cisco-fnic + info ( RO): cisco-fnic + host-uuid ( RO): 6de288e7-0f82-4563-b071-bcdc083b0ffd + active-variant ( RO): + selected-variant ( RO): generic + variants ( RO): generic/1.2 + variants-dev-status ( RO): generic=beta + variants-uuid ( RO): generic=abf5997b-f2ad-c0ef-b27f-3f8a37bf58a6 + variants-hw-present ( RO): +``` + +The variant can be inspected, too, using it's UUID. + +``` +[root@lcy2-dt110 log]# xe hostdriver-variant-list uuid=abf5997b-f2ad-c0ef-b27f-3f8a37bf58a6 +uuid ( RO) : abf5997b-f2ad-c0ef-b27f-3f8a37bf58a6 + name ( RO): generic + version ( RO): 1.2 + status ( RO): beta + active ( RO): false + selected ( RO): true + driver-uuid ( RO): c0fe459d-5f8a-3fb1-3fe5-3c602fafecc0 + driver-name ( RO): cisco-fnic + host-uuid ( RO): 6de288e7-0f82-4563-b071-bcdc083b0ffd + hw-present ( RO): false +``` + +## Class Host_driver + +Class `Host_driver` represents an instance of a multi-version driver on +a host. It references `Driver_variant` objects for the details of the +available and active variants. A variant has a version. + +### Fields + +All fields are read-only and can't be set directly. Be aware that names +in the CLI and the API may differ. + +* `host`: reference to the host where the driver is installed. +* `name`: string; name of the driver without ".ko" extension. +* `variants`: string set; set of variants available on the host for this + driver. The name of each variant of a driver is unique and used in + the CLI for selecting it. +* `selected_varinat`: variant, possibly empty. Variant that is selected, + i.e. the variant of the driver that will be considered by the kernel + when loading the driver the next time. May be null when none is + selected. +* `active_variant`: variant, possibly empty. Variant that is currently + loaded by the kernel. +* `type`, `info`, `description`: strings providing background + information. + +The CLI uses `hostdriver` and a dash instead of an underscore. The CLI +also offers convenience fields. Whenever selected and +active variant are not the same, a reboot is required to activate the +selected driver/variant combination. + +(We are not using `host-driver` in the CLI to avoid the impression that +this is part of a host object.) + +### Methods + +* All method invocations require `Pool_Operator` rights. "The Pool + Operator role manages host- and pool-wide resources, including setting + up storage, creating resource pools and managing patches, high + availability (HA) and workload balancing (WLB)" + +* `select (self, version)`; select `version` of driver `self`. Selecting + the version (a string) of an existing driver. + +* `deselect(self)`: this driver can't be loaded next time the kernel is + looking for a driver. This is a potentially dangerous operation, so it's + protected in the CLI with a `--force` flag. + +* `rescan (host)`: scan the host and update its driver information. + Called on toolstack restart and may be invoked from the CLI for + development. + +## Class `Driver_variant` + +An object of this class represents a variant of a driver on a host, +i.e., it is specific to both. + +* `name`: unique name +* `driver`: what host driver this belongs to +* `version`: string; a driver variant has a version +* `status`: string: development status, like "beta" +* `hardware_present`: boolean, true if the host has the hardware + installed supported by this driver + +The only method available is `select(self)` to select a variant. It has +the same effect as the `select` method on the `Host_driver` class. + +The CLI comes with corresponding `xe hostdriver-variant-*` commands to +list and select a variant. + +``` +[root@lcy2-dt110 log]# xe hostdriver-variant-list uuid=abf5997b-f2ad-c0ef-b27f-3f8a37bf58a6 +uuid ( RO) : abf5997b-f2ad-c0ef-b27f-3f8a37bf58a6 + name ( RO): generic + version ( RO): 1.2 + status ( RO): beta + active ( RO): false + selected ( RO): true + driver-uuid ( RO): c0fe459d-5f8a-3fb1-3fe5-3c602fafecc0 + driver-name ( RO): cisco-fnic + host-uuid ( RO): 6de288e7-0f82-4563-b071-bcdc083b0ffd + hw-present ( RO): false +``` + +### Database + +Each `Host_driver` and `Driver_variant` object is represented in the +database and data is persisted over reboots. This means this data will +be part of data collected in a `xen-bugtool` invocation. + +### Scan and Rescan + +On XAPI start-up, XAPI updates the `Host_driver` objects belonging to the +host to reflect the actual situation. This can be initiated from the +CLI, too, mostly for development. + + diff --git a/ocaml/idl/datamodel.ml b/ocaml/idl/datamodel.ml index 83d5d1740c3..6b156dfb5ae 100644 --- a/ocaml/idl/datamodel.ml +++ b/ocaml/idl/datamodel.ml @@ -10441,6 +10441,8 @@ let all_system = ; Datamodel_repository.t ; Datamodel_observer.t ; Datamodel_vm_group.t + ; Datamodel_host_driver.t + ; Datamodel_driver_variant.t ] (* If the relation is one-to-many, the "many" nodes (one edge each) must come before the "one" node (many edges) *) @@ -10522,6 +10524,7 @@ let all_relations = ; ((_network_sriov, "logical_PIF"), (_pif, "sriov_logical_PIF_of")) ; ((_certificate, "host"), (_host, "certificates")) ; ((_vm, "groups"), (_vm_group, "VMs")) + ; ((_driver_variant, "driver"), (_host_driver, "variants")) ] let update_lifecycles = @@ -10677,6 +10680,8 @@ let expose_get_all_messages_for = ; _repository ; _vtpm ; _observer + ; _host_driver + ; _driver_variant ] let no_task_id_for = [_task; (* _alert; *) _event] diff --git a/ocaml/idl/datamodel_common.ml b/ocaml/idl/datamodel_common.ml index 80c5076fef7..0ccbc5a12ac 100644 --- a/ocaml/idl/datamodel_common.ml +++ b/ocaml/idl/datamodel_common.ml @@ -311,6 +311,10 @@ let _repository = "Repository" let _observer = "Observer" +let _host_driver = "Host_driver" + +let _driver_variant = "Driver_variant" + let update_guidances = Enum ( "update_guidances" diff --git a/ocaml/idl/datamodel_driver_variant.ml b/ocaml/idl/datamodel_driver_variant.ml new file mode 100644 index 00000000000..607ac670105 --- /dev/null +++ b/ocaml/idl/datamodel_driver_variant.ml @@ -0,0 +1,56 @@ +(* + Copyright (c) Cloud Software Group, Inc. + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU Lesser General Public License as published + by the Free Software Foundation; version 2.1 only. with the special + exception on linking described in file LICENSE. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Lesser General Public License for more details. + *) + +open Datamodel_types +open Datamodel_common +open Datamodel_roles + +(** This is pure data with no methods *) + +let select = + call ~name:"select" ~in_oss_since:None ~lifecycle:[] + ~doc: + "UNSUPPORTED Select this variant of a driver to become active after \ + reboot or immediately if currently no version is active" + ~params: + [ + ( Ref _driver_variant + , "self" + , "Driver variant to become active (after reboot)." + ) + ] + ~allowed_roles:_R_POOL_ADMIN () + +let t = + create_obj ~in_db:true ~in_oss_since:None ~persist:PersistEverything + ~lifecycle:[] ~name:_driver_variant ~gen_constructor_destructor:false + ~descr:"UNSUPPORTED. Variant of a host driver" ~gen_events:true + ~doccomments:[] ~messages_default_allowed_roles:_R_POOL_ADMIN + ~contents: + [ + uid _driver_variant ~lifecycle:[] + ; field ~lifecycle:[] ~qualifier:DynamicRO ~ty:String "name" + "Name identifying the driver variant within the driver" + ; field ~lifecycle:[] ~qualifier:DynamicRO ~ty:(Ref _host_driver) "driver" + "Driver this variant is a part of" + ; field ~lifecycle:[] ~qualifier:DynamicRO ~ty:String "version" + "Unique versions of this driver variant" + ; field ~lifecycle:[] ~qualifier:DynamicRO ~ty:Bool "hardware_present" + "True if the hardware for this variant is present on the host" + ; field ~lifecycle:[] ~qualifier:DynamicRO ~ty:Float "priority" + "Priority; this needs an explanation how this is ordered" + ; field ~lifecycle:[] ~qualifier:DynamicRO ~ty:String "status" + "Development and release status of this variant, like 'alpha'" + ] + ~messages:[select] () diff --git a/ocaml/idl/datamodel_host.ml b/ocaml/idl/datamodel_host.ml index 78b68a35722..0505807dd67 100644 --- a/ocaml/idl/datamodel_host.ml +++ b/ocaml/idl/datamodel_host.ml @@ -2287,6 +2287,12 @@ let apply_updates = ~allowed_roles:(_R_POOL_OP ++ _R_CLIENT_CERT) () +let rescan_drivers = + call ~name:"rescan_drivers" ~in_oss_since:None ~lifecycle:[] + ~doc:"Scan the host and update its driver information." + ~params:[(Ref _host, "host", "The host to be rescanned")] + ~allowed_roles:_R_POOL_ADMIN () + let copy_primary_host_certs = call ~name:"copy_primary_host_certs" ~in_oss_since:None ~lifecycle:[(Published, "1.307.0", "")] @@ -2490,6 +2496,7 @@ let t = ; emergency_reenable_tls_verification ; cert_distrib_atom ; apply_updates + ; rescan_drivers ; copy_primary_host_certs ; set_https_only ; apply_recommended_guidances diff --git a/ocaml/idl/datamodel_host_driver.ml b/ocaml/idl/datamodel_host_driver.ml new file mode 100644 index 00000000000..e61ca372337 --- /dev/null +++ b/ocaml/idl/datamodel_host_driver.ml @@ -0,0 +1,86 @@ +(* + Copyright (c) Cloud Software Group, Inc. + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU Lesser General Public License as published + by the Free Software Foundation; version 2.1 only. with the special + exception on linking described in file LICENSE. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Lesser General Public License for more details. + *) + +open Datamodel_types +open Datamodel_common +open Datamodel_roles + +(** A Host_driver instance represents a driver on a host that is + installed in multiple versions. At most one version is active; a + different version can be selected to become active after reboot. If + no version is active, selecting a version makes it the active + version immediately. *) + +let select = + call ~name:"select" ~in_oss_since:None ~lifecycle:[] + ~doc: + "UNSUPPORTED. Select a variant of the driver to become active after \ + reboot or immediately if currently no version is active" + ~params: + [ + (Ref _host_driver, "self", "Driver to become active (after reboot).") + ; ( Ref _driver_variant + , "variant" + , "Driver version to become active (after reboot)." + ) + ] + ~allowed_roles:_R_POOL_ADMIN () + +let deselect = + call ~name:"deselect" ~in_oss_since:None ~lifecycle:[] + ~doc: + "UNSUPPORTED. Deselect the currently active variant of this driver after \ + reboot. No action will be taken if no variant is currently active." + ~params: + [(Ref _host_driver, "self", "Driver to become inactive (after reboot).")] + ~allowed_roles:_R_POOL_ADMIN () + +let rescan = + call ~name:"rescan" ~in_oss_since:None ~lifecycle:[] + ~doc: + "UNSUPPORTED. Re-scan a host's drivers and update information about \ + them. This is mostly for trouble shooting." + ~params:[(Ref _host, "host", "Update driver information of this host.")] + ~allowed_roles:_R_POOL_ADMIN () + +let t = + create_obj ~in_db:true ~in_oss_since:None ~persist:PersistEverything + ~lifecycle:[] ~name:_host_driver ~gen_constructor_destructor:false + ~descr:"UNSUPPORTED. A multi-version driver on a host" ~gen_events:true + ~doccomments:[] ~messages_default_allowed_roles:_R_POOL_ADMIN + ~contents: + [ + uid _host_driver ~lifecycle:[] + ; field ~lifecycle:[] ~qualifier:StaticRO ~ty:(Ref _host) "host" + "Host where this driver is installed" ~default_value:(Some (VRef "")) + ; field ~lifecycle:[] ~qualifier:DynamicRO ~ty:String "name" + "Name identifying the driver uniquely" + ; field ~lifecycle:[] ~qualifier:DynamicRO ~ty:String "friendly_name" + "Descriptive name, not used for identification" + ; field ~lifecycle:[] ~qualifier:DynamicRO ~ty:(Set (Ref _driver_variant)) + "variants" "Variants of this driver available for selection" + ; field ~lifecycle:[] ~qualifier:DynamicRO ~ty:(Ref _driver_variant) + "active_variant" + "Currently active variant of this driver, if any, or Null." + ; field ~lifecycle:[] ~qualifier:DynamicRO ~ty:(Ref _driver_variant) + "selected_variant" + "Variant (if any) selected to become active after reboot. Or Null" + ; field ~lifecycle:[] ~qualifier:DynamicRO ~ty:String "type" + "Devive type this driver supports, like network or storage" + ; field ~lifecycle:[] ~qualifier:DynamicRO ~ty:String "description" + "Description of the driver" + ; field ~lifecycle:[] ~qualifier:DynamicRO ~ty:String "info" + "Information about the driver" + ] + ~messages:[select; deselect; rescan] () diff --git a/ocaml/idl/datamodel_lifecycle.ml b/ocaml/idl/datamodel_lifecycle.ml index 9e3007f4744..cc252d2cf55 100644 --- a/ocaml/idl/datamodel_lifecycle.ml +++ b/ocaml/idl/datamodel_lifecycle.ml @@ -1,4 +1,8 @@ let prototyped_of_class = function + | "Driver_variant" -> + Some "24.36.0" + | "Host_driver" -> + Some "24.35.0" | "VM_group" -> Some "24.19.1" | "Observer" -> @@ -9,6 +13,40 @@ let prototyped_of_class = function None let prototyped_of_field = function + | "Driver_variant", "status" -> + Some "24.36.0" + | "Driver_variant", "priority" -> + Some "24.36.0" + | "Driver_variant", "hardware_present" -> + Some "24.36.0" + | "Driver_variant", "version" -> + Some "24.36.0" + | "Driver_variant", "driver" -> + Some "24.36.0" + | "Driver_variant", "name" -> + Some "24.36.0" + | "Driver_variant", "uuid" -> + Some "24.36.0" + | "Host_driver", "info" -> + Some "24.39.0" + | "Host_driver", "description" -> + Some "24.39.0" + | "Host_driver", "type" -> + Some "24.39.0" + | "Host_driver", "selected_variant" -> + Some "24.36.0" + | "Host_driver", "active_variant" -> + Some "24.36.0" + | "Host_driver", "variants" -> + Some "24.36.0" + | "Host_driver", "friendly_name" -> + Some "24.39.0" + | "Host_driver", "name" -> + Some "24.35.0" + | "Host_driver", "host" -> + Some "24.35.0" + | "Host_driver", "uuid" -> + Some "24.35.0" | "VM_group", "VMs" -> Some "24.19.1" | "VM_group", "placement" -> @@ -117,6 +155,14 @@ let prototyped_of_field = function None let prototyped_of_message = function + | "Driver_variant", "select" -> + Some "24.39.0" + | "Host_driver", "rescan" -> + Some "24.39.0-next" + | "Host_driver", "deselect" -> + Some "24.35.0" + | "Host_driver", "select" -> + Some "24.35.0" | "Observer", "set_components" -> Some "23.14.0" | "Observer", "set_endpoints" -> @@ -159,6 +205,8 @@ let prototyped_of_message = function Some "23.18.0" | "host", "set_https_only" -> Some "22.27.0" + | "host", "rescan_drivers" -> + Some "24.35.0" | "host", "set_numa_affinity_policy" -> Some "24.0.0" | "VM", "get_secureboot_readiness" -> diff --git a/ocaml/idl/dune b/ocaml/idl/dune index 84ad1c35a93..9530ec3f6e9 100644 --- a/ocaml/idl/dune +++ b/ocaml/idl/dune @@ -6,7 +6,8 @@ datamodel_pool datamodel_cluster datamodel_cluster_host dm_api escaping datamodel_values datamodel_schema datamodel_certificate datamodel_diagnostics datamodel_repository datamodel_lifecycle - datamodel_vtpm datamodel_observer datamodel_vm_group api_version) + datamodel_vtpm datamodel_observer datamodel_vm_group api_version + datamodel_host_driver datamodel_driver_variant) (libraries rpclib.core sexplib0 @@ -80,7 +81,7 @@ (public_name gen_lifecycle) (package xapi-datamodel) (modules gen_lifecycle) - (libraries + (libraries xapi-datamodel xapi-consts.xapi_version ) diff --git a/ocaml/idl/schematest.ml b/ocaml/idl/schematest.ml index 2c4a87453ba..a1d89e5cbfd 100644 --- a/ocaml/idl/schematest.ml +++ b/ocaml/idl/schematest.ml @@ -3,7 +3,7 @@ let hash x = Digest.string x |> Digest.to_hex (* BEWARE: if this changes, check that schema has been bumped accordingly in ocaml/idl/datamodel_common.ml, usually schema_minor_vsn *) -let last_known_schema_hash = "18df8c33434e3df1982e11ec55d1f3f8" +let last_known_schema_hash = "863c257bad0d20800297cf968c294e4e" let current_schema_hash : string = let open Datamodel_types in diff --git a/ocaml/libs/uuid/uuidx.ml b/ocaml/libs/uuid/uuidx.ml index 7bcb74aae04..b22c22ebd14 100644 --- a/ocaml/libs/uuid/uuidx.ml +++ b/ocaml/libs/uuid/uuidx.ml @@ -30,6 +30,8 @@ type without_secret = | `Generic | `GPU_group | `host + | `Host_driver + | `Driver_variant | `host_cpu | `host_crashdump | `host_metrics diff --git a/ocaml/libs/uuid/uuidx.mli b/ocaml/libs/uuid/uuidx.mli index 8561a975cc1..bd0865cf628 100644 --- a/ocaml/libs/uuid/uuidx.mli +++ b/ocaml/libs/uuid/uuidx.mli @@ -41,6 +41,8 @@ type without_secret = | `Generic | `GPU_group | `host + | `Host_driver + | `Driver_variant | `host_cpu | `host_crashdump | `host_metrics diff --git a/ocaml/tests/dune b/ocaml/tests/dune index b51bbca8b80..fbdc42b66fc 100644 --- a/ocaml/tests/dune +++ b/ocaml/tests/dune @@ -7,7 +7,7 @@ test_cluster_host test_cluster test_pusb test_network_sriov test_client test_valid_ref_list suite_alcotest_server test_vm_placement test_vm_helpers test_repository test_repository_helpers - test_ref test_xapi_helpers test_vm_group + test_ref test_xapi_helpers test_vm_group test_host_driver_helpers test_livepatch test_rpm test_updateinfo test_storage_smapiv1_wrapper test_storage_quicktest test_observer test_pool_periodic_update_sync test_pkg_mgr test_tar_ext test_pool_repository)) (libraries @@ -15,7 +15,7 @@ angstrom astring cstruct - + fmt http_lib httpsvr @@ -32,25 +32,26 @@ threads.posix uuid xapi-backtrace - xapi_cli_server xapi-consts - xapi_database xapi-datamodel xapi-idl xapi-idl.storage.interface xapi-idl.xen.interface xapi-idl.xen.interface.types - xapi_internal xapi-log xapi-stdext-date + xapi-stdext-pervasives xapi-stdext-std xapi-stdext-threads xapi-stdext-unix xapi-test-utils xapi-tracing xapi-types - xapi-stdext-pervasives xapi_xenopsd + xapi_cli_server + xapi_database + xapi_host_driver_helpers + xapi_internal xml-light2 ) (deps @@ -81,14 +82,14 @@ (names test_vm_helpers test_vm_placement test_network_sriov test_vdi_cbt test_bounded_psq test_auth_cache test_clustering test_pusb test_daemon_manager test_repository test_repository_helpers test_livepatch test_rpm test_updateinfo test_pool_periodic_update_sync test_pkg_mgr - test_xapi_helpers test_tar_ext test_pool_repository) + test_xapi_helpers test_tar_ext test_pool_repository test_host_driver_helpers) (package xapi) (modes exe) (modules test_vm_helpers test_vm_placement test_network_sriov test_vdi_cbt test_bounded_psq test_auth_cache test_event test_clustering test_cluster_host test_cluster test_pusb test_daemon_manager test_repository test_repository_helpers test_livepatch test_rpm test_updateinfo test_pool_periodic_update_sync test_pkg_mgr - test_xapi_helpers test_tar_ext test_pool_repository) + test_xapi_helpers test_tar_ext test_pool_repository test_host_driver_helpers) (libraries alcotest bos @@ -104,21 +105,22 @@ threads.posix uuid xapi-client - xapi_cli_server xapi-consts - xapi_database xapi-idl xapi-idl.cluster xapi-idl.storage xapi-idl.storage.interface xapi-idl.xen - xapi_internal - xapi-test-utils - xapi-tracing - xapi-types xapi-stdext-date xapi-stdext-threads xapi-stdext-unix + xapi-test-utils + xapi-tracing + xapi-types + xapi_cli_server + xapi_database + xapi_host_driver_helpers + xapi_internal xml-light2 yojson ) @@ -167,6 +169,27 @@ (action (run ./check-no-xenctrl %{x})) ) +(rule + (alias runtest) + (package xapi) + (targets + .note.XenServer + .note.Linux + .note.gnu.build-id + .note.XenServerTwo + ) + (deps + (:asm + test_data/xenserver.s + test_data/xenserver_two_notes.s + test_data/linux.s + test_data/buildid.s + ) + (:script test_data/gen_notes.sh) + ) + (action (bash "%{script} %{asm}")) +) + (env (_ (env-vars (XAPI_TEST 1)))) ; disassemble, but without sources diff --git a/ocaml/tests/test_data/buildid.s b/ocaml/tests/test_data/buildid.s new file mode 100644 index 00000000000..75f77766980 --- /dev/null +++ b/ocaml/tests/test_data/buildid.s @@ -0,0 +1,9 @@ +.section ".note.gnu.build-id", "a" + .p2align 2 + .long 1f - 0f # name size (not including padding) + .long 3f - 2f # desc size (not including padding) + .long 0x1 # type +0: .asciz "gnu.build-id" # name +1: .p2align 2 +2: .long 0x000000 # desc +3: .p2align 2 diff --git a/ocaml/tests/test_data/gen_notes.sh b/ocaml/tests/test_data/gen_notes.sh new file mode 100755 index 00000000000..9b173bd31da --- /dev/null +++ b/ocaml/tests/test_data/gen_notes.sh @@ -0,0 +1,22 @@ +#!/bin/bash +# +# Copyright (c) Cloud Software Group, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU Lesser General Public License as published +# by the Free Software Foundation; version 2.1 only. with the special +# exception on linking described in file LICENSE. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Lesser General Public License for more details. + +elf_file=test_data/xenserver_elf_file +as "$@" -o $elf_file + +sections=$(readelf -n $elf_file | grep -Po "(?<=Displaying notes found in: ).*") +for dep in $sections; do + objcopy "$elf_file" "$dep" --only-section="$dep" -O binary +done + diff --git a/ocaml/tests/test_data/linux.s b/ocaml/tests/test_data/linux.s new file mode 100644 index 00000000000..ca106e94af7 --- /dev/null +++ b/ocaml/tests/test_data/linux.s @@ -0,0 +1,9 @@ +.section ".note.Linux", "a" + .p2align 2 + .long 1f - 0f # name size (not including padding) + .long 3f - 2f # desc size (not including padding) + .long 0x257 # type +0: .asciz "Linux" # name +1: .p2align 2 +2: .asciz "4.19.0+1" # desc +3: .p2align 2 diff --git a/ocaml/tests/test_data/xenserver.s b/ocaml/tests/test_data/xenserver.s new file mode 100644 index 00000000000..f44575ce5eb --- /dev/null +++ b/ocaml/tests/test_data/xenserver.s @@ -0,0 +1,9 @@ +.section ".note.XenServer", "a" + .p2align 2 + .long 1f - 0f # name size (not including padding) + .long 3f - 2f # desc size (not including padding) + .long 0x1 # type +0: .asciz "XenServer" # name +1: .p2align 2 +2: .asciz "v2.1.3+0.1fix" # desc +3: .p2align 2 diff --git a/ocaml/tests/test_data/xenserver_two_notes.s b/ocaml/tests/test_data/xenserver_two_notes.s new file mode 100644 index 00000000000..cbde4916dd5 --- /dev/null +++ b/ocaml/tests/test_data/xenserver_two_notes.s @@ -0,0 +1,20 @@ +.section ".note.XenServerTwo", "a" + .p2align 2 + .long 1f - 0f # name size (not including padding) + .long 3f - 2f # desc size (not including padding) + .long 0x2 # type +0: .asciz "XenServer" # name +1: .p2align 2 +2: .asciz "Built on December 25th" # desc +3: .p2align 2 + +.section ".note.XenServerTwo", "a" + .p2align 2 + .long 1f - 0f # name size (not including padding) + .long 3f - 2f # desc size (not including padding) + .long 0x1 # type +0: .asciz "XenServer" # name +1: .p2align 2 +2: .asciz "2.0.0-rc.2" # desc +3: .p2align 2 + diff --git a/ocaml/tests/test_host_driver_helpers.ml b/ocaml/tests/test_host_driver_helpers.ml new file mode 100644 index 00000000000..bb1a49050b1 --- /dev/null +++ b/ocaml/tests/test_host_driver_helpers.ml @@ -0,0 +1,89 @@ +(* + Copyright (c) Cloud Software Group, Inc. + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU Lesser General Public License as published + by the Free Software Foundation; version 2.1 only. with the special + exception on linking described in file LICENSE. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Lesser General Public License for more details. + *) + +open Xapi_host_driver_helpers + +let note = + Alcotest.testable + (Fmt.of_to_string (fun n -> + Printf.sprintf "{typ=%d; name=%s; desc=%s}" (Int32.to_int n.typ) n.name + n.desc + ) + ) + ( = ) + +let versions = + [ + (".note.XenServer", Some "v2.1.3+0.1fix") + ; (".note.XenServerTwo", Some "2.0.0-rc.2") + ; (".note.Linux", None) + ; (".note.gnu.build-id", None) + ] + +let get_version_test = + List.map + (fun (filename, expected) -> + let test_version () = + let parsed_ver = Result.to_option (get_version filename) in + Printf.printf "%s\n" filename ; + Alcotest.(check (option string)) + "ELF notes should be parsed properly" expected parsed_ver + in + ( Printf.sprintf {|Validation of ELF note parsing: "%s"|} filename + , `Quick + , test_version + ) + ) + versions + +let notes = + [ + (".note.XenServer", [{typ= 1l; name= "XenServer"; desc= "v2.1.3+0.1fix"}]) + ; ( ".note.XenServerTwo" + , [ + {typ= 2l; name= "XenServer"; desc= "Built on December 25th"} + ; {typ= 1l; name= "XenServer"; desc= "2.0.0-rc.2"} + ] + ) + ; (".note.Linux", [{typ= 599l; name= "Linux"; desc= "4.19.0+1"}]) + ; ( ".note.gnu.build-id" + , [{typ= 1l; name= "gnu.build-id"; desc= "\x00\x00\x00"}] + ) + ] + +let note_parsing_test = + List.map + (fun (filename, expected) -> + let test_note () = + let parsed = + match get_notes filename with Ok res -> res | Error e -> failwith e + in + Printf.printf "%s\n" filename ; + Alcotest.(check (list note)) + "ELF notes should be parsed properly" expected parsed + in + ( Printf.sprintf {|Validation of ELF note parsing: "%s"|} filename + , `Quick + , test_note + ) + ) + notes + +let () = + Suite_init.harness_init () ; + Alcotest.run "Test Host Driver Helpers suite" + [ + ("Test_host_driver_helpers.get_note", note_parsing_test) + ; ("Test_host_driver_helpers.get_version", get_version_test) + ] diff --git a/ocaml/tests/test_host_driver_helpers.mli b/ocaml/tests/test_host_driver_helpers.mli new file mode 100644 index 00000000000..e69de29bb2d diff --git a/ocaml/util/dune b/ocaml/util/dune index 7a21f9bb24b..d9380eecbc9 100644 --- a/ocaml/util/dune +++ b/ocaml/util/dune @@ -17,3 +17,9 @@ (wrapped false) ) +(library + (name xapi_host_driver_helpers) + (modules xapi_host_driver_helpers) + (libraries yojson angstrom) + (wrapped false) +) diff --git a/ocaml/util/xapi_host_driver_helpers.ml b/ocaml/util/xapi_host_driver_helpers.ml new file mode 100644 index 00000000000..61ce6b8e537 --- /dev/null +++ b/ocaml/util/xapi_host_driver_helpers.ml @@ -0,0 +1,137 @@ +(* + Copyright (c) Cloud Software Group, Inc. + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU Lesser General Public License as published + by the Free Software Foundation; version 2.1 only. with the special + exception on linking described in file LICENSE. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Lesser General Public License for more details. + *) + +module J = Yojson +open Angstrom + +let defer finally = Fun.protect ~finally + +let int n = Int32.to_int n + +let ( // ) = Filename.concat + +(** Read a (small) file into a string *) +let read path = + let ic = open_in path in + defer (fun () -> close_in ic) @@ fun () -> + let size = in_channel_length ic in + really_input_string ic size + +type note = {typ: int32; name: string; desc: string} + +module JSON = struct + let note l = + let l = + List.map + (fun d -> + `Assoc + [ + ("type", `Int (int d.typ)) + ; ("name", `String d.name) + ; ("desc", `String d.desc) + ] + ) + l + in + `List l + + let emit json = J.pretty_to_channel stdout json +end + +(** return the smallest k >= n such that k is divisible by 4 *) +let align4 n = + let ( & ) = Int.logand in + n + (-n & 3) + +(** advance the cursor to position n *) +let advance_to n = + let* pos in + advance (max 0 (n - pos)) + +(** align the cursor to a multiple of 4 *) +let align = + let* pos in + advance_to (align4 pos) + +(** parse an ELF note entry; it assumes that name and desc are null + terminated strings. This should be always true for name but desc + depends on the entry. We don't capture the terminating zero for + strings. *) +let note = + let* name_length = LE.any_int32 in + let* desc_length = LE.any_int32 in + let* typ = LE.any_int32 in + let* name = take (int name_length - 1) in + (* skip over terminating null and re-align cursor *) + let* _ = char '\000' in + let* () = align in + let* desc = take (int desc_length - 1) in + (* skip over terminating null and re-align cursor *) + let* _ = char '\000' in + let* () = align in + return {typ; name; desc} + +(** parser for a sequence of note entries *) +let notes = many note + +(** parse a sequence of note entries from a string *) +let parse str = + let consume = Consume.Prefix in + parse_string ~consume notes str + +let get_version path = + let version = + read path + |> parse + |> Result.map + @@ List.filter_map (fun note -> + match (note.typ, note.name) with + | 1l, "XenServer" -> + Some note.desc + | _ -> + None + ) + in + match version with + | Ok (v :: _) -> + Ok v + | _ -> + Error + (Format.sprintf + "Failed to parse %s, didn't find a XenServer driver version notes \ + section" + path + ) + +let get_notes path = + let version = read path |> parse in + match version with + | Ok (_ :: _) as v -> + v + | _ -> + Error + (Format.sprintf "Failed to parse %s, didn't find a notes section" path) + +let dump_notes prefix = + let notes_dir = prefix // "notes" in + try + let lst = + Sys.readdir notes_dir + |> Array.to_list + |> List.map (fun n -> read (notes_dir // n)) + |> List.filter_map (fun note_str -> Result.to_option (parse note_str)) + |> List.map (fun note -> (prefix, JSON.note note)) + in + JSON.emit (`Assoc lst) + with _ -> () diff --git a/ocaml/util/xapi_host_driver_helpers.mli b/ocaml/util/xapi_host_driver_helpers.mli new file mode 100644 index 00000000000..6528d6bec94 --- /dev/null +++ b/ocaml/util/xapi_host_driver_helpers.mli @@ -0,0 +1,28 @@ +(* + Copyright (c) Cloud Software Group, Inc. + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU Lesser General Public License as published + by the Free Software Foundation; version 2.1 only. with the special + exception on linking described in file LICENSE. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Lesser General Public License for more details. + *) + +type note = {typ: int32; name: string; desc: string} + +(* Parse an ELF notes section, returning the specially-encoded driver version. + + The kernel does not reveal the location from where it loaded an active + driver. Hence the name is not sufficient to observe the currently active + version. For this, XS uses ELF notes, with the kernel presenting a particular + note section in `/sys/module//notes/.note.XenServer` *) +val get_version : string -> (string, string) result + +val get_notes : string -> (note list, string) result + +(* Dumps JSON-formatted parsed ELF notes of a driver *) +val dump_notes : string -> unit diff --git a/ocaml/xapi-cli-server/cli_frontend.ml b/ocaml/xapi-cli-server/cli_frontend.ml index 3de231f3cad..05c9890ff14 100644 --- a/ocaml/xapi-cli-server/cli_frontend.ml +++ b/ocaml/xapi-cli-server/cli_frontend.ml @@ -3705,6 +3705,54 @@ let rec cmdtable_data : (string * cmd_spec) list = ; flags= [] } ) + ; ( "hostdriver-select" + , { + reqd= ["uuid"; "variant-name"] + ; optn= [] + ; help= + "Select a variant of the specified driver. If no variant of the \ + driver is loaded, this variant will become active. If some other \ + variant is already active, a reboot will be required to activate \ + the new variant." + ; implementation= No_fd Cli_operations.Host_driver.select + ; flags= [] + } + ) + ; ( "hostdriver-deselect" + , { + reqd= ["uuid"] + ; optn= ["force"] + ; help= + "Deselect the currently active variant of the specified driver after \ + reboot. No action will be taken if no version is currently active. \ + Potentially dangerous operation, needs the '--force' flag \ + specified." + ; implementation= No_fd Cli_operations.Host_driver.deselect + ; flags= [] + } + ) + ; ( "hostdriver-variant-select" + , { + reqd= ["uuid"] + ; optn= [] + ; help= + "Select a variant of a driver. If no variant of the driver is \ + loaded, this variant will become active. If some other variant is \ + already active, a reboot will be required to activate the new \ + variant." + ; implementation= No_fd Cli_operations.Driver_variant.select + ; flags= [] + } + ) + ; ( "hostdriver-rescan" + , { + reqd= [] + ; optn= [] + ; help= "Scan a host and update its driver information." + ; implementation= No_fd Cli_operations.Host_driver.rescan + ; flags= [Host_selectors] + } + ) ; ( "vtpm-create" , { reqd= ["vm-uuid"] diff --git a/ocaml/xapi-cli-server/cli_operations.ml b/ocaml/xapi-cli-server/cli_operations.ml index 1e8ba0f3b37..b725890014d 100644 --- a/ocaml/xapi-cli-server/cli_operations.ml +++ b/ocaml/xapi-cli-server/cli_operations.ml @@ -29,6 +29,8 @@ open Records let failwith str = raise (Cli_util.Cli_failure str) +let failwith' fmt = Printf.ksprintf failwith fmt + exception ExitWithError of int let bool_of_string param string = @@ -1322,6 +1324,37 @@ let gen_cmds rpc session_id = ] rpc session_id ) + ; Client.Driver_variant.( + mk get_all_records_where get_by_uuid driver_variant_record + "hostdriver-variant" [] + [ + "uuid" + ; "driver-name" + ; "name" + ; "version" + ; "priotory" + ; "host-uuid" + ; "driver-uuid" + ; "active" + ; "selected" + ; "status" + ; "hw-present" + ] + rpc session_id + ) + ; Client.Host_driver.( + mk get_all_records_where get_by_uuid host_driver_record "hostdriver" [] + [ + "uuid" + ; "name" + ; "host-uuid" + ; "active-variant" + ; "selected-variant" + ; "variants" + ; "variants-uuid" + ] + rpc session_id + ) ; Client.VTPM.( mk get_all_records_where get_by_uuid vtpm_record "vtpm" [] ["uuid"; "vm-uuid"; "profile"] @@ -7954,6 +7987,52 @@ module Repository = struct ~value:gpgkey_path end +module Driver_variant = struct + let select _ rpc session_id params = + let uuid = List.assoc "uuid" params in + let self = Client.Driver_variant.get_by_uuid ~rpc ~session_id ~uuid in + Client.Driver_variant.select ~rpc ~session_id ~self +end + +module Host_driver = struct + let select _ rpc session_id params = + let driver_uuid = List.assoc "uuid" params in + let name = List.assoc "variant-name" params in + let driver = + Client.Host_driver.get_by_uuid ~rpc ~session_id ~uuid:driver_uuid + in + let by_name (_, variant) = variant.API.driver_variant_name = name in + let variants = + List.map + (fun self -> + (self, Client.Driver_variant.get_record ~rpc ~session_id ~self) + ) + (Client.Host_driver.get_variants ~rpc ~session_id ~self:driver) + in + + match List.find_opt by_name variants with + | None -> + failwith' "%s does not identify a variant of this driver" name + | Some (variant, _) -> + Client.Host_driver.select ~rpc ~session_id ~self:driver ~variant + + let deselect _ rpc session_id params = + fail_without_force params ; + let uuid = List.assoc "uuid" params in + let self = Client.Host_driver.get_by_uuid ~rpc ~session_id ~uuid in + Client.Host_driver.deselect ~rpc ~session_id ~self + + let rescan _printer rpc session_id params = + ignore + (do_host_op rpc session_id ~multiple:false + (fun _ host -> + let host = host.getref () in + Client.Host_driver.rescan ~rpc ~session_id ~host + ) + params [] + ) +end + module VTPM = struct let create printer rpc session_id params = let vm_uuid = List.assoc "vm-uuid" params in diff --git a/ocaml/xapi-cli-server/records.ml b/ocaml/xapi-cli-server/records.ml index cd7e2f5ae80..b2ca1d1d5fd 100644 --- a/ocaml/xapi-cli-server/records.ml +++ b/ocaml/xapi-cli-server/records.ml @@ -5290,6 +5290,217 @@ let repository_record rpc session_id repository = ] } +let driver_variant_record rpc session_id variant = + let _ref = ref variant in + let empty = + ToGet + (fun () -> Client.Driver_variant.get_record ~rpc ~session_id ~self:!_ref) + in + let record = ref empty in + let x () = lzy_get record in + + let empty_driver = + ToGet + (fun () -> + Client.Host_driver.get_record ~rpc ~session_id + ~self:(x ()).API.driver_variant_driver + ) + in + + let driver = ref empty_driver in + let xd () = lzy_get driver in + + { + setref= + (fun r -> + _ref := r ; + record := empty + ) + ; setrefrec= + (fun (a, b) -> + _ref := a ; + record := Got b + ) + ; record= x + ; getref= (fun () -> !_ref) + ; fields= + [ + make_field ~name:"uuid" + ~get:(fun () -> (x ()).API.driver_variant_uuid) + () + ; make_field ~name:"name" + ~get:(fun () -> (x ()).API.driver_variant_name) + () + ; make_field ~name:"version" + ~get:(fun () -> (x ()).API.driver_variant_version) + () + ; make_field ~name:"status" + ~get:(fun () -> (x ()).API.driver_variant_status) + () + ; make_field ~name:"priority" + ~get:(fun () -> + Printf.sprintf "%5.1f" (x ()).API.driver_variant_priority + ) + () + ; make_field ~name:"active" + ~get:(fun () -> + string_of_bool ((xd ()).API.host_driver_active_variant = !_ref) + ) + () + ; make_field ~name:"selected" + ~get:(fun () -> + string_of_bool ((xd ()).API.host_driver_selected_variant = !_ref) + ) + () + ; make_field ~name:"driver-uuid" + ~get:(fun () -> get_uuid_from_ref (x ()).API.driver_variant_driver) + () + ; make_field ~name:"driver-name" + ~get:(fun () -> (xd ()).API.host_driver_name) + () + ; make_field ~name:"host-uuid" + ~get:(fun () -> + try get_uuid_from_ref (xd ()).API.host_driver_host with _ -> nid + ) + () + ; make_field ~name:"hw-present" + ~get:(fun () -> + string_of_bool (x ()).API.driver_variant_hardware_present + ) + () + ] + } + +let host_driver_record rpc session_id host_driver = + let _ref = ref host_driver in + let none = "" in + let empty = + ToGet (fun () -> Client.Host_driver.get_record ~rpc ~session_id ~self:!_ref) + in + let record = ref empty in + let x () = lzy_get record in + + (* variants of this driver in priority order; should be a short list *) + let variants = + ToGet + (fun () -> + List.map + (fun self -> + (self, Client.Driver_variant.get_record ~rpc ~session_id ~self) + ) + (Client.Host_driver.get_variants ~rpc ~session_id ~self:host_driver) + |> List.stable_sort (fun (_, x) (_, y) -> + Float.compare x.API.driver_variant_priority + y.API.driver_variant_priority + |> Int.neg + ) + ) + in + + let variants = ref variants in + let xv () = lzy_get variants in + + { + setref= + (fun r -> + _ref := r ; + record := empty + ) + ; setrefrec= + (fun (a, b) -> + _ref := a ; + record := Got b + ) + ; record= x + ; getref= (fun () -> !_ref) + ; fields= + [ + make_field ~name:"uuid" ~get:(fun () -> (x ()).API.host_driver_uuid) () + ; make_field ~name:"name" ~get:(fun () -> (x ()).API.host_driver_name) () + ; make_field ~name:"type" ~get:(fun () -> (x ()).API.host_driver_type) () + ; make_field ~name:"description" + ~get:(fun () -> (x ()).API.host_driver_description) + () + ; make_field ~name:"info" ~get:(fun () -> (x ()).API.host_driver_info) () + ; make_field ~name:"host-uuid" + ~get:(fun () -> + try get_uuid_from_ref (x ()).API.host_driver_host with _ -> nid + ) + () + ; make_field ~name:"active-variant" + ~get:(fun () -> + let r = (x ()).API.host_driver_active_variant in + match List.find_opt (fun (r', _) -> r = r') (xv ()) with + | None -> + none + | Some (_, v) -> + v.API.driver_variant_name + ) + () + ; make_field ~name:"selected-variant" + ~get:(fun () -> + let r = (x ()).API.host_driver_selected_variant in + match List.find_opt (fun (r', _) -> r = r') (xv ()) with + | None -> + none + | Some (_, v) -> + v.API.driver_variant_name + ) + () + ; make_field ~name:"variants" + ~get:(fun () -> + xv () + |> List.map (fun (_, v) -> + (v.API.driver_variant_name, v.API.driver_variant_version) + ) + |> List.map (fun (name, version) -> + Printf.sprintf "%s/%s" name version + ) + |> String.concat "; " + ) + () + ; make_field ~name:"variants-dev-status" + ~get:(fun () -> + xv () + |> List.map (fun (_, v) -> + ( v.API.driver_variant_name + , v.API.driver_variant_version + , v.API.driver_variant_status + ) + ) + |> List.map (fun (name, _version, status) -> + Printf.sprintf "%s=%s" name status + ) + |> String.concat "; " + ) + () + ; make_field ~name:"variants-uuid" + ~get:(fun () -> + xv () + |> List.map (fun (_, v) -> + (v.API.driver_variant_name, v.API.driver_variant_uuid) + ) + |> List.map (fun (name, uuid) -> Printf.sprintf "%s=%s" name uuid) + |> String.concat "; " + ) + () + ; make_field ~name:"variants-hw-present" + ~get:(fun () -> + xv () + |> List.map (fun (_, v) -> + ( v.API.driver_variant_name + , v.API.driver_variant_version + , v.API.driver_variant_hardware_present + ) + ) + |> List.filter (fun (_, _, status) -> status = true) + |> List.map (fun (name, _, _) -> name) + |> String.concat "; " + ) + () + ] + } + let vtpm_record rpc session_id vtpm = let _ref = ref vtpm in let empty_record = diff --git a/ocaml/xapi/api_server_common.ml b/ocaml/xapi/api_server_common.ml index 1cd1758a078..f4167c1f36a 100644 --- a/ocaml/xapi/api_server_common.ml +++ b/ocaml/xapi/api_server_common.ml @@ -130,6 +130,8 @@ module Actions = struct module Diagnostics = Xapi_diagnostics module Repository = Repository module Observer = Xapi_observer + module Host_driver = Xapi_host_driver + module Driver_variant = Xapi_host_driver.Variant end (** Use the server functor to make an XML-RPC dispatcher. *) diff --git a/ocaml/xapi/db_gc_util.ml b/ocaml/xapi/db_gc_util.ml index 7972aa28ed9..202b51cc5eb 100644 --- a/ocaml/xapi/db_gc_util.ml +++ b/ocaml/xapi/db_gc_util.ml @@ -71,6 +71,27 @@ let gc_VDIs ~__context = ) (Db.VDI.get_all ~__context) +let gc_Host_drivers ~__context = + let all_host_drivers = Db.Host_driver.get_all ~__context in + List.iter + (fun self -> + if not (valid_ref __context (Db.Host_driver.get_host ~__context ~self)) + then + Db.Host_driver.destroy ~__context ~self + ) + all_host_drivers + +let gc_Host_driver_variants ~__context = + let variants = Db.Driver_variant.get_all ~__context in + List.iter + (fun self -> + if + not (valid_ref __context (Db.Driver_variant.get_driver ~__context ~self)) + then + Db.Driver_variant.destroy ~__context ~self + ) + variants + let gc_PIFs ~__context = gc_connector ~__context Db.PIF.get_all Db.PIF.get_record (fun x -> valid_ref __context x.pIF_host) @@ -581,17 +602,6 @@ let gc_PVS_cache_storage ~__context = (fun x -> valid_ref __context x.pVS_cache_storage_host) Db.PVS_cache_storage.destroy -(* -let timeout_alerts ~__context = - let all_alerts = Db.Alert.get_all ~__context in - let now = Unix.gettimeofday() in - List.iter (fun alert -> - let alert_time = Date.to_unix_time (Db.Alert.get_timestamp ~__context ~self:alert) in - if now -. alert_time > Xapi_globs.alert_timeout then - Db.Alert.destroy ~__context ~self:alert - ) all_alerts -*) - let gc_updates_requiring_reboot ~__context = List.iter (fun host -> @@ -632,9 +642,9 @@ let gc_subtask_list = ; ("PVS servers", gc_PVS_servers) ; ("PVS cache storage", gc_PVS_cache_storage) ; ("Certificates", gc_certificates) - ; ("VTPMs", gc_vtpms) - ; (* timeout_alerts; *) - (* CA-29253: wake up all blocked clients *) - ("Heartbeat", Xapi_event.heartbeat) + ; ("VTPMs", gc_vtpms) (* CA-29253: wake up all blocked clients *) + ; ("Heartbeat", Xapi_event.heartbeat) ; ("Updates requiring reboot", gc_updates_requiring_reboot) + ; ("Host drivers", gc_Host_drivers) + ; ("Host driver variants", gc_Host_driver_variants) ] diff --git a/ocaml/xapi/dbsync_slave.ml b/ocaml/xapi/dbsync_slave.ml index 3ff89881de3..1c1b5f1f241 100644 --- a/ocaml/xapi/dbsync_slave.ml +++ b/ocaml/xapi/dbsync_slave.ml @@ -353,6 +353,10 @@ let update_env __context sync_keys = ~value:current_bios_strings ) ) ; + switched_sync Xapi_globs.sync_host_driver (fun () -> + debug "%s" __FUNCTION__ ; + ignore (Xapi_host.rescan_drivers ~__context ~host:localhost) + ) ; (* CA-35549: In a pool rolling upgrade, the master will detect the end of upgrade when the software versions of all the hosts are the same. It will then assume that (for example) per-host patch records have diff --git a/ocaml/xapi/dune b/ocaml/xapi/dune index d2e2fb17de8..22b5376b8d6 100644 --- a/ocaml/xapi/dune +++ b/ocaml/xapi/dune @@ -209,6 +209,7 @@ xxhash yojson zstd + xapi_host_driver_helpers ) (preprocess (per_module ((pps ppx_sexp_conv) Cert_distrib) diff --git a/ocaml/xapi/message_forwarding.ml b/ocaml/xapi/message_forwarding.ml index 6423e8d7be3..ccaadb85e94 100644 --- a/ocaml/xapi/message_forwarding.ml +++ b/ocaml/xapi/message_forwarding.ml @@ -4143,6 +4143,14 @@ functor info "Host.apply_updates: host = '%s'; hash = '%s'" uuid hash ; Local.Host.apply_updates ~__context ~self ~hash + let rescan_drivers ~__context ~host = + let uuid = host_uuid ~__context host in + info "Host.rescan_drivers: host = '%s'" uuid ; + let local_fn = Local.Host.rescan_drivers ~host in + do_op_on ~local_fn ~__context ~host (fun session_id rpc -> + Client.Host.rescan_drivers ~rpc ~session_id ~host + ) + let set_https_only ~__context ~self ~value = let uuid = host_uuid ~__context self in info "Host.set_https_only: self = %s ; value = %b" uuid value ; @@ -6599,6 +6607,44 @@ functor module Certificate = struct end + module Host_driver = struct + (** select needs to be executed on the host of the driver *) + let select ~__context ~self ~variant = + info "%s" __FUNCTION__ ; + let host = Db.Host_driver.get_host ~__context ~self in + let local_fn = Local.Host_driver.select ~self ~variant in + do_op_on ~__context ~local_fn ~host (fun session_id rpc -> + Client.Host_driver.select ~rpc ~session_id ~self ~variant + ) + + (** deselect needs to be executed on the host of the driver *) + let deselect ~__context ~self = + info "%s" __FUNCTION__ ; + let host = Db.Host_driver.get_host ~__context ~self in + let local_fn = Local.Host_driver.deselect ~self in + do_op_on ~__context ~local_fn ~host (fun session_id rpc -> + Client.Host_driver.deselect ~rpc ~session_id ~self + ) + + let rescan ~__context ~host = + info "%s" __FUNCTION__ ; + let local_fn = Local.Host_driver.rescan ~host in + do_op_on ~__context ~local_fn ~host (fun session_id rpc -> + Client.Host_driver.rescan ~rpc ~session_id ~host + ) + end + + module Driver_variant = struct + let select ~__context ~self = + info "%s" __FUNCTION__ ; + let drv = Db.Driver_variant.get_driver ~__context ~self in + let host = Db.Host_driver.get_host ~__context ~self:drv in + let local_fn = Local.Driver_variant.select ~self in + do_op_on ~__context ~local_fn ~host (fun session_id rpc -> + Client.Driver_variant.select ~rpc ~session_id ~self + ) + end + module Repository = struct let introduce ~__context ~name_label ~name_description ~binary_url ~source_url ~update ~gpgkey_path = diff --git a/ocaml/xapi/xapi_globs.ml b/ocaml/xapi/xapi_globs.ml index efdcabfbdb6..4004bb39c5e 100644 --- a/ocaml/xapi/xapi_globs.ml +++ b/ocaml/xapi/xapi_globs.ml @@ -372,6 +372,8 @@ let sync_pci_devices = "sync_pci_devices" let sync_gpus = "sync_gpus" +let sync_host_driver = "sync_host_driver" + (* Allow dbsync actions to be disabled via the redo log, since the database isn't of much use if xapi won't start. *) let disable_dbsync_for = ref [] @@ -929,6 +931,14 @@ let xen_livepatch_cmd = ref "/usr/sbin/xen-livepatch" let xl_cmd = ref "/usr/sbin/xl" +let depmod = ref "/usr/sbin/depmod" + +let driver_tool = ref "/opt/xensource/debug/drivertool.sh" + +let dracut = ref "/usr/sbin/dracut" + +let udevadm = ref "/usr/sbin/udevadm" + let yum_repos_config_dir = ref "/etc/yum.repos.d" let remote_repository_prefix = ref "remote" @@ -1614,6 +1624,11 @@ let other_options = , (fun () -> string_of_bool !disable_webserver) , "Disable the host webserver" ) + ; ( "drivertool" + , Arg.Set_string driver_tool + , (fun () -> !driver_tool) + , "Path to drivertool for selecting host driver variants" + ) ] (* The options can be set with the variable xapiflags in /etc/sysconfig/xapi. @@ -1716,6 +1731,9 @@ module Resources = struct ; ("createrepo-cmd", createrepo_cmd, "Path to createrepo command") ; ("modifyrepo-cmd", modifyrepo_cmd, "Path to modifyrepo command") ; ("rpm-cmd", rpm_cmd, "Path to rpm command") + ; ("depmod", depmod, "Path to depmod command") + ; ("dracut", dracut, "Path to dracut command") + ; ("udevadm", udevadm, "Path to udevadm command") ] let nonessential_executables = diff --git a/ocaml/xapi/xapi_host.ml b/ocaml/xapi/xapi_host.ml index cd6ae3a7d35..5b37873219e 100644 --- a/ocaml/xapi/xapi_host.ml +++ b/ocaml/xapi/xapi_host.ml @@ -3089,6 +3089,8 @@ let apply_updates ~__context ~self ~hash = Db.Host.set_last_update_hash ~__context ~self ~value:hash ; warnings +let rescan_drivers ~__context ~host = Xapi_host_driver.scan ~__context ~host + let cc_prep () = let cc = "CC_PREPARATIONS" in Xapi_inventory.lookup ~default:"false" cc |> String.lowercase_ascii diff --git a/ocaml/xapi/xapi_host.mli b/ocaml/xapi/xapi_host.mli index f8fe73f8379..74131a59974 100644 --- a/ocaml/xapi/xapi_host.mli +++ b/ocaml/xapi/xapi_host.mli @@ -555,6 +555,8 @@ val get_host_updates_handler : Http.Request.t -> Unix.file_descr -> 'a -> unit val apply_updates : __context:Context.t -> self:API.ref_host -> hash:string -> string list list +val rescan_drivers : __context:Context.t -> host:API.ref_host -> unit + val copy_primary_host_certs : __context:Context.t -> host:API.ref_host -> unit val set_https_only : diff --git a/ocaml/xapi/xapi_host_driver.ml b/ocaml/xapi/xapi_host_driver.ml new file mode 100644 index 00000000000..443593d5a25 --- /dev/null +++ b/ocaml/xapi/xapi_host_driver.ml @@ -0,0 +1,160 @@ +(* + Copyright (c) Cloud Software Group, Inc. + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU Lesser General Public License as published + by the Free Software Foundation; version 2.1 only. with the special + exception on linking described in file LICENSE. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Lesser General Public License for more details. + *) + +module D = Debug.Make (struct let name = __MODULE__ end) + +open D +module Unixext = Xapi_stdext_unix.Unixext + +(* +module DriverMap = Map.Make (String) +module DriverSet = Set.Make (String) +*) +module T = Xapi_host_driver_tool + +let invalid_value field value = + raise Api_errors.(Server_error (invalid_value, [field; value])) + +let internal_error fmt = + Printf.ksprintf + (fun msg -> + error "%s" msg ; + raise Api_errors.(Server_error (internal_error, [msg])) + ) + fmt + +let drivertool args = + let path = !Xapi_globs.driver_tool in + try + let stdout, _stderr = Forkhelpers.execute_command_get_output path args in + debug "%s: executed %s %s" __FUNCTION__ path (String.concat " " args) ; + stdout + with e -> + internal_error "%s: failed to run %s %s: %s" __FUNCTION__ path + (String.concat " " args) (Printexc.to_string e) + +module Variant = struct + let create ~__context ~name ~version ~driver ~hw_present ~priority ~dev_status + = + info "%s: %s" __FUNCTION__ name ; + let ref = Ref.make () in + let uuid = Uuidx.to_string (Uuidx.make ()) in + Db.Driver_variant.create ~__context ~ref ~driver ~uuid ~name ~version + ~hardware_present:hw_present ~priority ~status:dev_status ; + ref + + let destroy ~__context ~self = + debug "Destroying driver variant %s" (Ref.string_of self) ; + Db.Driver_variant.destroy ~__context ~self + + let select ~__context ~self = + debug "%s: %s" __FUNCTION__ (Ref.string_of self) ; + let drv = Db.Driver_variant.get_driver ~__context ~self in + let d = Db.Host_driver.get_record ~__context ~self:drv in + let v = Db.Driver_variant.get_record ~__context ~self in + let stdout = + drivertool ["select"; d.API.host_driver_name; v.API.driver_variant_name] + in + info "%s: %s" __FUNCTION__ stdout ; + Db.Host_driver.set_selected_variant ~__context ~self:drv ~value:self +end + +let create ~__context ~host ~name ~friendly_name ~_type ~description ~info:inf + ~active_variant ~selected_variant = + D.info "%s: %s" __FUNCTION__ name ; + let ref = Ref.make () in + let uuid = Uuidx.to_string (Uuidx.make ()) in + Db.Host_driver.create ~__context ~ref ~uuid ~host ~name ~friendly_name ~_type + ~description ~info:inf ~active_variant ~selected_variant ; + ref + +(** destroy driver and recursively its variants, too *) +let destroy ~__context ~self = + D.debug "Destroying driver %s" (Ref.string_of self) ; + let variants = Db.Host_driver.get_variants ~__context ~self in + variants |> List.iter (fun self -> Variant.destroy ~__context ~self) ; + Db.Host_driver.destroy ~__context ~self + +(** Runs on the host where the driver is installed *) +let select ~__context ~self ~variant = + let drv = Ref.string_of self in + let var = Ref.string_of variant in + let variants = Db.Host_driver.get_variants ~__context ~self in + match List.mem variant variants with + | true -> + D.debug "%s selecting driver %s variant %s" __FUNCTION__ drv var ; + let d = Db.Host_driver.get_record ~__context ~self in + let v = Db.Driver_variant.get_record ~__context ~self:variant in + let stdout = + drivertool ["select"; d.API.host_driver_name; v.API.driver_variant_name] + in + info "%s: %s" __FUNCTION__ stdout ; + Db.Host_driver.set_selected_variant ~__context ~self ~value:variant + | false -> + error "%s variant %s does not belong to driver %s" __FUNCTION__ var drv ; + invalid_value "variant" var + +(** Runs on the host where the driver is installed *) +let deselect ~__context ~self = + D.debug "%s driver %s" __FUNCTION__ (Ref.string_of self) ; + let d = Db.Host_driver.get_record ~__context ~self in + let stdout = drivertool ["deselect"; d.API.host_driver_name] in + info "%s: %s" __FUNCTION__ stdout ; + Db.Host_driver.set_active_variant ~__context ~self ~value:Ref.null ; + Db.Host_driver.set_selected_variant ~__context ~self ~value:Ref.null + +(** remove all host driver entries for this host *) +let reset ~__context ~host = + D.debug "%s" __FUNCTION__ ; + let open Xapi_database.Db_filter_types in + let expr = Eq (Field "host", Literal (Ref.string_of host)) in + let drivers = Db.Host_driver.get_refs_where ~__context ~expr in + drivers |> List.iter (fun self -> destroy ~__context ~self) + +(** Runs on [host] *) +let scan ~__context ~host = + T.Mock.install () ; + let null = Ref.null in + reset ~__context ~host ; + drivertool ["list"] + |> T.parse + |> List.iter @@ fun (_name, driver) -> + let driver_ref = + create ~__context ~host ~name:driver.T.name ~friendly_name:driver.T.name + ~info:driver.T.info ~active_variant:null ~selected_variant:null + ~description:driver.T.descr ~_type:driver.T.ty + in + driver.T.variants + |> List.iter @@ fun (name, v) -> + let var_ref = + Variant.create ~__context ~name ~version:v.T.version + ~driver:driver_ref ~hw_present:v.T.hw_present ~priority:v.T.priority + ~dev_status:v.T.dev_status + in + ( match driver.T.selected with + | Some v when v = name -> + Db.Host_driver.set_selected_variant ~__context ~self:driver_ref + ~value:var_ref + | _ -> + () + ) ; + match driver.T.active with + | Some v when v = name -> + Db.Host_driver.set_active_variant ~__context ~self:driver_ref + ~value:var_ref + | _ -> + () + +(** Runs on [host] *) +let rescan ~__context ~host = debug "%s" __FUNCTION__ ; scan ~__context ~host diff --git a/ocaml/xapi/xapi_host_driver.mli b/ocaml/xapi/xapi_host_driver.mli new file mode 100644 index 00000000000..41076ae45f5 --- /dev/null +++ b/ocaml/xapi/xapi_host_driver.mli @@ -0,0 +1,61 @@ +(* + Copyright (c) Cloud Software Group, Inc. + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU Lesser General Public License as published + by the Free Software Foundation; version 2.1 only. with the special + exception on linking described in file LICENSE. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Lesser General Public License for more details. + *) + +(** A host driver variant, referred to by a host driver *) +module Variant : sig + val create : + __context:Context.t + -> name:string + -> version:string + -> driver:[`Host_driver] API.Ref.t + -> hw_present:bool + -> priority:float + -> dev_status:string + -> [`Driver_variant] Ref.t + + val destroy : __context:Context.t -> self:[`Driver_variant] API.Ref.t -> unit + + val select : __context:Context.t -> self:[`Driver_variant] API.Ref.t -> unit +end + +val create : + __context:Context.t + -> host:[`host] API.Ref.t + -> name:string + -> friendly_name:string + -> _type:string + -> description:string + -> info:string + -> active_variant:[`Driver_variant] API.Ref.t + -> selected_variant:[`Driver_variant] API.Ref.t + -> [`Host_driver] Ref.t +(** A host driver *) + +val destroy : __context:Context.t -> self:[`Host_driver] API.Ref.t -> unit + +val select : + __context:Context.t + -> self:[`Host_driver] API.Ref.t + -> variant:[`Driver_variant] API.Ref.t + -> unit + +val deselect : __context:Context.t -> self:[`Host_driver] API.Ref.t -> unit +(** This is just for completeness; don't see a use case right now *) + +val scan : __context:Context.t -> host:[`host] API.Ref.t -> unit +(** scan and re-scan scan the [host] for drivers and update the xapi + database accordingly. Previous entries are purged (this may change + in the future *) + +val rescan : __context:Context.t -> host:[`host] API.Ref.t -> unit diff --git a/ocaml/xapi/xapi_host_driver_tool.ml b/ocaml/xapi/xapi_host_driver_tool.ml new file mode 100644 index 00000000000..a35608b84f1 --- /dev/null +++ b/ocaml/xapi/xapi_host_driver_tool.ml @@ -0,0 +1,648 @@ +(* + Copyright (c) Cloud Software Group, Inc. + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU Lesser General Public License as published + by the Free Software Foundation; version 2.1 only. with the special + exception on linking described in file LICENSE. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Lesser General Public License for more details. + *) + +(* This module provides functions to communicate with with the + host-driver-tool that manages multi-version drivers and reports + the state of them in JSON *) + +module D = Debug.Make (struct let name = __MODULE__ end) + +open D + +let internal_error fmt = + Printf.ksprintf + (fun str -> + error "%s" str ; + raise Api_errors.(Server_error (internal_error, [str])) + ) + fmt + +(** types to represent the JSON output of the script that reports + drivers *) +type variant = { + version: string + ; hw_present: bool + ; priority: float + ; dev_status: string +} + +type driver = { + ty: string + ; name: string + ; descr: string + ; info: string + ; selected: string option + ; active: string option + ; variants: (string * variant) list +} + +type t = {protocol: string; operation: string; drivers: (string * driver) list} + +(** names for drivers and variants must only have certain characters *) +let is_legal_char = function + | 'a' .. 'z' -> + true + | 'A' .. 'Z' -> + true + | '0' .. '9' -> + true + | '_' -> + true + | '.' -> + true + | '@' -> + true + | _ -> + false + +module R = Result + +(** create an error result from printf format string *) +let error fmt = Printf.ksprintf (fun str -> R.error str) fmt + +(** currently unused but needs to be used for names of drivers *) +let _is_legal_name name = + match String.for_all is_legal_char name with + | true -> + R.ok name + | false -> + error "'%s' contains illegal characters" name + +(* Wrap the combinators to parse JSON such that they don't raise + exceptions but return Result.t values. These can be used for + monadic error handling *) +module J = struct + module U = Yojson.Basic.Util + module B = Yojson.Basic + + let wrap f x = + try f x |> R.ok with + | U.Type_error (msg, json) -> + error "%s in %s" msg (Yojson.Basic.to_string json) + | e -> + R.error (Printexc.to_string e) + + let wrap2 f x y = + try f x y |> R.ok with + | U.Type_error (msg, json) -> + error "%s in %s" msg (Yojson.Basic.to_string json) + | e -> + R.error (Printexc.to_string e) + + let _keys = wrap U.keys + + let _values = wrap U.values + + let _combine = wrap2 U.combine + + let member key json = + match U.member key json with + | x -> + R.ok x + | exception U.Type_error (msg, json) -> + error "%s in %s" msg (Yojson.Basic.to_string json) + | exception e -> + R.error (Printexc.to_string e) + + let _path = wrap2 U.path + + let _index = wrap2 U.index + + let _map = wrap2 U.map + + let to_assoc = wrap U.to_assoc + + let _to_option f = wrap (U.to_option f) + + let to_bool = wrap U.to_bool + + let _to_bool_option = wrap U.to_bool_option + + let to_number = wrap U.to_number + + let _to_number_option = wrap U.to_number_option + + let _to_float = wrap U.to_float + + let _to_float_option = wrap U.to_float_option + + let _to_int = wrap U.to_int + + let _to_int_option = wrap U.to_int_option + + let _to_list = wrap U.to_list + + let to_string = wrap U.to_string + + let to_string_option = wrap U.to_string_option + + let _convert_each f = wrap (U.convert_each f) +end + +(** combinators for results and accessing members in JSON objects *) +let ( let* ) = Result.bind + +let ( ||> ) = Result.bind + +let ( ||. ) json key = json ||> J.member key + +let ( |. ) json key = json |> J.member key + +(** combine a list of ok list of oks into an ok list: + ('a, 'b) result list -> ('a list, 'b) result. *) +let _combine results = + let rec loop acc = function + | [] -> + R.Ok (List.rev acc) + | R.Ok x :: xs -> + loop (x :: acc) xs + | (R.Error _ as err) :: _ -> + err + in + loop [] results + +(** combine an association list where the value is ok/error into an ok + association list *) +let combine_assoc results = + let rec loop acc = function + | [] -> + R.Ok (List.rev acc) + | (key, R.Ok x) :: xs -> + loop ((key, x) :: acc) xs + | (_, (R.Error _ as err)) :: _ -> + err + in + loop [] results + +let _protocol json = json |. "protocol" ||. "version" ||> J.to_string + +let _operation json = json |. "operation" ||. "reboot" ||> J.to_bool + +let variant json = + let* version = json |. "version" ||> J.to_string in + let* hw_present = json |. "hardware_present" ||> J.to_bool in + let* priority = json |. "priority" ||> J.to_number in + let* dev_status = json |. "status" ||> J.to_string in + R.ok {version; hw_present; priority; dev_status} + +let variants json = + let* assoc = J.to_assoc json in + assoc |> List.map (fun (name, json) -> (name, variant json)) |> combine_assoc + +let driver json = + let* ty = json |. "type" ||> J.to_string in + let* name = json |. "friendly_name" ||> J.to_string in + let* descr = json |. "description" ||> J.to_string in + let* info = json |. "info" ||> J.to_string in + let* selected = json |. "selected" ||> J.to_string_option in + let* active = json |. "active" ||> J.to_string_option in + let* variants = json |. "variants" ||> variants in + R.ok {ty; name; descr; info; selected; active; variants} + +let drivers json = + let* assoc = J.to_assoc json in + assoc |> List.map (fun (name, json) -> (name, driver json)) |> combine_assoc + +let t json = json |. "drivers" ||> drivers + +let parse str = + match J.B.from_string str |> t with + | R.Ok x -> + x + | R.Error msg -> + internal_error "%s parsing failed: %s" __FUNCTION__ msg + | exception e -> + raise e + +let read path = + match J.B.from_file path |> t with + | R.Ok x -> + x + | R.Error msg -> + internal_error "%s parsing %s failed: %s" __FUNCTION__ path msg + | exception e -> + raise e + +module Mock = struct + let drivertool_sh = + {|#!/usr/bin/env bash + +set -o errexit +set -o pipefail +if [[ -n "$TRACE" ]]; then set -o xtrace; fi +set -o nounset + +if [[ "${1-}" =~ ^-*h(elp)?$ ]]; then + cat <&1 + exit 1 + ;; +esac +|} + + let install () = + let path = !Xapi_globs.driver_tool in + try + Xapi_stdext_unix.Unixext.write_string_to_file path drivertool_sh ; + Unix.chmod path 0o755 + with e -> + internal_error "%s: can't install %s: %s" __FUNCTION__ path + (Printexc.to_string e) +end diff --git a/ocaml/xapi/xapi_host_driver_tool.mli b/ocaml/xapi/xapi_host_driver_tool.mli new file mode 100644 index 00000000000..6957aa0a866 --- /dev/null +++ b/ocaml/xapi/xapi_host_driver_tool.mli @@ -0,0 +1,43 @@ +(* + Copyright (c) Cloud Software Group, Inc. + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU Lesser General Public License as published + by the Free Software Foundation; version 2.1 only. with the special + exception on linking described in file LICENSE. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Lesser General Public License for more details. + *) + +type variant = { + version: string (** just a string, not interpreted right now *) + ; hw_present: bool (** hardware present *) + ; priority: float (** higher = higher priority *) + ; dev_status: string (** development status, like alpha, beta *) +} + +type driver = { + ty: string (** category, like "network" *) + ; name: string (** unique *) + ; descr: string + ; info: string + ; selected: string option (** refers to named variant below *) + ; active: string option (** refers to named variant below *) + ; variants: (string * variant) list (** named variants, name is unique *) +} + +type t = {protocol: string; operation: string; drivers: (string * driver) list} + +val parse : string -> (string * driver) list +(** parse from a string *) + +val read : string -> (string * driver) list +(** read from a file whose path is provided *) + +(** install a mock drivertool.sh *) +module Mock : sig + val install : unit -> unit +end diff --git a/ocaml/xe-cli/bash-completion b/ocaml/xe-cli/bash-completion index b4ba6127138..b259955b929 100644 --- a/ocaml/xe-cli/bash-completion +++ b/ocaml/xe-cli/bash-completion @@ -493,6 +493,17 @@ _xe() return 0 ;; + version) # for hostdriver-select + if [[ "$COMP_CWORD" == "3" ]]; then + __xe_debug "triggering autocompletion for hostdriver's version" + IFS=$'\n,' + local cmd="$xe hostdriver-list ${OLDSTYLE_WORDS[2]} --minimal params=versions 2>/dev/null" + __xe_debug "full list cmd is '$cmd'" + local vals=$(eval "$cmd") + set_completions "${vals//; /,}" "$value" + fi + ;; + data-source) # for host-data-source-* __xe_debug "param is 'data-source', list command is 'host-data-source-list'" IFS=$',' diff --git a/quality-gate.sh b/quality-gate.sh index a7ffefea72b..03c50efe132 100755 --- a/quality-gate.sh +++ b/quality-gate.sh @@ -25,15 +25,19 @@ verify-cert () { } mli-files () { - N=497 + N=499 + X="ocaml/tests" + X+="|ocaml/quicktest" + X+="|ocaml/message-switch/core_test" # do not count ml files from the tests in ocaml/{tests/quicktest} - MLIS=$(git ls-files -- '**/*.mli' | grep -vE "ocaml/tests|ocaml/quicktest|ocaml/message-switch/core_test" | xargs -I {} sh -c "echo {} | cut -f 1 -d '.'" \;) - MLS=$(git ls-files -- '**/*.ml' | grep -vE "ocaml/tests|ocaml/quicktest|ocaml/message-switch/core_test" | xargs -I {} sh -c "echo {} | cut -f 1 -d '.'" \;) - num_mls_without_mlis=$(comm -23 <(sort <<<"$MLS") <(sort <<<"$MLIS") | wc -l) - if [ "$num_mls_without_mlis" -eq "$N" ]; then - echo "OK counted $num_mls_without_mlis .ml files without an .mli" + M=$(comm -23 <(git ls-files -- '**/*.ml' | sed 's/.ml$//' | sort) \ + <(git ls-files -- '**/*.mli' | sed 's/.mli$//' | sort) |\ + grep -cvE "$X") + + if [ "$M" -eq "$N" ]; then + echo "OK counted $M .ml files without an .mli" else - echo "ERROR expected $N .ml files without .mlis, got $num_mls_without_mlis."\ + echo "ERROR expected $N .ml files without .mlis, got $M."\ "If you created some .ml files, they are probably missing corresponding .mli's" 1>&2 exit 1 fi diff --git a/scripts/bugtool-plugin/xapi/stuff.xml b/scripts/bugtool-plugin/xapi/stuff.xml index cc4cf61dc32..b7d8ad95162 100644 --- a/scripts/bugtool-plugin/xapi/stuff.xml +++ b/scripts/bugtool-plugin/xapi/stuff.xml @@ -14,4 +14,6 @@ cat @ETCXENDIR@/xapi-pool-tls.pem | @BINDIR@/openssl x509 -text rrd-cli save_rrds @OPTDIR@/bin/xe task-list params=all +ls -lR /lib/modules/$(uname -r)/updates +ls -lR /lib/modules/$(uname -r)/xenserver