AML - initial set of AML rules and severity update (#2500)

* initial ML updates * initial AML changes * corrected ML test * corrected ML JSON, backing out test data * finalised md doco for ML * inital ML disable local rule * ML disable admin rule * ML compute vnet added, md update on hyperlinks * links updated for ML rules * update ML compute test ref for extend to workspaces * ML Workspace rule added, ML titles added to mds * text upds * ML Wks pub access rule added. Refs updated * ML Wkspace md updated * ML wrkspc rule for use mg id * updates to rule conditions * minor text update * updated ML - WrkspUserMgId rule logic * correct ML.WrkspUserMgId error * Updates from previous PR * Final updates * Bump change log * Fix dup --------- Co-authored-by: Bernie White <[email protected]>
Azure · Oct 31, 2023 · c6efe6e · c6efe6e
1 parent 5e8bc3c
commit c6efe6e
Show file tree

Hide file tree

Showing 12 changed files with 1,021 additions and 2 deletions.
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -66,7 +66,7 @@ When writing documentation in Markdown, please follow these formatting guideline
 Before improving rule recommendations familiarize yourself with writing [rule markdown documentation][4].
 Rule documentation requires the following annotations for use with PSRule for Azure:
 
-- `severity` - A subjective rating of the impact of a rule the solution or platform.
+- `severity` - A subjective rating of the impact of a rule on the solution or platform.
   *NB* - the severity ratings reflect a productionised implementation, consideration should be applied for pre-production environments.
 
   Available severities are:

diff --git a/docs/CHANGELOG-v1.md b/docs/CHANGELOG-v1.md
@@ -30,6 +30,18 @@ See [upgrade notes][1] for helpful information when upgrading from previous vers
 
 What's changed since v1.30.3:
 
+- New rules:
+  - Machine Learning:
+    - Check compute instances are configured for an idle shutdown by @batemansogq.
+      [#2484](https://github.com/Azure/PSRule.Rules.Azure/issues/2484)
+    - Check workspace compute has local authentication disabled by @batemansogq.
+      [#2484](https://github.com/Azure/PSRule.Rules.Azure/issues/2484)
+    - Check workspace compute is connected to a VNET by @batemansogq.
+      [#2484](https://github.com/Azure/PSRule.Rules.Azure/issues/2484)
+    - Check public access to a workspace is disabled by @batemansogq.
+      [#2484](https://github.com/Azure/PSRule.Rules.Azure/issues/2484)
+    - Check workspaces use a user-assigned identity by @batemansogq.
+      [#2484](https://github.com/Azure/PSRule.Rules.Azure/issues/2484)
 - Engineering:
   - Bump development tools to .NET 7.0 SDK by @BernieWhite.
     [#1870](https://github.com/Azure/PSRule.Rules.Azure/issues/1870)

diff --git a/docs/en/rules/Azure.ML.ComputeIdleShutdown.md b/docs/en/rules/Azure.ML.ComputeIdleShutdown.md
@@ -0,0 +1,90 @@
+---
+reviewed: 2023-10-06
+severity: Critical
+pillar: Cost Optimization
+category: Provision
+resource: Machine Learning
+online version: https://azure.github.io/PSRule.Rules.Azure/en/rules/Azure.ML.ComputeIdleShutdown/
+---
+
+# Configure idle shutdown for compute instances
+
+## SYNOPSIS
+
+Configure an idle shutdown timeout for Machine Learning compute instances.
+
+## DESCRIPTION
+
+Machine Learning uses compute instances as a training or inference compute for development and testing.
+It's similar to a virtual machine on the cloud.
+
+To avoid getting charged for a compute instance that is switched on but not being actively used,
+you can configure when to automatically shutdown compute instances due to inactivity.
+
+## RECOMMENDATION
+
+Consider configuring ML - Compute Instances to automatically shutdown after a period of inactivity to optimize compute costs.
+
+## EXAMPLES
+
+### Configure with Azure template
+
+To deploy compute instances that passes this rule:
+
+- Set the `properties.properties.idleTimeBeforeShutdown` property with a ISO 8601 formatted string.
+  i.e. For an idle shutdown time of 15 minutes use `PT15M`.
+
+For example:
+
+```json
+{
+  "type": "Microsoft.MachineLearningServices/workspaces/computes",
+  "apiVersion": "2023-06-01-preview",
+  "name": "[format('{0}/{1}', parameters('name'), parameters('name'))]",
+  "location": "[parameters('location')]",
+  "properties": {
+    "computeType": "ComputeInstance",
+    "disableLocalAuth": true,
+    "properties": {
+      "vmSize": "[parameters('vmSize')]",
+      "idleTimeBeforeShutdown": "PT15M"
+    }
+  },
+  "dependsOn": [
+    "[resourceId('Microsoft.MachineLearningServices/workspaces', parameters('name'))]"
+  ]
+}
+```
+
+### Configure with Bicep
+
+To deploy compute instances that passes this rule:
+
+- Set the `properties.properties.idleTimeBeforeShutdown` property with a ISO 8601 formatted string.
+  i.e. For an idle shutdown time of 15 minutes use `PT15M`.
+
+For example:
+
+```bicep
+resource compute_instance 'Microsoft.MachineLearningServices/workspaces/computes@2023-06-01-preview' = {
+  parent: workspace
+  name: name
+  location: location
+  properties: {
+    computeType: 'ComputeInstance'
+    disableLocalAuth: true
+    properties: {
+      vmSize: vmSize
+      idleTimeBeforeShutdown: 'PT15M'
+    }
+  }
+}
+```
+
+## LINKS
+
+- [AI + Machine Learning cost estimates](https://learn.microsoft.com/azure/well-architected/cost/provision-ai-ml)
+- [Configure idle shutdown](https://learn.microsoft.com/azure/machine-learning/how-to-create-compute-instance#configure-idle-shutdown)
+- [ML Compute](https://learn.microsoft.com/azure/machine-learning/azure-machine-learning-glossary#compute)
+- [Azure deployment reference - Compute objects](https://learn.microsoft.com/azure/templates/microsoft.machinelearningservices/workspaces/computes#compute-objects)
+- [Azure deployment reference - Workspaces](https://learn.microsoft.com/azure/templates/microsoft.machinelearningservices/workspaces)
diff --git a/docs/en/rules/Azure.ML.ComputeVnet.md b/docs/en/rules/Azure.ML.ComputeVnet.md
@@ -0,0 +1,97 @@
+---
+reviewed: 2023-10-10
+severity: Critical
+pillar: Security
+category: Connectivity
+resource: Machine Learning
+online version: https://azure.github.io/PSRule.Rules.Azure/en/rules/Azure.ML.ComputeVnet/
+---
+
+# Host ML Compute in VNet
+
+## SYNOPSIS
+
+Azure Machine Learning Computes should be hosted in a virtual network (VNet).
+
+## DESCRIPTION
+
+When using Azure Machine Learning (ML), you can configure compute instances to be private or accessible from the public Internet.
+By default, the ML compute is configured to be accessible from the public Internet.
+
+ML compute can be deployed into an virtual network (VNet) to provide private connectivity, enhanaced security, and isolation.
+Using a VNet reduces the attack surface for your solution, and the chances of data exfiltration.
+Additionally, network controls such as Network Security Groups (NSGs) can be used to further restrict access.
+
+## RECOMMENDATION
+
+Consider using ML - compute hosted in a VNet to provide private connectivity, enhanaced security, and isolation.
+
+## EXAMPLES
+
+### Configure with Azure template
+
+To deploy an ML - compute that passes this rule:
+
+- Set the `properties.properties.subnet.id` property with a resource Id of a specific VNET subnet.
+
+For example:
+
+```json
+{
+  "type": "Microsoft.MachineLearningServices/workspaces/computes",
+  "apiVersion": "2023-06-01-preview",
+  "name": "[format('{0}/{1}', parameters('name'), parameters('name'))]",
+  "location": "[parameters('location')]",
+  "properties": {
+    "computeType": "ComputeInstance",
+    "disableLocalAuth": true,
+    "properties": {
+      "vmSize": "[parameters('vmSize')]",
+      "idleTimeBeforeShutdown": "PT15M",
+      "subnet": {
+        "id": "[resourceId('Microsoft.Network/virtualNetworks/subnets', split('vnet/subnet', '/')[0], split('vnet/subnet', '/')[1])]"
+      }
+    }
+  },
+  "dependsOn": [
+    "[resourceId('Microsoft.MachineLearningServices/workspaces', parameters('name'))]"
+  ]
+}
+```
+
+### Configure with Bicep
+
+To deploy an ML - compute that passes this rule:
+
+- Set the `properties.properties.subnet.id` property with a resource Id of a specific VNET subnet.
+
+For example:
+
+```bicep
+resource compute_instance 'Microsoft.MachineLearningServices/workspaces/computes@2023-06-01-preview' = {
+  parent: workspace
+  name: name
+  location: location
+  properties: {
+    computeType: 'ComputeInstance'
+    disableLocalAuth: true
+    properties: {
+      vmSize: vmSize
+      idleTimeBeforeShutdown: 'PT15M'
+      subnet: {
+        id: subnet.id
+      }
+    }
+  }
+}
+```
+
+## LINKS
+
+- [WAF - Azure services for securing network connectivity](https://learn.microsoft.com/azure/well-architected/security/design-network-connectivity)
+- [Managed compute in a managed virtual network](https://learn.microsoft.com/azure/machine-learning/how-to-managed-network-compute)
+- [ML - Network security and isolation](https://learn.microsoft.com/azure/machine-learning/concept-enterprise-security#network-security-and-isolation)
+- [ML Compute](https://learn.microsoft.com/azure/machine-learning/azure-machine-learning-glossary#compute)
+- [NS-1: Establish network segmentation boundaries](https://learn.microsoft.com/security/benchmark/azure/baselines/machine-learning-service-security-baseline#ns-1-establish-network-segmentation-boundaries)
+- [Azure deployment reference - Compute objects](https://learn.microsoft.com/azure/templates/microsoft.machinelearningservices/workspaces/computes#compute-objects)
+- [Azure deployment reference - Workspaces](https://learn.microsoft.com/azure/templates/microsoft.machinelearningservices/workspaces)
diff --git a/docs/en/rules/Azure.ML.DisableLocalAuth.md b/docs/en/rules/Azure.ML.DisableLocalAuth.md
@@ -0,0 +1,95 @@
+---
+reviewed: 2023-10-10
+severity: Critical
+pillar: Security
+category: Authentication
+resource: Machine Learning
+online version: https://azure.github.io/PSRule.Rules.Azure/en/rules/Azure.ML.DisableLocalAuth/
+---
+
+# Disable local authentication on ML Compute
+
+## SYNOPSIS
+
+Azure Machine Learning compute resources should have local authentication methods disabled.
+
+## DESCRIPTION
+
+Azure Machine Learning (ML) compute can have local authenication enabled or disabled.
+When enabled local authentication methods must be managed and audited separately.
+
+Disabling local authentication ensures that Entra ID (previously Azure Active Directory) is used exclusively for authentication.
+Using Entra ID, provides consistency as a single authoritative source which:
+
+- Increases clarity and reduces security risks from human errors and configuration complexity.
+- Provides support for advanced identity security and governance features.
+
+## RECOMMENDATION
+
+Consider disabling local authentication on ML - Compute as part of a broader security strategy.
+
+## EXAMPLES
+
+### Configure with Azure template
+
+To deploy ML - compute that passes this rule:
+
+- Set the `properties.disableLocalAuth` property to `true`.
+
+For example:
+
+```json
+{
+  "type": "Microsoft.MachineLearningServices/workspaces/computes",
+  "apiVersion": "2023-06-01-preview",
+  "name": "[format('{0}/{1}', parameters('name'), parameters('name'))]",
+  "location": "[parameters('location')]",
+  "properties": {
+    "computeType": "ComputeInstance",
+    "disableLocalAuth": true,
+    "properties": {
+      "vmSize": "[parameters('vmSize')]",
+      "idleTimeBeforeShutdown": "PT15M"
+    }
+  },
+  "dependsOn": [
+    "[resourceId('Microsoft.MachineLearningServices/workspaces', parameters('name'))]"
+  ]
+}
+```
+
+### Configure with Bicep
+
+To deploy ML - compute that passes this rule:
+
+- Set the `properties.disableLocalAuth` property to `true`.
+
+For example:
+
+```bicep
+resource compute_instance 'Microsoft.MachineLearningServices/workspaces/computes@2023-06-01-preview' = {
+  parent: workspace
+  name: name
+  location: location
+  properties: {
+    computeType: 'ComputeInstance'
+    disableLocalAuth: true
+    properties: {
+      vmSize: vmSize
+      idleTimeBeforeShutdown: 'PT15M'
+      subnet: {
+        id: subnet.id
+      }
+    }
+  }
+}
+```
+
+## LINKS
+
+- [WAF - Authentication with Azure AD](https://learn.microsoft.com/azure/well-architected/security/design-identity-authentication)
+- [Disable local authentication](https://learn.microsoft.com/azure/machine-learning/how-to-integrate-azure-policy#disable-local-authentication)
+- [ML Compute](https://learn.microsoft.com/azure/machine-learning/azure-machine-learning-glossary#compute)
+- [Azure Policy Regulatory Compliance controls for Azure Machine Learning](https://learn.microsoft.com/azure/machine-learning/security-controls-policy)
+- [Azure deployment reference - Compute objects](https://learn.microsoft.com/azure/templates/microsoft.machinelearningservices/workspaces/computes#compute-objects)
+- [Azure deployment reference - Workspaces](https://learn.microsoft.com/azure/templates/microsoft.machinelearningservices/workspaces)