Closes #25: Add configuration management

.github/workflows/commit.yaml

@@ -31,7 +31,7 @@ jobs:
     strategy:
       max-parallel: 10
       matrix:
-        netbox_version: ["v3.5.9", "v3.6.1"]
+        netbox_version: ["v3.5.9", "v3.6.4"]
     steps:
       - name: Checkout
         uses: actions/checkout@v3

README.md

@@ -16,13 +16,29 @@
 <!--about-start-->
 ## About
 
+### Collecting diffs
+
 With this plugin you can find diff between the rendered configuration for a device to its actual configuration, retrieved from the device itself, or stored in DataSource.
 Read about [DataSources](https://demo.netbox.dev/static/docs/models/core/datasource/) for further details.
 
 Device configuration renders natively in NetBox. This [feature](https://demo.netbox.dev/static/docs/features/configuration-rendering/) was introduced in 3.5 version.
  NetBox Labs [blog](https://netboxlabs.com/blog/how-to-generate-device-configurations-with-netbox/) post about it.
 
 Plugin supports a wide list of vendors (Cisco, Juniper, Huawei, MicroTik etc.) with the help of Scrapli. Read [Scrapli](https://carlmontanari.github.io/scrapli/user_guide/project_details/#supported-platforms) and [scrapli-community](https://scrapli.github.io/scrapli_community/user_guide/project_details/#supported-platforms) documentations to find full list of vendors.
+
+### Pushing configuration
+
+Also you can push rendered configuration from NetBox to device and apply it.
+
+Supported platforms:
+
+* `arista_eos`
+* `cisco_iosxe`
+* `cisco_iosxr`
+* `cisco_nxos`
+* `juniper_junos`
+
+This is possible thanks to the scrapli_cfg. Read [Scrapli](https://github.com/scrapli/scrapli_cfg/) documentation for more info.
 <!--about-end-->
 
 ## Compatibility

docs/changelog.md

@@ -1,5 +1,9 @@
 # Changelog
 
+## 2.0.0 (2023-10-XX)
+
+* [#25](https://github.com/miaow2/netbox-config-diff/issues/25) Add configuration management
+
 ## 1.2.2 (2023-09-29)
 
 * [#28](https://github.com/miaow2/netbox-config-diff/issues/28) Add legacy ssh algorithms to support old OS versions

docs/usage.md → docs/colliecting-diffs.md

@@ -32,7 +32,7 @@ In the script, you can define a site, on which devices run compliance, or device
  If you define both fields, script will run only on devices from `Devices` field
 
 !!! warning
-    Script runs only on devices with status `Active`, assigned Primary IP, Platform and PlatformSetting
+    Script runs only on devices with status `Active`, assigned Primary IP, Config Template, Platform and PlatformSetting
 
 If you have configs in NetBox DataSource, you can define it, the script instead of connecting to devices will find configs in DataSource by device's names.
 

docs/configuratiom-management.md

@@ -0,0 +1,114 @@
+# Usage
+
+With plugin you can push rendered configuration from NetBox to devices.
+
+Supported platforms:
+
+* `arista_eos`
+* `cisco_iosxe`
+* `cisco_iosxr`
+* `cisco_nxos`
+* `juniper_junos`
+
+Plugin using [scrapli-cfg](https://github.com/scrapli/scrapli_cfg) for this feature.
+
+## Substitutes
+
+If you render not full configuration, it is acceptable to pull missing config sections from the actual configuration to render full configuration.
+
+!!! note
+    If you render full configuration in NetBox, you can proceed to `Configuration Request` part
+
+To do that you should create substitute.
+
+Substitutes is a "tag" that needs to be replaced with output from the real device, and a regex pattern that "pulls" this section from the actual device itself.
+
+![Screenshot of the substitute](media/screenshots/substitute.png)
+
+In screenshot below we add substitute for Arista PlatformSetting
+
+* **Name** is a "tag", you should put this as jinja2 variable in your config template in NetBox
+* **Regexp** is a regex, that "pulls" what matched from device and replace `Name` jinja2 variable in config template
+
+In example substitute `ethernet_interfaces` section will be replaced with whatever the provided pattern finds from the real device.
+
+This pattern matches all ethernet interfaces on a Arista device.
+
+To correctly render substitute in config template you have two options:
+
+```
+{{ "{{ ethernet_interfaces }}" }}
+```
+
+or
+
+```
+{% raw %}{{ ethernet_interfaces }}{% endraw %}
+```
+
+Config template will look like:
+
+![Screenshot of the config template with substitute](media/screenshots/config-temp-substitute.png)
+
+And rendered config template with substitute
+
+![Screenshot of the rendered template with substitute](media/screenshots/render-temp-substitute.png)
+
+## Configuration Request
+
+Now you let's create `Configuration Request` with devices you want to configure.
+
+!!! warning
+    For request only accepts devices with `Active` status and assigned Platform, Primary IP, Config Template and PlatformSetting
+
+Find `Configuration Requests` in navbar.
+
+Now collect diffs for devices pressing `Collecting diffs` button.
+
+![Screenshot of the Collecting diffs button](media/screenshots/cr-collecting-diff-button.png)
+
+On tab `Diffs` you can review diffs for devices.
+
+![Screenshot of the Diffs tab](media/screenshots/cr-diffs-tab.png)
+
+To continue approve request by pressing `Approve` button.
+
+![Screenshot of the Approve button](media/screenshots/cr-approve-button.png)
+
+Also you can cancel approve after that.
+
+![Screenshot of the Unapprove button](media/screenshots/cr-unapprove-button.png)
+
+After approval you can see by whom configuration request is approved.
+
+![Screenshot of the Approved request](media/screenshots/cr-approved.png)
+
+At this moment you can schedule job that will push rendered configuration to devices in configuration request by pressing schedule button.
+
+![Screenshot of the Schedule button](media/screenshots/cr-schedule-button.png)
+
+After that you can see by whom configuration request is scheduled and time.
+
+![Screenshot of the Scheduled request](media/screenshots/cr-scheduled.png)
+
+Also you can cancel scheduled job by pressing `Unschedule` button.
+
+![Screenshot of the Unschedule button](media/screenshots/cr-unschedule-button.png)
+
+!!! warning
+    Approve and Schedule buttons is accessable only to user with `netbox_config_diff.approve_configurationrequest`
+    permission
+
+!!! warning
+    If you unapprove scheduled configuration request, scheduled job will be canceled
+
+After scheduled job is completed you can job logs on configuration request page.
+
+![Screenshot of the Unschedule button](media/screenshots/cr-job-log.png)
+
+!!! note
+    Completed configuration requests can't be edited.
+
+## Rollback
+
+If an error occurs while executing a job that pushes configurations to devices then all configured devices will be rollbacked to the previous version of the configuration.

docs/index.md

@@ -12,4 +12,4 @@
 
 ## Usage
 
-Read this [doc](usage.md) to find out how to use plugin
+Read this [doc](colliecting-diffs.md) about collecting diffs, for configuration management read [this](configuratiom-management.md)

mkdocs.yml

@@ -6,7 +6,8 @@ repo_name: miaow2/netbox-config-diff
 nav:
   - Home: index.md
   - User Guide:
-    - Quick Start Guide: usage.md
+    - Collecting diffs: colliecting-diffs.md
+    - Configuration management: configuratiom-management.md
     - Integration with secrets: secrets.md
   - Screenshots: screenshots.md
   - Contributing: contributing.md

netbox_config_diff/api/serializers.py

@@ -1,10 +1,15 @@
 from dcim.api.serializers import NestedDeviceSerializer, NestedPlatformSerializer
-from netbox.api.fields import ChoiceField
+from dcim.models import Device
+from netbox.api.fields import ChoiceField, SerializedPKRelatedField
 from netbox.api.serializers import NetBoxModelSerializer
 from rest_framework import serializers
+from rest_framework.serializers import ValidationError
+from users.api.nested_serializers import NestedUserSerializer
+from utilities.utils import local_now
 
-from netbox_config_diff.choices import ConfigComplianceStatusChoices
-from netbox_config_diff.models import ConfigCompliance, PlatformSetting
+from netbox_config_diff.choices import ConfigComplianceStatusChoices, ConfigurationRequestStatusChoices
+from netbox_config_diff.constants import ACCEPTABLE_DRIVERS
+from netbox_config_diff.models import ConfigCompliance, ConfigurationRequest, PlatformSetting, Substitute
 
 
 class ConfigComplianceSerializer(NetBoxModelSerializer):
@@ -45,6 +50,113 @@ class Meta:
             "driver",
             "command",
             "exclude_regex",
+            "description",
+            "tags",
+            "custom_fields",
+            "created",
+            "last_updated",
+        )
+
+
+class NestedPlatformSettingSerializer(NetBoxModelSerializer):
+    url = serializers.HyperlinkedIdentityField(view_name="plugins-api:netbox_config_diff-api:platformsetting-detail")
+
+    class Meta:
+        model = PlatformSetting
+        fields = ("id", "url", "display", "driver")
+
+
+class ConfigurationRequestSerializer(NetBoxModelSerializer):
+    url = serializers.HyperlinkedIdentityField(
+        view_name="plugins-api:netbox_config_diff-api:configurationrequest-detail"
+    )
+    devices = SerializedPKRelatedField(
+        queryset=Device.objects.all(),
+        serializer=NestedDeviceSerializer,
+        many=True,
+    )
+    status = ChoiceField(choices=ConfigurationRequestStatusChoices, read_only=True)
+    created_by = NestedUserSerializer(read_only=True)
+    approved_by = NestedUserSerializer(read_only=True)
+    scheduled_by = NestedUserSerializer(read_only=True)
+
+    class Meta:
+        model = ConfigurationRequest
+        fields = (
+            "id",
+            "url",
+            "display",
+            "devices",
+            "created_by",
+            "approved_by",
+            "scheduled_by",
+            "scheduled",
+            "started",
+            "completed",
+            "status",
+            "description",
+            "comments",
+            "tags",
+            "custom_fields",
+            "created",
+            "last_updated",
+        )
+        read_only_fields = ["started", "scheduled", "completed"]
+
+    def validate(self, data):
+        if data.get("devices"):
+            if devices := data["devices"].filter(platform__platform_setting__isnull=True):
+                platforms = {d.platform.name for d in devices}
+                raise ValidationError({"devices": f"Assign PlatformSetting for platform(s): {', '.join(platforms)}"})
+
+            if drivers := {
+                device.platform.platform_setting.driver
+                for device in data["devices"]
+                if device.platform.platform_setting.driver not in ACCEPTABLE_DRIVERS
+            }:
+                raise ValidationError({"devices": f"Driver(s) not supported: {', '.join(drivers)}"})
+
+        return super().validate(data)
+
+
+class ConfigurationRequestRWSerializer(ConfigurationRequestSerializer):
+    created_by = NestedUserSerializer()
+
+
+class NestedConfigurationRequestSerializer(NetBoxModelSerializer):
+    url = serializers.HyperlinkedIdentityField(
+        view_name="plugins-api:netbox_config_diff-api:configurationrequest-detail"
+    )
+    status = ChoiceField(choices=ConfigurationRequestStatusChoices)
+
+    class Meta:
+        model = ConfigurationRequest
+        fields = ("id", "url", "display", "status")
+
+
+class ConfigurationRequestScheduleSerializer(serializers.Serializer):
+    schedule_at = serializers.DateTimeField()
+
+    def validate_schedule_at(self, value):
+        if value < local_now():
+            raise serializers.ValidationError("Scheduled time must be in the future.")
+        return value
+
+
+class SubstituteSerializer(NetBoxModelSerializer):
+    url = serializers.HyperlinkedIdentityField(view_name="plugins-api:netbox_config_diff-api:substitute-detail")
+    platform_setting = NestedPlatformSettingSerializer()
+
+    class Meta:
+        model = Substitute
+        fields = (
+            "id",
+            "url",
+            "display",
+            "platform_setting",
+            "name",
+            "description",
+            "regexp",
             "tags",
             "custom_fields",
             "created",

netbox_config_diff/api/urls.py

@@ -7,5 +7,7 @@
 router = NetBoxRouter()
 router.register("config-compliances", views.ConfigComplianceViewSet)
 router.register("platform-settings", views.PlatformSettingViewSet)
+router.register("configuration-requests", views.ConfigurationRequestViewSet)
+router.register("substitutes", views.SubstituteViewSet)
 
 urlpatterns = router.urls