decode20-demos

This is sample code for the following de:code 20 session demonstration:

X05 SaaS で迅速に IoT を実現 - Azure IoT Central 最新アップデートと活用術

• Real-time object detection on Azure IoT Edge device (NVIDIA Jetson Nano)
• Visualize insights in Azure IoT Central
• Trigger Teams alerts from Azure IoT Central

Architecture overview

Prerequisites

• See the Prerequisites section in NVIDIA Deepstream + Azure IoT Edge on a NVIDIA Jetson Nano
  • JetPack 4.4 is the latest as of May 2020, but it does not work properly, so please use JetPack 4.3 image
• + USB Camera for real-time image processing

Installation

Clone this repository

clone this repository by following command.

git clone [email protected]:yahanda/decode20-demos.git

Create some configuration for Deepstream application

1. Open SSH connection on your Nano device
2. Create new folders

   cd /var
   sudo mkdir deepstream
   sudo chmod -R 777 deepstream
   mkdir ./deepstream/custom_configs
   mkdir ./deepstream/custom_models

3. Copy config files from your PC to your Nano device with SCP

   scp -r <cloned-folder>/configs/custom_configs username@your-nano-ip-address:/var/deepstream
   scp -r <cloned-folder>/configs/custom_models username@your-nano-ip-address:/var/deepstream
   

Create an Azure Container Registry

1. Follow the instructions to Quickstart: Create a private container registry using the Azure portal
2. After your container registry is created, browse to it, and from the left pane select Access keys from the menu located under Settings.
3. Enable Admin user and copy the values for Login server, Username, and Password and save them somewhere convenient.

Build and Push IoT Edge Solution

1. Open git cloned folder with VS Code.
2. Update the .env with the values you made a note from Azure Container Registry.
3. Open the Visual Studio Code integrated terminal by selecting View > Terminal.
4. Sign in to Docker with the Azure container registry credentials that you saved after creating the registry.

   docker login -u <ACR username> -p <ACR password> <ACR login server>
   

5. Open the command palette and search for Azure IoT Edge: Set Default Target Platform for Edge Solution, or select the shortcut icon in the side bar at the bottom of the window.
6. In the command palette, select the target architecture from the list of options. For this sample, we're using a Jetson Nano as the IoT Edge device, so change the architecture to arm64v8.
7. Build and Push IoT Edge Solution by right clicking on deployment.template.json file.
8. Verify deployment.arm64v8.json file is generated in the config folder.

Create an Azure IoT Edge device template to your Azure IoT Central

1. Complete the Create an Azure IoT Central application quickstart to create an IoT Central application using the Custom app > Custom application template.
2. In your IoT Central application, navigate to Device templates and select + New.
3. On the Select template type page, select the Azure IoT Edge tile. Then select Next: Customize.
4. On the Upload an Azure IoT Edge deployment manifest page, select Browse to upload the deployment.arm64v8.json you generated previously. Then select Next: Review.
5. On the Review page, select Create.
6. When the template has been created, select Module Upstreamer and + Add capability.
7. Select Custom and add the following interface.

   Display name	Name	Capability type	Schema
   is_car	is_car	Telemetry	Integer
   x_car	x_car	Telemetry	Double
   y_car	y_car	Telemetry	Double
   is_person	is_person	Telemetry	Integer
   x_person	x_person	Telemetry	Double
   y_person	y_person	Telemetry	Double

8. Select Save to update the template.
9. Select Views in the IoT Edge Device template.
10. On the Select to add a new view page, select the Visualizing the device tile.
11. Select the is_car and is_person telemetry types. Then select Add tile.
12. Select the x_car and x_person telemetry types. Then select Add tile.
13. Select the y_car and y_person telemetry types. Then select Add tile.
14. Select Save to save the View IoT Edge device telemetry view.
15. Navigate to the IoT Edge Device template and select Publish. Then select Publish to publish the template.

Add an Azure IoT Edge device to your IoT Central

1. In your IoT Central application, navigate to the Devices page and select + New to add a new device.
2. Select the IoT Edge device template you created previously and select Create.
3. When the device has been created, select Connect on the Device page.
4. On the Device connection page, make a note of the ID Scope, the Device ID, and the Primary Key. You use these values later.

Deploy an IoT Edge device

1. Open SSH connection on your Nano device
2. Edit the IoT Edge config.yaml file:

   sudo vi /etc/iotedge/config.yaml

3. Scroll down until you see # Manual provisioning configuration. Comment out the next three lines as shown in the following snippet:

   # Manual provisioning configuration
   #provisioning:
   #  source: "manual"
   #  device_connection_string: "<ADD DEVICE CONNECTION STRING HERE>"

4. Scroll down until you see # DPS symmetric key provisioning configuration. Uncomment the next eight lines as shown in the following snippet:

   # DPS symmetric key provisioning configuration
   provisioning:
   source: "dps"
   global_endpoint: "https://global.azure-devices-provisioning.net"
   scope_id: "{scope_id}"
   attestation:
       method: "symmetric_key"
       registration_id: "{registration_id}"
       symmetric_key: "{symmetric_key}"

5. Replace {scope_id} with the ID Scope for you made a note of previously.
6. Replace {registration_id} with the Device ID you made a note of previously.
7. Replace {symmetric_key} with the Primary key you made a note of previously.
8. Save the changes.
9. Run the following command to restart the IoT Edge daemon:

   sudo systemctl restart iotedge

10. To check the status of the IoT Edge modules, run the following command:

    iotedge list

    The output looks like the following:

    NAME                 STATUS           DESCRIPTION      CONFIG
    NVIDIADeepStreamSDK  running          Up 1 hours       marketplace.azurecr.io/nvidia/deepstream-iot2-l4t:latest
    Upstreamer           running          Up 1 hours       yahanda1.azurecr.io/upstreamer:0.0.1-arm64v8
    edgeAgent            running          Up 1 hours       mcr.microsoft.com/azureiotedge-agent:1.0.9
    edgeHub              running          Up 1 hours       mcr.microsoft.com/azureiotedge-hub:1.0.9

Usage

View the real-time object detection via Real Time Streaming Protocol (RTSP)

1. Open VLC
2. Go to Media > Open Network Stream Paste the default RTSP Video URL generated by deepstream, which follows the format rtsp://your-nano-ip-address:8554/ds-test
3. Click Play

View the telemetry

1. Open your IoT Central application, and select the IoT Edge Devices you created previously.
2. You can see the telemetry on the View page

Note

If you use your own AI model with Custom Vision, please see the instructions below.

Create a Custom vision model

1. Go to http://customvision.ai and Sign-in
2. Create a new Project
3. Pick up your resource, if none select create new and select SKU - F0 (F0 is free) or (S0)
4. Select Project Type = Object Detection
5. Select Domains = General (Compact)
6. Choose training images, upload and tag images, and train the detector. See the Quickstart: How to build an object detector with Custom Vision for further information.
7. Export the detector by going to the Performance tab, clicking on Export and choosing ONNX.
8. Download your custom AI model and unzip it.

Replace files on the Jetson Nano device

1. Replace the model.onnx and labels.txt on the nano device with SCP.

   scp <unzipped-folder>/model.onnx username@your-nano-ip-address:/var/deepstream/custom_models/model.onnx
   scp <unzipped-folder>/label.txt username@your-nano-ip-address:/var/deepstream/custom_models/label.txt
   

2. Open the /var/deepstream/custom_configs/msgconv_config_soda_cans.txt file and update the num-detected-classes property in maps to the number of objects that you've trained your custom vision model for.

Modify the Upstreamer module

1. Open main.py in Upstreamer module with VS Code.
2. Modify the input1_listener function as appropriate for your model output.
3. Build and Push IoT Edge Solution again by right clicking on deployment.template.json file.

Update the interfaces in Azure IoT Central

1. Open your IoT Central application.
2. Update the interface and view of the device template as appropriate for your Upstreamer output.
3. Migrate the device to new template.
4. Run the following command to restart the IoT Edge daemon:

   sudo systemctl restart iotedge

5. To check the status of the IoT Edge modules, run the following command:

   iotedge list

   The output looks like the following:

   NAME                 STATUS           DESCRIPTION      CONFIG
   NVIDIADeepStreamSDK  running          Up 1 hours       marketplace.azurecr.io/nvidia/deepstream-iot2-l4t:latest
   Upstreamer           running          Up 1 hours       yahanda1.azurecr.io/upstreamer:0.0.1-arm64v8
   edgeAgent            running          Up 1 hours       mcr.microsoft.com/azureiotedge-agent:1.0.9
   edgeHub              running          Up 1 hours       mcr.microsoft.com/azureiotedge-hub:1.0.9