[email protected]

## [3.1.0-exp.3] - 2022-03-01 ### Changed - Upgraded the version of Input System package `1.3.0`. - Upgraded the version of WebRTC package `2.4.0-exp.6`. - Added microphone device option for streaming audio in the "Bidirectional" sample. - Integrate AR Foundation Sample (only use AR Foundation version 4.1.9, deprecated version 2.1) ### Fixed - Fixed the worker thread is not closed when stopping the signaling client. - Fixed keyboard input sending only sends one key at a time from browser. - Fixed video flipped vertically when uging GLES/GLCore graphics API. - Fixed the crach when encoding the high resolution on macOS using HWA. - Fixed the gap of the input position between sender and receiver. - Fixed the unexpected behaviour of input after using Gyro sample.
needle-mirror · Mar 1, 2022 · ab9568d · ab9568d
1 parent 0e9bb06
commit ab9568d
Show file tree

Hide file tree

Showing 63 changed files with 548 additions and 3,345 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -4,6 +4,24 @@ All notable changes to com.unity.renderstreaming package will be documented in t
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
 
+## [3.1.0-exp.3] - 2022-03-01
+
+### Changed
+
+- Upgraded the version of Input System package `1.3.0`.
+- Upgraded the version of WebRTC package `2.4.0-exp.6`.
+- Added microphone device option for streaming audio in the "Bidirectional" sample.
+- Integrate AR Foundation Sample (only use AR Foundation version 4.1.9, deprecated version 2.1)
+
+### Fixed
+
+- Fixed the worker thread is not closed when stopping the signaling client.
+- Fixed keyboard input sending only sends one key at a time from browser.
+- Fixed video flipped vertically when uging GLES/GLCore graphics API.
+- Fixed the crach when encoding the high resolution on macOS using HWA.
+- Fixed the gap of the input position between sender and receiver.
+- Fixed the unexpected behaviour of input after using Gyro sample.
+
 ## [3.1.0-exp.2] - 2021-12-01
 
 ### Added

diff --git a/Documentation~/audio-streaming.md b/Documentation~/audio-streaming.md
@@ -1,11 +1,11 @@
 # Audio Streaming
 
-`AudioStreamSender` class and `AudioStreamReceiver` class are the base classes for sending and receiving audio. And various components are implemented using inheritance relationships with these class. Components are provided for the sender and receiver.
+[`AudioStreamSender`](../api/Unity.RenderStreaming.AudioStreamSender.html) class and [`AudioStreamReceiver`]((../api/Unity.RenderStreaming.AudioStreamReceiver.html)) class are the base classes for sending and receiving audio. And various components are implemented using inheritance relationships with these class. Components are provided for the sender and receiver.
 
 > [!NOTE] 
 > There is currently a limitation in the audio codec selection. Currently, only **Opus** is used for the audio codec.
 
-## `AudioStreamSender` component
+## [`AudioStreamSender`](../api/Unity.RenderStreaming.AudioStreamSender.html) component
 
 This component streams the audio rendering results from [`AudioListener`](https://docs.unity3d.com/ScriptReference/AudioListener.html) component or [`AudioSource`](https://docs.unity3d.com/ScriptReference/AudioSource.html) component.
 
@@ -17,7 +17,7 @@ This component streams the audio rendering results from [`AudioListener`](https:
 | --------- | ----------- | ------- |
 | **Audio Source** | [`AudioSource`](https://docs.unity3d.com/ScriptReference/AudioSource.html) instance for sending audio | None |
 
-## `MicrophoneStreamSender` component
+## [`MicrophoneStreamSender`](../api/Unity.RenderStreaming.MicrophoneStreamSender.html) component
 
 This component streams the audio rendering results from [`Microphone`](https://docs.unity3d.com/ScriptReference/Microphone.html).
 
@@ -33,7 +33,7 @@ This component streams the audio rendering results from [`Microphone`](https://d
 An index of the list of available microphone devices
 
 
-## `AudioStreamReceiver` component
+## [`AudioStreamReceiver`](../api/Unity.RenderStreaming.AudioStreamReceiver.html) component
 
 The components receive audio track stream and rendering to [`AudioSource`](https://docs.unity3d.com/ScriptReference/AudioSource.html).
 

diff --git a/Documentation~/control-camera.md b/Documentation~/control-camera.md
@@ -2,6 +2,10 @@
 
 To control the camera on the remote PC,  we need to link the input events on web browsers to the actions in Unity. **Input System** makes it easy to implement the input processing.
 
+## Check settings of Input System
+
+First, please check [the settings](use-inputsystem.md) for using Input System.
+
 ## Adding components for input processing
 
 Open the Unity scene which created in [the previous page](create-camera.md), and select **Main Camera** in the Hierarchy window. Add the **Input Receiver** component in the Inspector window.
@@ -12,6 +16,10 @@ The initial condition of the inspector of **Input Receiver** is below.
 
 ![Add InputReceiver component](images/inputreceiver_inspector.png)
 
+Add the **Input Receiver** component to the **Broadcast** component property.
+
+![Assign InputReceiver to streams](images/assign_inputreceiver_to_streams.png)
+
 ## Setting Input Actions
 
 We will use the [**Input Actions**](https://docs.unity3d.com/Packages/[email protected]/manual/Actions.html) feature of the Input System. Input Actions is a mechanism for mapping various inputs to Unity actions, and it provides editing tools that make it easy for anyone to use. We will use Input Actions to define mouse actions.
@@ -44,9 +52,9 @@ The result will look like this. Note that the last step is to press the **Save A
 
 ![Control camera 01](images/control_camera_07.png)
 
-## Setting `InputReceiver` component
+## Setting [`InputReceiver`](../api/Unity.RenderStreaming.InputReceiver.html) component
 
-Drag the **Control** asset to the **Input Actions** property of the `InputReceiver` component.
+Drag the **Control** asset to the **Input Actions** property of the [`InputReceiver`](../api/Unity.RenderStreaming.InputReceiver.html) component.
 
 ![Control camera 01](images/assign_inputactions_to_inputreceiver.png)
 
@@ -70,7 +78,7 @@ public class PlayerController : MonoBehaviour
 }
 ```
 
-Once you have copied the script and saved the file, assign `PlayerController.Look` to the **InputReceiver** component's **Look** event.
+Once you have copied the script and saved the file, assign `PlayerController.Look` to the [`InputReceiver`](../api/Unity.RenderStreaming.InputReceiver.html) component's **Look** event.
 
 ![Control camera 01](images/assign_event_to_inputreceiver.png)
 

diff --git a/Documentation~/data-streaming.md b/Documentation~/data-streaming.md
@@ -4,7 +4,7 @@ Data streaming is using [`RTCDataChannel`](https://docs.unity3d.com/Packages/com
 
 Using the data streaming feature, we provide a component to send messages of [Input System](https://docs.unity3d.com/Packages/com.unity.inputsystem@latest). For more details, please check the section on [Using with Input System](use-inputsystem.md).
 
-## `InputSender` component
+## [`InputSender`](../api/Unity.RenderStreaming.InputSender.html) component
 
 This component sends input events fired from all input devices connected machine.
 
@@ -17,7 +17,7 @@ This component sends input events fired from all input devices connected machine
 | **Local** | Channel is local or remote | `False` |
 | **Label** | Channel label | Empty |
 
-## `InputReceiver` component
+## [`InputReceiver`](../api/Unity.RenderStreaming.InputReceiver.html) component
 
 This component receives input events from Unity.
 
@@ -29,6 +29,6 @@ This component receives input events from Unity.
 | --------- | ----------- | ------- |
 | **Local** | Channel is local or remote. | `False` |
 | **Label** | Channel label. | Empty |
-| **Actions** | The set of [**Input Actions**](https://docs.unity3d.com/Packages/com.unity.inputsystem@1.0/manual/Actions.html) associated with the player. To receive input, each player must have an associated set of Actions. See documentation on Actions for details. | `None` |
+| **Actions** | The set of [**Input Actions**](https://docs.unity3d.com/Packages/com.unity.inputsystem@1.3/manual/Actions.html) associated with the player. To receive input, each player must have an associated set of Actions. See documentation on Actions for details. | `None` |
 | *Default Map* | Which Action Map in `Actions` to enable by default. If set to `None`, then the player starts with no Actions being enabled. | `None` |
 | **Events** | Uses a separate UnityEvent for each individual type of message.  | Empty |
diff --git a/Documentation~/images/assign_inputreceiver_to_streams.png b/Documentation~/images/assign_inputreceiver_to_streams.png
diff --git a/Documentation~/index.md b/Documentation~/index.md
@@ -31,6 +31,7 @@ This version of Render Streaming is compatible with the following versions of th
 
 - **Unity 2019.4**
 - **Unity 2020.3**
+- **Unity 2021.2**
 
 ### Platform
 

diff --git a/Documentation~/installation.md b/Documentation~/installation.md
@@ -36,7 +36,7 @@ Check Package Manager window, Click `+` button and select `Add package from git
 Input the string below to the input field.
 
 ```
-[email protected].2
+[email protected].3
 ```
 
 The list of version string is [here](https://github.com/Unity-Technologies/com.unity.renderstreaming/tags). In most cases, the latest version is recommended to use.

diff --git a/Documentation~/sample-arfoundation.md b/Documentation~/sample-arfoundation.md
@@ -5,32 +5,26 @@ This sample demonstrates operating the position and direction of a camera on ano
 > [!NOTE]
 > If no codec is available on both sender and receiver, the video can not stream. Please note that the compatibility of codecs is different corresponding to platforms and graphics API.
 
-## Sample scenes
-
-There are two scenes in this sample because the verified package version of AR Foundation is different depends on Unity version.
-
-| Unity version | scene name    |
-| ------------- | ------------- |
-| Unity2019.4   | `ARFoundation2.1` |
-| Unity2020.3   | `ARFoundation4.1` |
-
 ## Additional packages
 
-To use this sample, You need to install additional packages shown following table.
+To use this sample, You need to install following additional packages.
 
-| Unity version | Platform | packages |
-| ------------- | -------- | -------- |
-| Unity2019.4   | Android, iOS | `ARFoundation 2.1`, `XR Plugin Management 4.2` |
-| Unity2020.3   | Android, iOS | `ARFoundation 4.1`, `XR Plugin Management 4.2` |
+- `ARFoundation 4.1.9`
+- `XR Plugin Management 4.2`
+
+> [!NOTE]
+> `AR Foundation 4.1.9` is compatible with the following versions of the Unity Editor:
+> - 2019.4.15f1 or later
+> - 2020.3
+> - 2021.1
+> - 2021.2
 
 In addition, depending on the platform, each `XR plugin` package must be installed.
 
-| Unity version | Platform | packages |
-| ------------- | -------- | -------- |
-| Unity2019.4   | iOS      | `ARKit XR Plugin 2.1` |
-| Unity2019.4   | Android  | `ARCore XR Plugin 2.1` |
-| Unity2020.3   | iOS      | `ARKit XR Plugin 4.1` |
-| Unity2020.3   | Android  | `ARCore XR Plugin 4.1` |
+| Platform | packages |
+| -------- | -------- |
+| iOS      | `ARKit XR Plugin 4.1.9` |
+| Android  | `ARCore XR Plugin 4.1.9` |
 
 ## Build settings
 
@@ -54,6 +48,9 @@ Open **Project Settings** window and select **XR Plug-in Management**. Set enabl
 
 ![XRPlugin Enable ARCore](images/xrplugin_enable_arcore.png)
 
+> [!NOTE]
+> If you get an error about Gradle version when building with Unity 2019.4, please refer to [this page](https://developers.google.com/ar/develop/unity/android-11-build).
+
 ## Usage
 
 ### Hardware acceleration 

diff --git a/Documentation~/signaling-type.md b/Documentation~/signaling-type.md
@@ -10,7 +10,7 @@ In the example, the schema given to `URL Signaling` is used to determine which t
 
 ![Render Streaming backend](images/websocket_signaling_inspector.png)
 
-If it starts with `http`, `HttpSignaling` is used. If it starts with `ws`, `WebSocketSignaling` is used.
+If it starts with `http`, `Http Signaling` is used. If it starts with `ws`, `WebSocket Signaling` is used.
 
 ```
 # launch server for HTTP

diff --git a/Documentation~/streaming-management.md b/Documentation~/streaming-management.md
@@ -2,7 +2,7 @@
 
 Unity Render Streaming automatically creates and destroys streams as needed. It also allows the developer to customize the mechanism for controlling the stream. For a description of components related to individual streams, see the section on components for streaming.
 
-## `RenderStreaming` component
+## [`RenderStreaming`](../api/Unity.RenderStreaming.RenderStreaming.html) component
 
 **Render Streaming** includes the following features.
 
@@ -28,7 +28,7 @@ Unity Render Streaming automatically creates and destroys streams as needed. It
 | **Handlers** | List of handlers of signaling process. | |
 | **Run On Awake** | Automatically started when called `Awake` method. | `True` |
 
-## `SingleConnection` component
+## [`SingleConnection`](../api/Unity.RenderStreaming.SingleConnection.html) component
 
 This component lists components that stream media or data via a single peer connection.
 
@@ -40,7 +40,7 @@ This component lists components that stream media or data via a single peer conn
 | --------- | ----------- | ------- |
 | **Streams** | List of streams | [] |
 
-## `Broadcast` component
+## [`Broadcast`](../api/Unity.RenderStreaming.SingleConnection.html) component
 
 This component lists components that stream media or data via multiple peer connections.
 

diff --git a/Documentation~/tutorial.md b/Documentation~/tutorial.md
@@ -40,7 +40,7 @@ Check Package Manager window, Click `+` button and select `Add package from git
 Input the string below to the input field.
 
 ```
-[email protected].2
+[email protected].3
 ```
 
 The list of version string is [here](https://github.com/Unity-Technologies/com.unity.renderstreaming/tags). In most cases, the latest version is recommended to use.

diff --git a/Documentation~/use-inputsystem.md b/Documentation~/use-inputsystem.md
@@ -10,9 +10,7 @@ You need to configure settings described below to use Input System. Note that di
 
 ### For Unity 2021.2
 
-1. Open **Package Manager** window and check Input System **version 1.1** installed.
-
-![Input System version 1.1](images/inputsystem-version.png)
+1. Open **Package Manager** window and check Input System version **1.3** installed.
 
 2. Open **Project Settings** window, and **Player > Resolution and Presentatoin**, and enable **Run In Background**.
 
@@ -28,7 +26,7 @@ You need to configure settings described below to use Input System. Note that di
 
 ### For Unity 2020.3 and 2019.4
 
-1. Check Input System version **1.0** installed.
+1. Check Input System version **1.3** installed.
 2. Open **Project Settings** window, and **Player > Resolution and Presentatoin**, and enable **Run In Background**.
 
 ![Enable Run In Background](images/enable_run_in_background.png)

diff --git a/Documentation~/video-streaming.md b/Documentation~/video-streaming.md
@@ -1,11 +1,11 @@
 # Video Streaming
 
-The base class for sending video is `VideoStreamSender`, and the base class for receiving video is `VideoStreamReceiver`. And various components are implemented using the inheritance relation. Components are provided for the sender and receiver sides.
+The base class for sending video is [`VideoStreamSender`](../api/Unity.RenderStreaming.VideoStreamSender.html), and the base class for receiving video is [`VideoStreamReceiver`](../api/Unity.RenderStreaming.VideoStreamReceiver.html). And various components are implemented using the inheritance relation. Components are provided for the sender and receiver sides.
 
 > [!NOTE]
 > The choice of video codec is currently limited. Use **H.264** if using a hardware encoder, or **VP8** if using a software encoder.
 
-## `VideoStreamSender` component
+## [`VideoStreamSender`](../api/Unity.RenderStreaming.VideoStreamSender.html) component
 
 This component sends the texture.
 
@@ -20,7 +20,7 @@ This component sends the texture.
 | **Depth Buffer** | The precision of the render texture's depth buffer in bits | No depth buffer |
 
 
-## `ScreenStreamSender` component
+## [`ScreenStreamSender`](../api/Unity.RenderStreaming.ScreenStreamSender.html) component
 
 This component sends the image of the main display.
 
@@ -34,7 +34,7 @@ This component sends the image of the main display.
 | **Anti-aliasing** | The antialiasing level for the RenderTexture | None |
 | **Depth Buffer** | The precision of the render texture's depth buffer in bits | No depth buffer |
 
-## `CameraStreamSender` component
+## [`CameraStreamSender`](../api/Unity.RenderStreaming.CameraStreamSender.html) component
 
 This component streams the `Camera` component's camera rendering results.  Uses `Target Texture` to store the rendering results.
 
@@ -52,9 +52,9 @@ This component streams the `Camera` component's camera rendering results.  Uses
 | **Anti-aliasing** | The antialiasing level for the RenderTexture | None |
 | **Depth Buffer** | The precision of the render texture's depth buffer in bits | No depth buffer |
 
-## `WebCamStreamSender` component
+## [`WebCamStreamSender`](../api/Unity.RenderStreaming.WebCamStreamSender.html) component
 
-This component streams the `WebCamTexture` rendering results. Please refer [WebCamTexture](https://docs.unity3d.com/ScriptReference/WebCamTexture.html) pages.
+This component streams the [WebCamTexture](https://docs.unity3d.com/ScriptReference/WebCamTexture.html) rendering results.
 
 ![WebCamStreamSender inspector](images/webcamstreamsender_inspector.png)
 
@@ -69,10 +69,10 @@ This component streams the `WebCamTexture` rendering results. Please refer [WebC
 | Parameter | Description | Default |
 | --------- | ----------- | ------- |
 | **Streaming Size** | Size of the frame buffer used for streaming | 1280, 720 |
-| **Device Index** | Index of `WebCamTexutre.devices` to use | 0 |
+| **Device Index** | Index of [`WebCamTexutre.devices`](https://docs.unity3d.com/ScriptReference/WebCamTexture-devices.html) to use | 0 |
 
 
-## `VideoStreamReceiver` component
+## [`VideoStreamReceiver`](../api/Unity.RenderStreaming.VideoStreamReceiver.html) component
 
 This component receives a videostream and exposes a texture that rendered the receiving frame buffer. 
 

diff --git a/Editor/PropertyDrawers/StreamingSizeDrawer.cs b/Editor/PropertyDrawers/StreamingSizeDrawer.cs
@@ -4,7 +4,7 @@
 namespace Unity.RenderStreaming.Editor
 {
     [CustomPropertyDrawer(typeof(StreamingSizeAttribute))]
-    public class StreamingSizeDrawer : PropertyDrawer
+    class StreamingSizeDrawer : PropertyDrawer
     {
         readonly GUIContent[] streamingSizeText =
         {

diff --git a/Editor/RenderStreamingEditor.cs b/Editor/RenderStreamingEditor.cs
@@ -58,16 +58,18 @@ static void ShowSignalingTypes(SerializedProperty signalingType){
             int i = 0;
 
             foreach (var assembly in System.AppDomain.CurrentDomain.GetAssemblies()){
-                foreach (System.Type type in assembly.GetTypes()) {
-                    if (type.IsVisible && type.IsClass && typeof(ISignaling).IsAssignableFrom(type)) {
-                        if(type.FullName == signalingType.stringValue){
-                            selected = i;
+                try { 
+                    foreach (System.Type type in assembly.GetTypes()) {
+                        if (type.IsVisible && type.IsClass && typeof(ISignaling).IsAssignableFrom(type)) {
+                            if(type.FullName == signalingType.stringValue){
+                                selected = i;
+                            }
+                            options.Add(type.Name);
+                            types.Add(type.FullName);
+                            i++;
                         }
-                        options.Add(type.Name);
-                        types.Add(type.FullName);
-                        i++;
                     }
-                }
+                } catch { }
             }
 
             selected = EditorGUILayout.Popup("Signaling Type", selected, options.ToArray());

diff --git a/Editor/WebAppDownloader.cs b/Editor/WebAppDownloader.cs
@@ -7,7 +7,7 @@ namespace Unity.RenderStreaming.Editor
     internal static class WebAppDownloader
     {
         const string URLRoot = "https://github.com/Unity-Technologies/UnityRenderStreaming";
-        const string LatestKnownVersion = "3.1.0-exp.2";
+        const string LatestKnownVersion = "3.1.0-exp.3";
 
         // TODO::fix release process of webserver runtime.
         const string FileNameWebAppForMac = "webserver_mac";