-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
## [3.1.0-exp.2] - 2021-12-01 ### Added - Added audio streaming receiver. - Added the `Multiplay` scene into samples. - Added components for streaming. - `MicrophoneStreamSender` - `AudioStreamReceiver` - `ScreenStreamSender` - Added components for controlling user input. - `InputSender` - `InputReceiver` ### Changed - Upgraded WebRTC package to `2.4.0-exp.4`. - Upgraded Input System package to `1.0.2`. - Renamed components. - `CameraStreamer` -> `CameraStreamSender` - `ReceiveVideoViewer` -> `VideoStreamReceiver` - `WebCamStreamer` -> `WebCamStreamSender` - `AudioStreamer` -> `AudioStreamSender` - `InputSystemChannnelSender` -> `InputSender` - `InputSystemChannnelReceiver` -> `InputReceiver` - Improved controlling user input from browsers. - Updated manual documents. ### Fixed - Fixed resource leak of sender stream tracks.
- Loading branch information
Unity Technologies
committed
Dec 1, 2021
1 parent
aacf8f7
commit 0e9bb06
Showing
236 changed files
with
15,175 additions
and
6,307 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| # Audio Streaming | ||
|
|
||
| `AudioStreamSender` class and `AudioStreamReceiver` class are the base classes for sending and receiving audio. And various components are implemented using inheritance relationships with these class. Components are provided for the sender and receiver. | ||
|
|
||
| > [!NOTE] | ||
| > There is currently a limitation in the audio codec selection. Currently, only **Opus** is used for the audio codec. | ||
| ## `AudioStreamSender` component | ||
|
|
||
| This component streams the audio rendering results from [`AudioListener`](https://docs.unity3d.com/ScriptReference/AudioListener.html) component or [`AudioSource`](https://docs.unity3d.com/ScriptReference/AudioSource.html) component. | ||
|
|
||
|  | ||
|
|
||
| ### Properties | ||
|
|
||
| | Parameter | Description | Default | | ||
| | --------- | ----------- | ------- | | ||
| | **Audio Source** | [`AudioSource`](https://docs.unity3d.com/ScriptReference/AudioSource.html) instance for sending audio | None | | ||
|
|
||
| ## `MicrophoneStreamSender` component | ||
|
|
||
| This component streams the audio rendering results from [`Microphone`](https://docs.unity3d.com/ScriptReference/Microphone.html). | ||
|
|
||
|  | ||
|
|
||
| ### Properties | ||
|
|
||
| | Parameter | Description | Default | | ||
| | --------- | ----------- | ------- | | ||
| | **Audio Source** | [`AudioSource`](https://docs.unity3d.com/ScriptReference/AudioSource.html) instance for sending audio | None | | ||
| | **Device Index** | An index of the list of available microphone devices | 0 | | ||
|
|
||
| An index of the list of available microphone devices | ||
|
|
||
|
|
||
| ## `AudioStreamReceiver` component | ||
|
|
||
| The components receive audio track stream and rendering to [`AudioSource`](https://docs.unity3d.com/ScriptReference/AudioSource.html). | ||
|
|
||
|  |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,153 +1,8 @@ | ||
| # Components Settings | ||
|
|
||
| ## `Render Streaming` | ||
| This package provides some useful components to make the streaming feature to the application made with Unity. | ||
|
|
||
| **Render Streaming** includes the following features. | ||
|
|
||
| - Communication with signaling servers | ||
| - Streaming input from cameras | ||
| - Associating browser input with events | ||
|
|
||
|  | ||
|
|
||
| ### Properties | ||
|
|
||
| | Parameter | Description | Default | | ||
| | ---------------------------- | -------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | | ||
| | **Signaling Type** | Class type of the signaling | `WebSocketSignaling` | | ||
| | **Signaling URL** | Set the signaling server URL <br>If you use WebSocket as a signaling protocol, you should specify a URL starting with `ws` or `wss`. | `ws://localhost` | | ||
| | **Ice Servers** | Set a list of STUN/TURN servers | | | ||
| | *Urls* | Set URLs of STUN/TURN servers | `[stun:stun.l.google.com:19302\]` | | ||
| | *Username* | The username to use when logging into the TURN server | | | ||
| | *Credencial* | The credential to use when logging into the TURN server | | | ||
| | *Credencial Type* | This attribute specifies what kind of `credential` is to be used when connecting to the TURN server | `Password` | | ||
| | **Interval** | Set the polling frequency (in seconds) to the signaling server <br>Valid only if you are using Http as the signaling protocol. | 5 (sec) | | ||
| | **Hardware Encoder Support** | Enable the checkbox to make this video encoding replace values that use a [hardware encoder](index.md#hardware-encoder) | `False` | | ||
| | **Handlers** | Register browser button click events | | | ||
| | **Run On Awake** | Automatically started when called `Awake` method | `True` | | ||
|
|
||
| ## `CameraStreamer` | ||
|
|
||
| This component streams the `Camera` component's camera rendering results. Uses `Target Texture` to store the rendering results. | ||
|
|
||
| > [!NOTE] | ||
| > You can attach the `Target Texture` to the `Camera` component. | ||
| > If `Target Texture` is attached on Camera, use that `Render Texture` setting first. | ||
|  | ||
|
|
||
| ### Properties | ||
|
|
||
| | Parameter | Description | Default | | ||
| | ---------------------------- | ------------------------------------------------------------------- | ------------------------------- | | ||
| | **Streaming Size** | Size of the frame buffer used for streaming | 1280, 720 | | ||
| | **Anti-aliasing** | The antialiasing level for the RenderTexture | None | | ||
| | **Depth Buffer** | The precision of the render texture's depth buffer in bits | No depth buffer | | ||
|
|
||
| ## `WebCamStreamer` | ||
|
|
||
| This component streams the `WebCamTexture` rendering results. Please refer [WebCamTexture](https://docs.unity3d.com/ScriptReference/WebCamTexture.html) pages. | ||
|
|
||
|  | ||
|
|
||
| > [!NOTE] | ||
| > When building application for iOS platform, you need set `Camera Usage Description` on Player Settings. | ||
| > If not set this, your application exits. (refer [this page](https://developer.apple.com/library/archive/documentation/General/Reference/InfoPlistKeyReference/Articles/CocoaKeys.html#//apple_ref/doc/uid/TP40009251-SW24)) | ||
|  | ||
|
|
||
| ### Properties | ||
|
|
||
| | Parameter | Description | Default | | ||
| | ---------------------------- | ------------------------------------------------------------------- | ------------------------------- | | ||
| | **Streaming Size** | Size of the frame buffer used for streaming | 1280, 720 | | ||
| | **Device Index** | Index of `WebCamTexutre.devices` to use | 0 | | ||
|
|
||
| ## `AudioStreamer` | ||
|
|
||
| This component streams the `AudioListener` component's audio rendering results. | ||
|
|
||
| ### Properties | ||
|
|
||
| None | ||
|
|
||
| ## `ReceiveVideoViewer` | ||
|
|
||
| This component receives a videostream and exposes a texture that rendered the receiving frame buffer. | ||
|
|
||
|  | ||
|
|
||
| ### Properties | ||
|
|
||
| | Parameter | Description | Default | | ||
| | ---------------------------- | ------------------------------------------------------------------- | ------------------------------- | | ||
| | **Streaming Size** | Size of the frame buffer used for streaming | 1280, 720 | | ||
| | **Connection Id** | ID of receiving videostream | Empty | | ||
|
|
||
| ## `InputSystemChannelSender` | ||
|
|
||
| This component sends input events from Unity. | ||
|
|
||
|  | ||
|
|
||
| ### Properties | ||
|
|
||
| | Parameter | Description | Default | | ||
| | ---------------------------- | ------------------------------------------------------------------- | ------------------------------- | | ||
| | **Local** | Channel is local or remote | `False` | | ||
| | **Label** | Channel label | Empty | | ||
|
|
||
| ## `InputSystemChannelReceiver` | ||
|
|
||
| This component receives input events from Unity. | ||
|
|
||
|  | ||
|
|
||
|
|
||
| ### Properties | ||
|
|
||
| | Parameter | Description | Default | | ||
| | ---------------------------- | ------------------------------------------------------------------- | ------------------------------- | | ||
| | **Local** | Channel is local or remote | `False` | | ||
| | **Label** | Channel label | Empty | | ||
|
|
||
| ## `WebBrowserInputChannelReceiver` | ||
|
|
||
| This component receives input events from web browsers. | ||
|
|
||
|  | ||
|
|
||
|
|
||
| ### Properties | ||
|
|
||
| | Parameter | Description | Default | | ||
| | ---------------------------- | ------------------------------------------------------------------- | ------------------------------- | | ||
| | **Local** | Channel is local or remote | `False` | | ||
| | **Label** | Channel label | Empty | | ||
| | **Array Button Click Event** | Register browser button click events | [] | | ||
| | *Element Id* | Set an ID that is sent from browsers | | | ||
| | *Click* | Set button click event | | | ||
|
|
||
| ## `SingleConnection` | ||
|
|
||
| This component lists components that stream media or data via a single peer connection. | ||
|
|
||
|  | ||
|
|
||
| ### Properties | ||
|
|
||
| | Parameter | Description | Default | | ||
| | ---------------------------- | ------------------------------------------------------------------- | ------------------------------- | | ||
| | **Streams** | List of streams | [] | | ||
|
|
||
| ## `Broadcast` | ||
|
|
||
| This component lists components that stream media or data via multiple peer connections. | ||
|
|
||
|  | ||
|
|
||
| ### Properties | ||
|
|
||
| | Parameter | Description | Default | | ||
| | ---------------------------- | ------------------------------------------------------------------- | ------------------------------- | | ||
| | **Streams** | List of streams | [] | | ||
| - [Streaming Management](streaming-management.md) | ||
| - [Video Streaming](video-streaming.md) | ||
| - [Audio Streaming](audio-streaming.md) | ||
| - [Data Streaming](data-streaming.md) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,83 @@ | ||
| # Setting Up Device Input | ||
|
|
||
| To control the camera on the remote PC, we need to link the input events on web browsers to the actions in Unity. **Input System** makes it easy to implement the input processing. | ||
|
|
||
| ## Adding components for input processing | ||
|
|
||
| Open the Unity scene which created in [the previous page](create-camera.md), and select **Main Camera** in the Hierarchy window. Add the **Input Receiver** component in the Inspector window. | ||
|
|
||
|  | ||
|
|
||
| The initial condition of the inspector of **Input Receiver** is below. | ||
|
|
||
|  | ||
|
|
||
| ## Setting Input Actions | ||
|
|
||
| We will use the [**Input Actions**](https://docs.unity3d.com/Packages/[email protected]/manual/Actions.html) feature of the Input System. Input Actions is a mechanism for mapping various inputs to Unity actions, and it provides editing tools that make it easy for anyone to use. We will use Input Actions to define mouse actions. | ||
|
|
||
| Right-click in the project window, and then click **Create > Input Actions**. | ||
|
|
||
|  | ||
|
|
||
| The asset will be added to the project window and rename to **Control**. Double-click on the **Control** asset in the project window, and the **Action editor** will appear. | ||
|
|
||
|  | ||
|
|
||
| Click the **+** button in **Actions Maps** to add an item. | ||
|
|
||
|  | ||
|
|
||
| Rename the **Actions** item to `Look`. | ||
|
|
||
|  | ||
|
|
||
| Select the item and change **Action Type** to `Value`, and **Control Type** to `Vector2`. | ||
|
|
||
|  | ||
|
|
||
| Next, select the **\<No Binding\>** item and add **Mouse > Delta** to the **Path** of **Binding**. | ||
|
|
||
|  | ||
|
|
||
| The result will look like this. Note that the last step is to press the **Save Asset** button to save your edits. | ||
|
|
||
|  | ||
|
|
||
| ## Setting `InputReceiver` component | ||
|
|
||
| Drag the **Control** asset to the **Input Actions** property of the `InputReceiver` component. | ||
|
|
||
|  | ||
|
|
||
| Add the code to control the rotation of the camera. Move to the Inspector window, click **Add component** button and select **New Script** Script to create the asset. Change the asset name to `PlayerControler`. | ||
|
|
||
|  | ||
|
|
||
| The contents of the script will be as follows. | ||
|
|
||
| ```csharp | ||
| using UnityEngine; | ||
| using UnityEngine.InputSystem; | ||
|
|
||
| public class PlayerController : MonoBehaviour | ||
| { | ||
| public void Look(InputAction.CallbackContext value) | ||
| { | ||
| Vector2 input = value.ReadValue<Vector2>(); | ||
| transform.eulerAngles += new Vector3(-input.y, input.x, 0); | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| Once you have copied the script and saved the file, assign `PlayerController.Look` to the **InputReceiver** component's **Look** event. | ||
|
|
||
|  | ||
|
|
||
| ## Checking on browsers | ||
|
|
||
| The mouse operation in the browser should be reflected in the rotation of the camera. If the camera is not moved, please check [the settings](use-inputsystem.md). | ||
|
|
||
| ## Next step | ||
|
|
||
| We were able to incorporate the input processing using the Input Actions feature. If you would like to learn more about the features of Unity Render Streaming, please see the [Sample page](samples.md). Also, check out [Customizing your web application](customize-webapp.md) for information on how to modify the web user interface. |
Oops, something went wrong.