Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Poc #15

Merged
merged 30 commits into from
Aug 7, 2024
Merged

Poc #15

Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
2a17063
webrtc, rtmp, mp4
mat-hek May 10, 2024
541c098
add tests
mat-hek Jun 6, 2024
f338b2c
move protocol-specific stuff to separate modules
mat-hek Jun 14, 2024
f8b9830
small pipeline refactor
mat-hek Jun 17, 2024
86356e7
sole audio/video
mat-hek Jun 19, 2024
17930ad
satisfy credo and dialyzer
mat-hek Jun 21, 2024
a0b94a4
use mp4 from github
mat-hek Jun 21, 2024
eda08a9
support passing uri for mp4 and rtmp
mat-hek Jun 21, 2024
abc938c
add examples
mat-hek Jun 21, 2024
85f625c
use released rtp h264
mat-hek Jun 24, 2024
d976188
improve fixture comparisons
mat-hek Jun 24, 2024
81be637
improve async_test
mat-hek Jun 24, 2024
f54acb1
add more comments
mat-hek Jun 24, 2024
b3cfe5b
Apply suggestions from code review
mat-hek Jul 9, 2024
b0901b3
new rtmp api basic functionality works
bartkrak Jul 17, 2024
da15082
little refactor, format
bartkrak Jul 17, 2024
e97f190
code cleanup
bartkrak Jul 18, 2024
f292864
url parsing
bartkrak Jul 24, 2024
60ec898
rtmp server under utility supervisor
bartkrak Jul 24, 2024
e4435cf
ci fix
bartkrak Jul 24, 2024
08a557d
dialyzer fix
bartkrak Jul 24, 2024
9b28379
replace lists with tuples for i/o spec
mat-hek Jul 29, 2024
7401a91
small fixes
bartkrak Jul 31, 2024
64d9955
lint fix
bartkrak Jul 31, 2024
13870e1
Merge pull request #12 from membraneframework-labs/use_new_rtmp_api
bartkrak Jul 31, 2024
3d43968
template -> boombox
mat-hek Jul 31, 2024
8812bd4
fix rtmp dep
mat-hek Jul 31, 2024
5f6c646
update examples
mat-hek Jul 31, 2024
72f56ad
use mp4 from WIP branch, as it doesn't work with isom-avc3 branch due…
mat-hek Aug 2, 2024
aebd077
address CR
mat-hek Aug 7, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,20 @@ workflows:
build:
jobs:
- elixir/build_test:
cache-version: 3
filters: &filters
tags:
only: /v.*/
- elixir/test:
cache-version: 3
filters:
<<: *filters
- elixir/lint:
cache-version: 3
filters:
<<: *filters
- elixir/hex_publish:
cache-version: 3
requires:
- elixir/build_test
- elixir/test
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/fetch_changes.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jobs:
# Runs a set of commands using the runners shell
- name: Add remote
run: |
git remote add source [email protected]:membraneframework/membrane_template_plugin.git
git remote add source [email protected]:membraneframework/boombox.git
git remote update
echo "CURRENT_BRANCH=$(git branch --show-current)" >> $GITHUB_ENV
Expand Down
24 changes: 10 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,35 +1,31 @@
# Membrane Template Plugin
# Boombox

[![Hex.pm](https://img.shields.io/hexpm/v/membrane_template_plugin.svg)](https://hex.pm/packages/membrane_template_plugin)
[![API Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/membrane_template_plugin)
[![CircleCI](https://circleci.com/gh/membraneframework/membrane_template_plugin.svg?style=svg)](https://circleci.com/gh/membraneframework/membrane_template_plugin)
[![Hex.pm](https://img.shields.io/hexpm/v/boombox.svg)](https://hex.pm/packages/boombox)
[![API Docs](https://img.shields.io/badge/api-docs-yellow.svg?style=flat)](https://hexdocs.pm/boombox)
[![CircleCI](https://circleci.com/gh/membraneframework/boombox.svg?style=svg)](https://circleci.com/gh/membraneframework/boombox)

This repository contains a template for new plugins.

Check out different branches for other flavors of this template.

It's a part of the [Membrane Framework](https://membrane.stream).
Boombox is a powerful tool for audio & video streaming based on the [Membrane Framework](https://membrane.stream).

## Installation

The package can be installed by adding `membrane_template_plugin` to your list of dependencies in `mix.exs`:
The package can be installed by adding `boombox` to your list of dependencies in `mix.exs`:

```elixir
def deps do
[
{:membrane_template_plugin, "~> 0.1.0"}
{:boombox, "~> 0.1.0"}
]
end
```

## Usage

TODO
See `examples.livemd` for usage examples.

## Copyright and License

Copyright 2020, [Software Mansion](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=membrane_template_plugin)
Copyright 2020, [Software Mansion](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=boombox)

[![Software Mansion](https://logo.swmansion.com/logo?color=white&variant=desktop&width=200&tag=membrane-github)](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=membrane_template_plugin)
[![Software Mansion](https://logo.swmansion.com/logo?color=white&variant=desktop&width=200&tag=membrane-github)](https://swmansion.com/?utm_source=git&utm_medium=readme&utm_campaign=boombox)

Licensed under the [Apache License, Version 2.0](LICENSE)
126 changes: 126 additions & 0 deletions examples.livemd
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
# Boombox examples

```elixir
Logger.configure(level: :info)

# For ffmpeg and ffplay commands to work on Mac Livebook Desktop
System.put_env("PATH", "/opt/homebrew/bin:#{System.get_env("PATH")}")

Mix.install([{:boombox, path: __DIR__}, :kino])
```

## Boombox

```elixir
:ok = :inets.start()

{:ok, _server} =
:inets.start(:httpd,
bind_address: ~c"localhost",
port: 1234,
document_root: ~c"#{__DIR__}/examples_assets/",
server_name: ~c"assets_server",
server_root: "/tmp",
erl_script_nocache: true
)
```

```elixir
bbb_mp4 = "#{__DIR__}/test/fixtures/bun10s.mp4"
out_dir = "#{__DIR__}/examples_outputs"
```

<!-- livebook:{"branch_parent_index":0} -->

## MP4 to WebRTC

To receive the stream, visit http://localhost:1234/stream_to_browser/index.html after running the cell below

```elixir
Boombox.run(input: bbb_mp4, output: {:webrtc, "ws://localhost:8830"})
```

<!-- livebook:{"branch_parent_index":0} -->

## WebRTC to MP4

To send the stream, visit http://localhost:1234/stream_from_browser/index.html

```elixir
Boombox.run(input: {:webrtc, "ws://localhost:8829"}, output: "#{out_dir}/webrtc_to_mp4.mp4")
```

```elixir
System.shell("ffplay #{out_dir}/webrtc_to_mp4.mp4")
```

<!-- livebook:{"branch_parent_index":0} -->

## WebRTC to WebRTC

Visit http://localhost:1234/stream_from_browser/index.html to send the stream and http://localhost:1234/stream_to_browser/index.html to receive it

```elixir
Boombox.run(input: {:webrtc, "ws://localhost:8829"}, output: {:webrtc, "ws://localhost:8830"})
```

<!-- livebook:{"branch_parent_index":0} -->

## RTMP to MP4

```elixir
uri = "rtmp://localhost:5432"

t =
Task.async(fn ->
Boombox.run(input: uri, output: "#{out_dir}/rtmp_to_mp4.mp4")
end)

{_output, 0} = System.shell("ffmpeg -re -i #{bbb_mp4} -c copy -f flv #{uri}")

Task.await(t)
```

```elixir
System.shell("ffplay #{out_dir}/rtmp_to_mp4.mp4")
```

<!-- livebook:{"branch_parent_index":0} -->

## RTMP to WebRTC

To receive the stream, visit http://localhost:1234/stream_to_browser/index.html

```elixir
uri = "rtmp://localhost:5432"

t =
Task.async(fn ->
Boombox.run(input: uri, output: {:webrtc, "ws://localhost:8830"})
end)

{_output, 0} = System.shell("ffmpeg -re -i #{bbb_mp4} -c copy -f flv #{uri}")

Task.await(t)
```

<!-- livebook:{"branch_parent_index":0} -->

## MP4 via WebRTC to MP4

```elixir
signaling = Membrane.WebRTC.SignalingChannel.new()

t =
Task.async(fn ->
Boombox.run(input: bbb_mp4, output: {:webrtc, signaling})
end)

Boombox.run(input: {:webrtc, signaling}, output: "#{out_dir}/mp4_webrtc_mp4.mp4")

Task.await(t)
```

```elixir
System.shell("ffplay #{out_dir}/mp4_webrtc_mp4.mp4")
```
20 changes: 20 additions & 0 deletions examples_assets/stream_from_browser/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
<!DOCTYPE html>
<html lang="en">

<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">
<title>Membrane WebRTC browser to file example</title>
</head>

<body
style="background-color: black; color: white; font-family: Arial, Helvetica, sans-serif; min-height: 100vh; margin: 0px; padding: 5px 0px 5px 0px">
<main>
<h1>Membrane WebRTC browser to file example</h1>
<div id="status">Connecting</div>
</main>
<script src="stream_from_browser.js"></script>
</body>

</html>
58 changes: 58 additions & 0 deletions examples_assets/stream_from_browser/stream_from_browser.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
const pcConfig = { 'iceServers': [{ 'urls': 'stun:stun.l.google.com:19302' },] };
const mediaConstraints = { video: true, audio: true }

const ws = new WebSocket(`ws://localhost:8829`);
const connStatus = document.getElementById("status");
ws.onopen = _ => start_connection(ws);
ws.onclose = event => {
connStatus.innerHTML = "Disconnected"
console.log("WebSocket connection was terminated:", event);
}

const start_connection = async (ws) => {
const localStream = await navigator.mediaDevices.getUserMedia(mediaConstraints);
const pc = new RTCPeerConnection(pcConfig);

pc.onicecandidate = event => {
if (event.candidate === null) return;
console.log("Sent ICE candidate:", event.candidate);
ws.send(JSON.stringify({ type: "ice_candidate", data: event.candidate }));
};

pc.onconnectionstatechange = () => {
if (pc.connectionState == "connected") {
const button = document.createElement('button');
button.innerHTML = "Disconnect";
button.onclick = () => {
ws.close();
localStream.getTracks().forEach(track => track.stop())
}
connStatus.innerHTML = "Connected ";
connStatus.appendChild(button);
}
}

for (const track of localStream.getTracks()) {
pc.addTrack(track, localStream);
}

ws.onmessage = async event => {
const { type, data } = JSON.parse(event.data);

switch (type) {
case "sdp_answer":
console.log("Received SDP answer:", data);
await pc.setRemoteDescription(data);
break;
case "ice_candidate":
console.log("Recieved ICE candidate:", data);
await pc.addIceCandidate(data);
break;
}
};

const offer = await pc.createOffer();
await pc.setLocalDescription(offer);
console.log("Sent SDP offer:", offer)
ws.send(JSON.stringify({ type: "sdp_offer", data: offer }));
};
20 changes: 20 additions & 0 deletions examples_assets/stream_to_browser/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
<!DOCTYPE html>
<html lang="en">

<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="X-UA-Compatible" content="ie=edge">
<title>Membrane WebRTC file to browser example</title>
</head>

<body
style="background-color: black; color: white; font-family: Arial, Helvetica, sans-serif; min-height: 100vh; margin: 0px; padding: 5px 0px 5px 0px">
<main>
<h1>Membrane WebRTC file to browser example</h1>
<video id="videoPlayer" controls muted autoplay></video>
</main>
<script src="stream_to_browser.js"></script>
</body>

</html>
37 changes: 37 additions & 0 deletions examples_assets/stream_to_browser/stream_to_browser.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
const videoPlayer = document.getElementById("videoPlayer");
const pcConfig = { 'iceServers': [{ 'urls': 'stun:stun.l.google.com:19302' },] };
const proto = window.location.protocol === "https:" ? "wss:" : "ws:"
const ws = new WebSocket(`${proto}//localhost:8830`);
ws.onopen = () => start_connection(ws);
ws.onclose = event => console.log("WebSocket connection was terminated:", event);

const start_connection = async (ws) => {
videoPlayer.srcObject = new MediaStream();

const pc = new RTCPeerConnection(pcConfig);
pc.ontrack = event => videoPlayer.srcObject.addTrack(event.track);
pc.onicecandidate = event => {
if (event.candidate === null) return;

console.log("Sent ICE candidate:", event.candidate);
ws.send(JSON.stringify({ type: "ice_candidate", data: event.candidate }));
};

ws.onmessage = async event => {
const { type, data } = JSON.parse(event.data);

switch (type) {
case "sdp_offer":
console.log("Received SDP offer:", data);
await pc.setRemoteDescription(data);
const answer = await pc.createAnswer();
await pc.setLocalDescription(answer);
ws.send(JSON.stringify({ type: "sdp_answer", data: answer }));
console.log("Sent SDP answer:", answer)
break;
case "ice_candidate":
console.log("Recieved ICE candidate:", data);
await pc.addIceCandidate(data);
}
};
};
3 changes: 3 additions & 0 deletions examples_outputs/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
*
!.gitkeep
!.gitignore
Empty file added examples_outputs/.gitkeep
Empty file.
26 changes: 26 additions & 0 deletions lib/boombox.ex
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
defmodule Boombox do
@moduledoc """
Boombox is a tool for audio and video streaming.

See `t:input/0` and `t:output/0` for supported protocols.
"""
@type webrtc_opts :: Membrane.WebRTC.SignalingChannel.t() | URI.t()

@type input ::
URI.t()
| Path.t()
| {:file, :mp4, Path.t()}
| {:webrtc, webrtc_opts()}
| {:rtmp, URI.t()}
@type output :: URI.t() | Path.t() | {:file, :mp4, Path.t()} | {:webrtc, webrtc_opts()}

@spec run(input: input, output: output) :: :ok
def run(opts) do
{:ok, supervisor, _pipeline} = Membrane.Pipeline.start_link(Boombox.Pipeline, opts)
Process.monitor(supervisor)

receive do
{:DOWN, _monitor, :process, ^supervisor, _reason} -> :ok
end
end
end
Loading