Skip to content

Commit

Permalink
Update the readme to emphasize how to get the http UI working with a …
Browse files Browse the repository at this point in the history
…remote server.
  • Loading branch information
LaurentMazare committed Sep 23, 2024
1 parent b20815e commit f00e391
Show file tree
Hide file tree
Showing 2 changed files with 26 additions and 5 deletions.
16 changes: 16 additions & 0 deletions FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,19 @@ We only use a fixed buffer, and we do not discard past entries.
The PyTorch version should work for unlimited times, although this is mostly untested and we
expect the quality to degrade after a bit (we have no attention sink or other mechanism to improve the streaming
beyond the finite context used at training).

### The server seems to be running but nothing happens on connect.

For diagnosis, look at your browser console if there is any error being
reported.

If you see issues that look like the following:
```
Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'addModule')
```
this is likely caused by the http server being remote and audio being disabled
for http in such a case.

To get around this, tunnel the 8998 port from the remote server to the localhost
via ssh and access [localhost:8998](http://localhost:8998) via http normally
after that.
15 changes: 10 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,11 +113,16 @@ Start the server with:
python -m moshi.server [--gradio-tunnel] [--hf-repo kyutai/moshika-pytorch-bf16]
```

And then access the web UI on [localhost:8998](http://localhost:8998). If your GPU is on a distant machine
with no direct access, `--gradio-tunnel` will create a tunnel with a URL accessible from anywhere.
Keep in mind that this tunnel goes through the US and can add significant latency (up to 500ms from Europe).
You can use `--gradio-tunnel-token` to set a fixed secret token and reuse the same address over time.
Alternatively, you might want to use SSH to redirect your connection.
And then access the web UI on [localhost:8998](http://localhost:8998).
If your GPU is on a distant machine this will not work as websites using http
are not allowed to use the audio worklet api. There are two ways to get around
this:
- Forward the remote 8998 port to your localhost using ssh `-L` flag. Then
connects to [localhost:8998](http://localhost:8998) as mentionned previously.
- Use the `--gradio-tunnel` argument, this sets up a tunnel with a URL accessible from anywhere.
Keep in mind that this tunnel goes through the US and can add significant
latency (up to 500ms from Europe). You can use `--gradio-tunnel-token` to set a
fixed secret token and reuse the same address over time.

You can use `--hf-repo` to select a different pretrained model, by setting the proper Hugging Face repository.

Expand Down

0 comments on commit f00e391

Please sign in to comment.