Running text-generation-webui on a VPS server login is at talk.domain.com

[ResearchForumOnline]

Hello, i have managed to setup text-generation-webui on my domain but the SSL is not working correctly, I am most likely going to fix this issue, I will share info on how you can do the same, I was going to install a gradio server but would be cheaper to do it on the server where oobabooga/text-generation-webui is installed, I have done it but no SSL, anyone achieved putting this system on a domain with SSL?

What ports need to be open etc? Using nginx certbot and my nginx.conf is:

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
worker_connections

1024;
}

http {
# ... other directives (if any)

server {
    listen 7860 ssl;
    server_name something.domain.org;

    ssl_certificate /etc/letsencrypt/live/something.domain.org/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/something.domain.org/privkey.pem;

    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers

ENCRYPTION-Key Exchange Algorithm-IS-HERE-OR-JUST-REMOVE-THIS-LINE-RSA-AES256-GCM-SHA384;

ssl_prefer_server_ciphers

on;
ssl_session_cache shared:SSL:10m;

    location / {
        proxy_pass http://localhost:7861;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

}

More info:

(base) zero@zero:~/text-generation-webui-main$
I've analyzed the output of the openssl s_client command, and here's the key information:

Connection Refused: The error message Connection refused (errno=111) indicates that there's no process listening on port 7861 to accept incoming HTTPS connections.
Potential Causes:

Chatbot Script Not Running: As you mentioned, the script might not be running.
Incorrect Port in Nginx: The Nginx configuration might be listening on a different port for HTTPS.
Firewall Blocking Port: Firewall rules might be preventing access to port 7861.
Chatbot Server Configuration: If the chatbot server is a separate process, it might not be configured to listen for HTTPS connections on port 7861.
Troubleshooting Steps:

Start Chatbot Script: If it's not already running, start the chatbot script and ensure it's configured to listen on port 7861 for HTTPS.
Verify Nginx Configuration: Double-check the Nginx configuration to confirm that it's listening on port 7861 for HTTPS and that the server_name and SSL certificate paths are correct.
Review Firewall Rules: Examine any firewall rules that might be blocking port 7861 and adjust them if necessary.
Check Chatbot Server Configuration: If the chatbot server is a separate process, verify that it's configured to listen for HTTPS connections on port 7861.
After addressing these potential causes, retry the openssl s_client command and attempt to access the chatbot through HTTPS. If the issue persists, provide more details about the outcomes of these steps and any specific error messages you encounter.

[ResearchForumOnline]

local host managing is working fine but the URL keeps saying:
Error
Connection errored out. it works but keeps saying Error
Connection errored out when I click save or load etc.

(base) zero@zero:/text-generation-webui-main$ sudo ./start_linux.sh --listen --gradio-auth username:MYPASSWORD --extension LLM_Web_search --listen-port 7860
[sudo] password for zero:
13:06:40-983064 INFO Starting Text generation web UI
13:06:40-985879 INFO Loading settings from settings.yaml
13:06:40-988806 INFO Loading the extension "LLM_Web_search"
13:06:42-776605 INFO Loading the extension "gallery"
╭───────────────────────────────────────────────────────── Traceback (most recent call last) ─────────────────────────────────────────────────────────╮
│ /home/zero/text-generation-webui-main/server.py:254 in │
│ │
│ 253 # Launch the web UI │
│ ❱ 254 create_interface() │
│ 255 while True: │
│ │
│ /home/zero/text-generation-webui-main/server.py:162 in create_interface │
│ │
│ 161 with OpenMonkeyPatch(): │
│ ❱ 162 shared.gradio['interface'].launch( │
│ 163 prevent_thread_lock=True, │
│ │
│ /home/zero/text-generation-webui-main/installer_files/env/lib/python3.11/site-packages/gradio/blocks.py:2033 in launch │
│ │
│ 2032 server, │
│ ❱ 2033 ) = networking.start_server( │
│ 2034 self, │
│ │
│ /home/zero/text-generation-webui-main/installer_files/env/lib/python3.11/site-packages/gradio/networking.py:207 in start_server │
│ │
│ 206 else: │
│ ❱ 207 raise OSError( │
│ 208 f"Cannot find empty port in range: {min(server_ports)}-{max(server_ports)}. │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
OSError: Cannot find empty port in range: 7860-7860. You can specify a different port by setting the GRADIO_SERVER_PORT environment variable or passing
the server_port parameter to launch().
(base) zero@zero:/text-generation-webui-main$

[ResearchForumOnline]

command used:
sudo ./start_linux.sh --listen --gradio-auth username:mypassword --extension LLM_Web_search --listen-port 7860

current nginx.conf which is pointing to zero.talktoai.org

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
worker_connections

1024;
}

http {
# ... other directives (if any)

server {
    listen 443 ssl;
    server_name zero.talktoai.org;  # Replace with your actual domain name

    ssl_certificate /etc/letsencrypt/live/zero.talktoai.org/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/zero.talktoai.org/privkey.pem;

    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers GET-THE-AI-TO-EDIT-THIS-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384;  # Adjust cip>
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m;

    location / {
        proxy_pass http://localhost:7860;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

}

But I got those errors, will keep trying to fix.

[ResearchForumOnline]

I am so close to making it work, please give any advice.

[ResearchForumOnline]

(base) zero@zero:~/text-generation-webui-main$ sudo ./start_linux.sh --gradio-auth username:mypassword --extension LLM_Web_search --listen --listen-host zero.talktoai.org
20:03:22-458009 INFO Starting Text generation web UI
20:03:22-460439 INFO Loading settings from settings.yaml
20:03:22-463518 INFO Loading the extension "LLM_Web_search"
20:03:24-420659 INFO Loading the extension "gallery"
Running on local URL: http://zero.talktoai.org:7860/

To create a public link, set share=True in launch().
WARNING: Invalid HTTP request received.
WARNING: Invalid HTTP request received.
WARNING: Invalid HTTP request received.
WARNING: Invalid HTTP request received.
WARNING: Invalid HTTP request received.
WARNING: Invalid HTTP request received.

[ResearchForumOnline]

Fixed: use firefox browser! Well, i thought so only works on the same server IP (locally) using firefox with the domain URL, but not working if I try access outside the server where this software is running.

[ResearchForumOnline]

Fixed, notes below:

working 100% on linux ubuntu:

cd /home/zero/text-generation-webui-main/

./start_linux.sh --gradio-auth username:mypass --extension LLM_Web_search --listen --listen-port 7860 --listen-host zero.talktoai.org --ssl-certfile=/etc/letsencrypt/live/zero.talktoai.org/fullchain.pem --ssl-keyfile=/etc/letsencrypt/live/zero.talktoai.org/privkey.pem

https://zero.talktoai.org:7860/

nginx.conf file:

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
worker_connections

1024;
}

http {

... other directives (if any)

server {
listen 443 ssl;
server_name zero.talktoai.org; # Replace with your actual domain name

ssl_certificate /etc/letsencrypt/live/zero.talktoai.org/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/zero.talktoai.org/privkey.pem;

ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers GET-THE-AI-TO-EDIT-THIS-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384;  # Adjust cip>
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:10m;

location / {
    proxy_pass http://localhost:7860;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}

}
}

[ResearchForumOnline]

Remember to check ufw firewall and add ports etc.
sudo allow from 7860 TCP
sudo allow from 7860
etc.
Works on chrome and firefox no issues.

[demetera]

For me it worked with adding these lines to nginx server location section:

proxy_pass http://localhost:7861;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

after that, receive the certificate and launch the server with:

Minimal setup with no auth:
./start_linux.sh --listen --listen-port 7861

With authentication:
./start_linux.sh --gradio-auth user:pass --listen --listen-port 7861