Skip to content

A multi-modal style conversational AI assistant ๐Ÿ˜‡ based on LLM.

License

Notifications You must be signed in to change notification settings

dayunyan/EchoChronos

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

54 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

EchoChronos ๐Ÿฅฐ

Description

EchoChronos (Time Echo Chronos) is a multi-modal style conversational AI assistant based on a large language model, designed to provide users with a new style of conversational experience. The AI integrates RAG, TTS, and other technologies to enable real-time interaction with users, allowing them to immerse themselves in the charm of classic dialogues and dialogues with history. ๐Ÿฅธ

lindaiyu

Software Architecture

EchoChronos
โ”œโ”€ ChatStyle  # Conversation style module
โ”œโ”€ managers   # Used to provide interfaces for various modules
โ”‚  โ”œโ”€ __init__.py
โ”‚  โ”œโ”€ connect.py  # Connection manager, currently only supports WebSocket
โ”‚  โ”œโ”€ constants.py  # Constants
โ”‚  โ”œโ”€ model.py  # Style dialogue model manager
โ”‚  โ”œโ”€ rag.py  # RAG model manager
โ”‚  โ”œโ”€ runner.py  # Runner manager, used for writing inference logic
โ”‚  โ””โ”€ tts.py  # TTS model manager
โ”œโ”€ RAG  # RAG module
โ”œโ”€ TTS  # TTS module
โ”œโ”€ utils  # Toolkit
โ”œโ”€ inference_torch.py  # Inference code using PyTorch
โ”œโ”€ inference.py  # Inference code using MindSpore
โ”œโ”€ launch.py  # Project entry point
โ”œโ”€ README.en.md
โ””โ”€ README.md

Installation

python>=3.10

Install ffmpeg

conda install ffmpeg

Install MindNLP

pip install mindnlp==0.4.0

Install ๅ…ถๅฎƒไพ่ต–

git clone --recursive https://gitee.com/xujunda2024/echochronos.git
cd echochronos
pip install -r requirements.txt

Instructions

  1. Set up a Python environment โ˜๏ธ๐Ÿค“

  2. After successfully installing the dependencies, prepare the configuration file in the format of examples/infer_qwen2_lora_fp32.yaml (be sure to modify the parameters in the configuration file according to your needs).

  3. Start the GPT-SOVITS service.

    • Prepare the model: For details, please refer to the README.md file of the GPT-SOVITS project.

    • Start the service:

      cd TTS/GPT-SoVITS-main/GPT_SOVITS
      python Server.py
  4. Currently, this project provides three modes of operation, which can be changed by modifying the "isTerminal", "isWebsocket", and "isWebUI" parameters in the YAML file (Replace <your_yaml_path> with your YAML-formatted configuration file).

    • Terminal: python launch.py <your_yaml_path>

    • WebSocket: python launch.py <your_yaml_path>

    • WebUI (recommended ๐Ÿคฉ): streamlit run launch.py <your_yaml_path>

MindSpore Users ๐Ÿคฏ

GPU Users

  • It is recommended to use CUDA 11.6 and cuDNN.

  • If you encounter [ERROR] libcuda.so (needed by mindspore-gpu) is not found., please export CUDA_HOME=/path/to/your/cuda

Video Memory

  • Due to the accuracy issues of the Qwen2 models in the MindNLP, inference can only be performed using float32, with memory consumption around 46G.

Instructions for MindSpore Users

  • MindSpore has two inference script entry points, and the startup methods are as follows:

    • python inference_ms.py --isTerminal or python inference_ms.py --isWebsocket

    • streamlit run webui.py (recommended ๐Ÿคฉ)

About

A multi-modal style conversational AI assistant ๐Ÿ˜‡ based on LLM.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 3

  •  
  •  
  •