From 29dbf8b89ae6ad4021892a481bcc34e029292afe Mon Sep 17 00:00:00 2001 From: Sameer Segal Date: Sat, 25 May 2024 15:52:44 +0530 Subject: [PATCH 1/2] Removed stale docs and moved Developer.md as a how to guide --- DeepDive.md | 152 ---------------------- README.md | 14 +- Setup.md | 42 ------ Developer.md => docs/how-tos/Developer.md | 0 docs/how-tos/index.md | 3 +- 5 files changed, 5 insertions(+), 206 deletions(-) delete mode 100644 DeepDive.md delete mode 100644 Setup.md rename Developer.md => docs/how-tos/Developer.md (100%) diff --git a/DeepDive.md b/DeepDive.md deleted file mode 100644 index ab376898..00000000 --- a/DeepDive.md +++ /dev/null @@ -1,152 +0,0 @@ -# Deep Dive - -This document explains the inner workings of the JB Manager. It also provides the gaps in the current implementation. - -## Architecture - -All the services are generic i.e. they are designed for multiple bots or use cases. The only customization that exists lies in the tiny bot code (finite state machine code) which is published by the Studio. - -![](docs/assets/jb-architecture.png) - -Language Flow -![](docs/assets/language%20flow.png) - - -## DB Structure - -**Bot** - -bot_id (e.g. "12345") is at the highest level. All other entries like person, sessions are against a particular bot. We support multiple bots from the same backend. - -**User** - -A user needs to be registered for the bot. Phone Number & Bot ID make the user unique. - -**Session** - -We maintain session for 24 hours. After that a fresh conversation starts. - -**Turn** - -Every message received from the user creates a new turn in the conversation. Against one user message, we can have multiple bot responses. - -**Message** - -Every message received or sent is logged to the DB. - -**FSM State** - -This table currently stores the state (stage in the workflow) and variables. This needs to be merged with session table. - -## Services - -### API - -Provides APIs to interact with JB Manager. Here are the top buckets of APIs: -1. Callback from channels like WhatsApp or Telegram -2. JB Studio like publishing the bot -3. Managing bots changing their states and variables -4. Viewing chat logs and analytics -5. Triggering indexing jobs - -**Callback** - -1. Figure out `bot_id` based on the callback -2. Create person if doesn't exists. Fetch `pid` -3. Create `session_id` if session has expired (> 24 hours). -4. Creates `turn_id` for the new conversation -5. Create `msg_id` - -Later: - - [ ] Indexer Job & Status - - Upload files and start a job in the indexer - - Update the status of the Job - - [ ] Install new bot -- [ ] - Support multiple bots - -### Channel - -Responsible for both input and output communication with the channel (WhatsApp / Telegram). It needs to handle channel specific API calls and rendering of UI messages (e.g. WhatsApp Flow, List & Buttons) - -1. If media present in the message, fetch and upload to Azure. -2. Provides standard dialogues (list/buttons) for features that are common to all bots. It handles rendering (outbound) and the related user action. It let's Flow service know that the user has completed a dialogue. - 1. Language - It shows the dialog and saves the user's language preference - 2. Reset (not yet there) - -In the inbound scenario: -* It mostly enqueues messages for the Language service -* Unless the message related to a dialogue (e.g. language), WhatsApp Flow or Button. In this case it directly talks to the Flow service. - -In the outbound scenario: -* It mostly receives messages from the Language service. -* Unless the message is related to WhatsApp flow in which case Flow directly writes to it's queue. - -### Language - -1. There are two flows: - 1. Receiving users message - 2. Sending the response to the user. - -It's responsible for translating labels in case of interactives messages e.g. Lists & Buttons - -We assume that all messages sent by the user obey the user's `preferred_language`. - -For outbound messages, Language service will look at the message type from the Flow Service and then decide on the following: -1. If `text` - 1. Then fetch details about the turn -- In what mode did the user ask their question: - 1. If user asks in text / form / interactive, we respond in text - 2. If user asks in voice, we respond in voice + text -- Language service creates two inputs for Channel service - 3. In all other cases, send text -2. If `document`, - 1. Translate the caption -3. `image` - 1. Translate the caption -5. If `interactive` - 1. Translate the text, header, footer - 2. Translate the labels for buttons - -- [ ] Cache the generated respones based on hash of input and language_code - -### Flow - -1. Retrieve session from DB based on session_id. This provides it memory of the current state (stage of the workflow), variables. -2. Process the message via a finite state machine code (this code is generated in the Studio and pushed to the JB Manager) - 1. It can send multiple messages for every user message - 2. It can make API calls or use plugins - 3. The responses that it sends can include the following: - 1. Text - 2. Interactive - WhatsApp flow, List, Buttons - 3. Documents, Images - 4. Dialogues -- it uses keywords `language` to trigger Channel service into sending a standardized dialogue. -3. On complete (or logical pause in the workflow e.g. user input or plugin input), save the state and variables. - - -- [ ] Merge Session & FSM State table -- [ ] Enable going back to the previous state -- [ ] Allow queries at any state -- [ ] Address slow response times -- [ ] Enhance conversational aspects - - Example: Select Slot: (Number, Time, Name) - -### Retrieval - -This fetches data from the Vector DB (PG Vector / Postgres) based on query and metadata if any. The metadata is used to create filter query to restrict the similarity search even further. - -- [ ] Rejig the query -- [ ] Reranking logic - -### Indexer - -This service receives input files (urls from Azure) and pre-processes them and indexes them into the Vector DB. - -We need to make this generic -- [ ] Stop words -- [ ] Chunking logic -- [ ] Metadata fields - -## Code Enhancements - -- [ ] Relationship between models - foreign keys -- [ ] Implement a reset feature to clear conversation history -- [ ] Improve state management to reduce confusion (Current State) -- [ ] Utilize conversation history for better responses and follow-ups diff --git a/README.md b/README.md index adde7259..064eb450 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,7 @@ -# JB Manager -Jugalbandi v2 that supports both information and actions. Now an NGO, Government Department or Social Enterprise user can build with the Jugalbandi Studio and with one-click deploy their bot and make it available to their beneficiaries. +# Jugalbandi Manager -We have repackaged the code into an event-driven architecture to make it scalable and also composable to meet varying needs. +Jugalbandi (JB) Manager is a full AI-powered conversational chatbot platform. It's platform agnostic and can serve multiple channels such as WhatsApp or custom web interfaces. It can handle conversations in both text and voice across any language. It comes with Bhashini Speech models out of the box and can failover to Azure. ![](docs/assets/jb-manager.png) -## Deep Dive -To deep dive into the architecture and various components, look [here](DeepDive.md) - -## Local Setup -Please look at the [Setup Notes](Setup.md) for local system setup. - -## Developer Notes (moved) -Please look at the updated [Developer Notes](Developer.md). +Read the documentation [here](https://opennyai.github.io/Jugalbandi-Manager/) diff --git a/Setup.md b/Setup.md deleted file mode 100644 index 7375e057..00000000 --- a/Setup.md +++ /dev/null @@ -1,42 +0,0 @@ -## Steps - -1. Rename the **.env-dev.template** file to **.env-dev** -2. Start the **docker daemon** by using docker-cli or Docker desktop or colima. -3. Run the application by running the following command from the root folder in the terminal or command prompt. - ``` - For Linux/Unix systems: - ./scripts/run.sh - - For Windows: - sh ./scripts/run.sh - ``` -4. Once all the services mentioned in the **docker-compose.yml** file have started running, you can check the status of the services by running the `docker ps` command in another terminal. -5. The DB tables have to be populated in the running postgres service. It can be achieved by running the below command from the root folder. - ``` - For Linux/Unix systems: - ./scripts/upgrade-db.sh - - For Windows: - sh ./scripts/upgrade-db.sh - ``` -6. Missing keys procurement: - * ***Storage keys***: - 1. In Azure, create a new service in storage accounts and copy the values of storage account name and key present in access keys tab. They will form the values for STORAGE_ACCOUNT_URL and STORAGE_ACCOUNT_KEY present in the .env-dev respectively. - 2. In the newly created same storage account, create a container with your preferred name. This name will form the value for STORAGE_AUDIOFILES_CONTAINER. - * ***Language keys***: - 1. For Bhashini keys, register an account in [Bhashini website](https://bhashini.gov.in/ulca/user/register) and get the values for BHASHINI_USER_ID, BHASHINI_API_KEY and BHASHINI_PIPELINE_ID in .env-dev file. - 2. For Azure keys, create resources in translator and speech services and copy the copys for the values for AZURE_TRANSLATION_KEY, AZURE_TRANSLATION_RESOURCE_LOCATION, AZURE_SPEECH_KEY and AZURE_SPEECH_REGION present in the respective **keys and endpoint** console. - * ***Whatsapp keys***: - 1. Contact the pinnacle services to get whatsapp keys for the values of WA_API_HOST, WABA_NUMBER and WA_API_KEY. -7. Frontend configuration: - * Once the necessary keys are procured, go to this running [frontend link](http://0.0.0.0:4173/) and click on install new bot and provide the required data to create your bot. The detailed information about the fields are given below: - 1. **Name [Mandatory]** is the name of the bot - 2. **DSL [Optional]** is the domain specific language code for fsm.py - 3. **Code [Mandatory]** is the fsm.py file python code - 4. **Requirements [Optional if no specialised pacakge is used in code]** is the required packages name with their versions as we put them usually in requirements.txt or pyproject.toml dependencies - 5. **index_urls [Optional]** is for custom and private packages links to download them from (This is for the case you use a library that your team has developed and internally published) - 6. **version [Mandatory]** is the version of the bot - 7. **required_credentials [Mandatory]** These are the variable names present in the .env-dev file. Only the variable names should be present with comma separated between them. - * Once the bot is created, click on the **settings (⚙) icon** to enter the given credentials values and click save to save the credentials values. - * Then click on the **play (▶️) icon** to activate the bot by providing the whatsapp bot number and whatsapp api key. - * Once the above steps are completed, the bot status will be changed from **inactive to active.** \ No newline at end of file diff --git a/Developer.md b/docs/how-tos/Developer.md similarity index 100% rename from Developer.md rename to docs/how-tos/Developer.md diff --git a/docs/how-tos/index.md b/docs/how-tos/index.md index dfaef903..5fef72bd 100644 --- a/docs/how-tos/index.md +++ b/docs/how-tos/index.md @@ -1,4 +1,5 @@ --- layout: default title: How to Guides ---- \ No newline at end of file +--- +1. [Local Development](Developer.md) From b4cc9d309ca7fba911b52c42d9b7b14f3dc40d4e Mon Sep 17 00:00:00 2001 From: Sameer Segal Date: Sat, 25 May 2024 15:53:23 +0530 Subject: [PATCH 2/2] Fixed image reference --- docs/how-tos/Developer.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/how-tos/Developer.md b/docs/how-tos/Developer.md index 85670e01..3be514c0 100644 --- a/docs/how-tos/Developer.md +++ b/docs/how-tos/Developer.md @@ -24,7 +24,7 @@ It's important that we test the data contracts (input / output JSONs) for our se ./scripts/read-message.sh topic ``` -![](docs/assets/local-terminal.png) +![](../assets/local-terminal.png) ## Setting up Local DB