Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
…ctors into sharepoint-oauth

# Conflicts:
#	README.md
#	cockroach/dev/load_data.py
#	confluence/dev/load_data.py
#	elastic/docker-compose.yml
#	gdrive/README.md
#	jira/dev/load_data.py
#	mongodb/dev/load_data.py
#	mongodb/docker-compose.yml
#	opensearch/docker-compose.yml
#	pinecone/dev/generate_embeddings.py
#	postgres/docker-compose.yml
#	redis/dev/load_data.py
#	redis/docker-compose.yml
#	sharepoint/.env-template
#	sharepoint/README.md
#	sharepoint/provider/client.py
#	sharepoint/provider/provider.py
#	sharepoint/provider/unstructured.py
#	snowflake/README.md
#	solr/docker-compose.yml
#	vespa/docker-compose.yml
#	weaviate/docker-compose.yml
#	wordpress/dev/load_data.py
  • Loading branch information
tianjing-li committed Dec 13, 2023
2 parents cff39ac + ee28963 commit 213881a
Show file tree
Hide file tree
Showing 34 changed files with 32,767 additions and 27 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@

# Overview

Cohere's Build-Your-Own-Connector framework allows you to integrate Cohere's Command LLM via the [co.chat api endpoint](https://docs.cohere.com/reference/chat) to any datastore/software that holds text information and has a corresponding search endpoint exposed in its API. This allows the Command model to generated responses to user queries that are grounded in proprietary information.
Cohere's Build-Your-Own-Connector framework allows you to integrate Cohere's Command LLM via the [Chat api endpoint](https://docs.cohere.com/reference/chat) to any datastore/software that holds text information and has a corresponding search endpoint exposed in its API. This allows the Command model to generated responses to user queries that are grounded in proprietary information.

Some examples of the use-cases you can enable with this framework:

Expand Down Expand Up @@ -46,7 +46,7 @@ After configuring the `.env`, you will be able to use `poetry`'s CLI to start a

# Integrating With Cohere

All of the connectors in this repository have been tailored to integrate with Cohere's [co.chat](https://docs.cohere.com/reference/chat) API to make creating a grounded chatbot quick and easy.
All of the connectors in this repository have been tailored to integrate with Cohere's [Chat](https://docs.cohere.com/reference/chat) API to make creating a grounded chatbot quick and easy.

Cohere's API requires that connectors return documents as an array of JSON objects. Each document should be an object with string keys and string values containing all the relevant information about the document (e.g. `title`, `url`, etc.). For best results the largest text content should be stored in the `text` key.

Expand All @@ -69,7 +69,7 @@ For example, a connector that returns documents about company expensing policy m
]
```

Cohere's [co.chat](https://docs.cohere.com/reference/chat) API will query the connector and use these documents to generated answers with direct citations.
Cohere's [Chat](https://docs.cohere.com/reference/chat) API will query the connector and use these documents to generated answers with direct citations.

# Contributing

Expand Down
2,338 changes: 2,338 additions & 0 deletions cockroach/dev/bbq.csv

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion cockroach/dev/load_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ def load_data():
conn = psycopg2.connect(COCKROACH_DATABASE_URL)
cursor = conn.cursor()

with open("../../testdata/bbq.csv", "r") as csv_file:
with open("./dev/bbq.csv", "r") as csv_file:
reader = csv.DictReader(csv_file)
for row in reader:
insert_query = """
Expand Down
2,338 changes: 2,338 additions & 0 deletions confluence/dev/bbq.csv

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion confluence/dev/load_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
)
space = os.environ.get("CONFLUENCE_SPACE_NAME")

with open("../testdata/bbq.csv", "r") as csv_file:
with open("./dev/bbq.csv", "r") as csv_file:
reader = csv.DictReader(csv_file)
for row in reader:
title = row["Name"]
Expand Down
2,338 changes: 2,338 additions & 0 deletions elastic/dev/bbq.csv

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion elastic/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ services:
depends_on:
- elasticsearch
volumes:
- ../testdata/bbq.csv:/bbq.csv
- ./dev/bbq.csv:/bbq.csv
command:
- elasticdump
- --input=csv:///bbq.csv
Expand Down
4 changes: 2 additions & 2 deletions gdrive/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ The `GDRIVE_SERVICE_ACCOUNT_INFO` variable should contain the JSON content of th
}
```

4. Convert the JSON credentails to a string through `json.dumps(credentials)` and save the result in the `GDRIVE_SERVICE_ACCOUNT_INFO` environment variable.
4. Convert the JSON credentials to a string through `json.dumps(credentials)` and save the result in the `GDRIVE_SERVICE_ACCOUNT_INFO` environment variable.
5. Make sure to [share the folder(s) you want to search with the service account email address](https://support.google.com/a/answer/7337554?hl=en).

#### `GDRIVE_CONNECTOR_API_KEY`
Expand Down Expand Up @@ -93,7 +93,7 @@ Create a virtual environment and install dependencies with poetry. We recommend
Next, start up the search connector server:

```bash
$ poetry flask --app provider --debug run --port 5000
$ poetry run flask --app provider --debug run --port 5000
```

and check with curl to see that everything works:
Expand Down
2,338 changes: 2,338 additions & 0 deletions jira/dev/bbq.csv

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion jira/dev/load_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
)
project_key = os.environ.get("JIRA_ISSUE_KEY")

with open("../testdata/bbq.csv", "r") as csv_file:
with open("./dev/bbq.csv", "r") as csv_file:
reader = csv.DictReader(csv_file)
for i, row in enumerate(reader, start=1):
issue_key = f"{project_key}-{i}"
Expand Down
2,338 changes: 2,338 additions & 0 deletions mongodb/dev/bbq.csv

Large diffs are not rendered by default.

9 changes: 4 additions & 5 deletions mongodb/dev/load_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,12 @@

load_dotenv()

client = pymongo.MongoClient(
host=os.environ.get("MONGODB_HOST", "mongo"),
port=os.environ.get("MONGODB_PORT", 27017),
username=os.environ.get("MONGODB_ROOT_USERNAME", "root"),
password=os.environ.get("MONGODB_ROOT_PASSWORD", "example"),
connection_string = os.environ.get(
"MONGODB_CONNECTION_STRING", "mongodb://root:example@mongo:27017"
)

client = pymongo.MongoClient(connection_string)

fields = []

db = client.bbq
Expand Down
8 changes: 5 additions & 3 deletions mongodb/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
version: "1"
version: "3.8"
services:
mongo:
image: mongo:latest
container_name: mongo
restart: unless-stopped
restart: always
ports:
- 27017:27017
environment:
Expand All @@ -18,8 +18,10 @@ services:
container_name: data-loader
depends_on:
- mongo
environment:
MONGODB_CONNECTION_STRING: mongodb://root:example@mongo:27017
volumes:
- ../testdata/bbq.csv:/bbq.csv
- ./dev/bbq.csv:/bbq.csv
- ./dev/load_data.py:/load_data.py
command: sh -c "pip install pymongo python-dotenv && python3 /load_data.py"
profiles:
Expand Down
Loading

0 comments on commit 213881a

Please sign in to comment.