Merge pull request #207 from eye-on-surveillance/AI/public-comments-bias

AI/public-comments-bias
eye-on-surveillance · Jan 3, 2024 · 0c837b8 · 0c837b8
2 parents b6b09f8 + b57a635
commit 0c837b8
Show file tree

Hide file tree

Showing 11 changed files with 1,497 additions and 10 deletions.
diff --git a/packages/backend/src/cache/faiss_index_general.dvc b/packages/backend/src/cache/faiss_index_general.dvc
@@ -1,6 +1,6 @@
 outs:
-- md5: a3801ca1eb86b9981418e5594c9ff419.dir
-  size: 137244851
+- md5: 36fba85a3903236524e814d1f78b8aed.dir
+  size: 137244321
   nfiles: 2
   hash: md5
   path: faiss_index_general
diff --git a/packages/backend/src/cache/faiss_index_in_depth.dvc b/packages/backend/src/cache/faiss_index_in_depth.dvc
@@ -1,6 +1,6 @@
 outs:
-- md5: 48350758567819a301f732710b7d73cf.dir
-  size: 137244851
+- md5: 92e16223ddd6a98d03b4aa93256ff5f0.dir
+  size: 137244321
   nfiles: 2
   hash: md5
-  path: faiss_index_in_depth
+  path: faiss_index_in_Depth
diff --git a/packages/backend/src/json_public_comment_directory/Public Comments.json b/packages/backend/src/json_public_comment_directory/Public Comments.json
diff --git a/... Council Committee Meeting 4-12-2022.json → ... Council Committee Meeting 4-12-2022.json b/... Council Committee Meeting 4-12-2022.json → ... Council Committee Meeting 4-12-2022.json
diff --git a/... Council Committee Meeting 6-15-2022.json → ... Council Committee Meeting 6-15-2022.json b/... Council Committee Meeting 6-15-2022.json → ... Council Committee Meeting 6-15-2022.json
diff --git a/...gular City Council Meeting 3-10-2022.json → ...gular City Council Meeting 3-10-2022.json b/...gular City Council Meeting 3-10-2022.json → ...gular City Council Meeting 3-10-2022.json
diff --git a/...gular City Council Meeting 7-21-2022.json → ...gular City Council Meeting 7-21-2022.json b/...gular City Council Meeting 7-21-2022.json → ...gular City Council Meeting 7-21-2022.json
diff --git a/...le Cities Committee Meeting 7-8-2022.json → ...le Cities Committee Meeting 7-8-2022.json b/...le Cities Committee Meeting 7-8-2022.json → ...le Cities Committee Meeting 7-8-2022.json
diff --git a/packages/googlecloud/functions/getanswer/cache/faiss_index_general.dvc b/packages/googlecloud/functions/getanswer/cache/faiss_index_general.dvc
@@ -1,6 +1,6 @@
 outs:
-- md5: a3801ca1eb86b9981418e5594c9ff419.dir
-  size: 137244851
+- md5: 36fba85a3903236524e814d1f78b8aed.dir
+  size: 137244321
   nfiles: 2
   hash: md5
   path: faiss_index_general
diff --git a/packages/googlecloud/functions/getanswer/cache/faiss_index_in_depth.dvc b/packages/googlecloud/functions/getanswer/cache/faiss_index_in_depth.dvc
@@ -1,6 +1,6 @@
 outs:
-- md5: 48350758567819a301f732710b7d73cf.dir
-  size: 137244851
+- md5: 92e16223ddd6a98d03b4aa93256ff5f0.dir
+  size: 137244321
   nfiles: 2
   hash: md5
-  path: faiss_index_in_depth
+  path: faiss_index_in_Depth
diff --git a/packages/googlecloud/functions/getanswer/inquirer.py b/packages/googlecloud/functions/getanswer/inquirer.py
@@ -212,6 +212,14 @@ def get_indepth_response_from_query(df, db, query, k):
     template = """
     Question: {question}
 
+    ### Bias Guidelines:
+    
+    Please be aware of inherent biases within the document corpus, especially an overrepresentation of certain types of documents, such as those about surveillance. These biases may result in the retrieval of documents that are irrelevant to the question. When analyzing documents to answer the question, it is crucial to critically evaluate their relevance to the question at hand.
+
+    To ensure accuracy and relevance in your analysis you must identify and disregard irrelevant documents by actively identifying documents that, despite being returned by the database, do not substantively address the query. Such documents should be disregarded in the analysis.
+
+    ### Response Guidelines:
+
     Based on the information from the New Orleans city council documents provided, answer the following question: {question}. 
     Your answer must not exceed 5,000 tokens.