modify retrieval top_k parameter to 1 for benchmark (#206)

Signed-off-by: letonghan <[email protected]>

evals/benchmark/README.md

@@ -99,7 +99,7 @@ test_cases:
       service_name: "retriever-svc"
       parameters:
         search_type: "similarity"
-        k: 4
+        k: 1
         fetch_k: 20
         lambda_mult: 0.5
         score_threshold: 0.2
@@ -121,5 +121,6 @@ test_cases:
       dataset: # Activate if random_prompt=true: leave blank = default dataset(WebQuestions) or sharegpt
       prompts: # User-customized prompts, activate if random_prompt=false.
       max_output: 128  # max number of output tokens
+      k: 1 # number of retrieved documents
 ```
 If you'd like to use sharegpt dataset, please download the dataset according to the [guide](https://github.com/lm-sys/FastChat/issues/90#issuecomment-1493250773). Merge all downloaded data files into one file named sharegpt.json and put the file at `evals/benchmark/stresscli/dataset`.

evals/benchmark/benchmark.yaml

@@ -39,7 +39,7 @@ test_cases:
       service_name: "retriever-svc"  # Replace with your service name
       parameters:
         search_type: "similarity"
-        k: 4
+        k: 1
         fetch_k: 20
         lambda_mult: 0.5
         score_threshold: 0.2
@@ -81,6 +81,7 @@ test_cases:
       dataset:  # Activate if random_prompt=true: leave blank = default dataset(WebQuestions) or sharegpt
       prompts: In an increasingly complex world where technology has rapidly advanced and evolved far beyond our wildest dreams, humanity now stands on the precipice of a revolutionary new era that is filled with endless possibilities, profound and significant changes, as well as intricate challenges that we must actively address. The year is now 2050, and artificial intelligence has seamlessly woven itself deeply and intricately into the very fabric of everyday life. Autonomous vehicles glide effortlessly and smoothly through the bustling, vibrant, and lively city streets, while drones swiftly and accurately deliver packages with pinpoint precision, making logistics and delivery systems more efficient, streamlined, and advanced than ever before in the entire history of humankind and technological development. Smart homes, equipped with cutting-edge advanced sensors and sophisticated algorithms, anticipate every possible need and requirement of their inhabitants, creating an environment of unparalleled convenience, exceptional comfort, and remarkable efficiency that enhances our daily lives. However, with these remarkable and groundbreaking advancements come a host of new challenges, uncertainties, and ethical dilemmas that society must confront, navigate, and address in a thoughtful and deliberate manner. As we carefully navigate through this brave new world filled with astonishing technological marvels, innovations, and breakthroughs, questions about the implications and consequences of AI technologies become increasingly pressing, relevant, and urgent for individuals and communities alike. Issues surrounding privacy—how our personal data is collected, securely stored, processed, and utilized—emerge alongside significant concerns about security in a rapidly evolving digital landscape where vulnerabilities can be easily and readily exploited by malicious actors, hackers, and cybercriminals. Moreover, philosophical inquiries regarding the very nature of consciousness itself rise prominently to the forefront of public discourse, debate, and discussion, inviting diverse perspectives, opinions, and ethical considerations from various stakeholders. In light of these profound developments and transformative changes that we are witnessing, I would like to gain a much deeper, broader, and more comprehensive understanding of what artificial intelligence truly is and what it encompasses in its entirety and complexity. Could you elaborate extensively, thoroughly, and comprehensively on its precise definition, its wide-ranging and expansive scope, as well as the myriad and diverse ways it significantly impacts our daily lives, personal experiences, and society as a whole in various dimensions and aspects? # User-customized prompts, activate if random_prompt=false.
       max_output: 128  # max number of output tokens
+      k: 1 # number of retrieved documents
 
   codegen:
     llm: