Roadmap 0.2.0 🍁

[queryproc]

Description

This issue keeps track of the main features we plan to deliver in the upcoming releases:

0.2.0

Features

• Provide support for Azure as model provider (Feature/model provider setup #45)
• Provide support for Bedrock as model provider
• Provide support running with local LLMs using Ollama / models (Ollama provider onboarding #55)
• Add the equivalent to llm_complete with an llm_reduce (Add llm_reduce Aggregate Function #53)
• Add fusion algorithm implementation (Add fusion_relative Function for Score Normalization and Max Aggregation #65)

Performance Improvements

• Support batching for llm_embedding. It is currently only supported for the LLM completion scalar functions. (Add the batching to the llm_embedding #54)
• Our batching uses JSON when batching rows leading to a lot more token usage then necessary -> Markdown. (Convert JSON to Markdown Table, Remove Inja, and Refactor Code #62)
• Prediction caching based on exact prompt equality. (In-memory vs Disk)
• Requests for unique rows (given relevant columns) per DataChunk
• Performance Enhancement: Split equally the rows if a DataChunk needs more than 1 LLM prediction request #15

Usability Improvements

• We should further specialize certain functions and provide out of the box very well written prompts for certain common tasks (e.g., llm_summarize, llm_extract_facts_and_verify_output)
• We need to support out-of-the-box versioning for PROMPTs (Add Support for Prompt Versioning #63)
• Update all function parameters to the 3 of model, prompt, input attributes (Refactor llm_* Function Call Structure for Enhanced Flexibility in Prompt Specification #61)

Bugs

• Bug: Add tiktoken as dependency to count exactly the number of tokens #16
• Bug: OpenAI API Key env variable is not usable within DuckDB WASM (Add Support for Configuring Secrets for Multiple Model Providers #66)