Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

_dialect.py Re-Work, modifier Argument for LLMQA, Documentation updates #35

Merged
merged 76 commits into from
Oct 26, 2024

Conversation

parkervg
Copy link
Owner

@parkervg parkervg commented Oct 26, 2024

  • Re-worked _dialect.py to better handle different SQL dialects
    • Previously, we forced sqlglot to interpret all queries as SQLite, and relied on the serendipitous interpretation of ingredients as STRUCT nodes to do our processing
    • Now, we have the get_dialect() function to route queries to the correct sqlglot dialect, and interpret all aliased ingredients (i.e. {{A()}}, {{B()}}, etc.) as a TokenType.FUNCTION via INGREDIENT_TOKEN_TYPE_MAPPING
  • Added support for modifier argument
    • LLMQA now works in a context where a list datatype is required, such as IN or VALUES clauses
    • To unify handling of data types across ingredients, the DataType object in _constants.py was created
  • Added AzurePhiModel
  • Added blendsql-by-example.ipynb

Before:

Task Average Runtime # Unique Queries
financials 0.0440116 7
rugby 1.26541 4
national_parks 3.77385 5
1966_nba_draft 4.02796 2

After:

Task Average Runtime # Unique Queries
financials 0.0415093 7
rugby 1.18449 4
national_parks 3.00202 5
1966_nba_draft 4.00881 2

This is where the 2nd argument to an 'IN' clause shows up, e.g. 'WHERE x IN {{Ingredient()}}'
Else, this will break the `name=value` call to guidance
This allows us to skip the final `query_context.to_string()` call before executing the compiled blendsql query
@parkervg parkervg merged commit 3eccdea into main Oct 26, 2024
16 checks passed
@parkervg parkervg deleted the feature/dialect-rework branch October 26, 2024 20:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant