feat: simple mistral lora integration tests #2180
Conversation
| "predibase/dbpedia", | ||
| "predibase/customer_support", | ||
| ], | ||
| cuda_graphs=[0], |
There was a problem hiding this comment.
currently theres a bug with lora and cuda graphs but wanted to make sure there were basic tests for expected responses. I'll push a update for cuda graphs soon along with some of the other refactoring changes. Happy to keep this open until it's resolved
There was a problem hiding this comment.
What is the bug ?
Does load with many different loader breaks things ?
There was a problem hiding this comment.
currently the adapter is ignored when graphs are enabled because adapter_data is passed to model.forward and is not included in cuda_graph_warmup. I need to update the graph warmup to include adapter data and also replace the values before calling replay
|
this PR has been updated to disable CUDA_GRAPHS if lora adapters are enabled. In the future we'll add support however until support is added we will disable and warn the user |
drbh
force-pushed
the
integration-test-lora
branch
from
f3e6041 to
1ebcc9d
Compare
* feat: simple mistral lora integration tests * fix: include args in docker launcher * fix: disable cuda graphs with lora and warn * fix: adjust docs and precommit issues * fix: re update docs
* feat: simple mistral lora integration tests * fix: include args in docker launcher * fix: disable cuda graphs with lora and warn * fix: adjust docs and precommit issues * fix: re update docs
* feat: simple mistral lora integration tests * fix: include args in docker launcher * fix: disable cuda graphs with lora and warn * fix: adjust docs and precommit issues * fix: re update docs
This PR adds integration tests for
mistralai/Mistral-7B-v0.1with two lora adapterspredibase/dbpediaandpredibase/customer_support. These test will be improved in the future, and exist to ensure that basic lora functionality works as expected