LLM Finetuning Toolkit

[benjaminye]

This PR introduces a config-driven LLM fine-tuning experience.

Key Features

• Launch fine-tuning by defining a single config.yaml file
• Interactive and helpful print outputs
• Experiment checkpointing that saves and loads experiment artefacts at various stages, enabling one to pickup where left off

Key Modules

• DatasetGenerator: loads model into a huggingface Dataset object based on file extension (csv, json, or huggingface repo)
• ModelLoader: manages loading of base model, appending lora weights, and merging lora weights
• InferenceRunner: runs merged model on test set, outputs live streaming predictions, and saves predictions into csv

Road Map

• adopt poetry for dependency management
• create a docker container for predictable deployment
• multi-GPU support
• stratified sampling for dataset
• ablation studies
• documentation

[RohitSaha]

@benjaminye Aren't we doing stratified sampling now?

[RohitSaha]

How to differentiate between proportion and # of samples?

[benjaminye]

This is uniquely determined by the data type passed in.

proportion: float;
# of samples: int;

this is built into huggingface dataset's train_test_split() method.

[RohitSaha]

Did we include neftune functionality?

[benjaminye]

Yes; you can enable it under sft_args.
You can see all supported config settings under config_model.py.

I have added explanations, expected type, and defaults for each setting (where applicable), and we can then use the model definitions to automatically generate documentation.

[RohitSaha]

@benjaminye Let's ensure the packages we use only contain permissive licenses, i.e, Apache 2.0 and MIT. We should remove any dependencies that have copyleft license.

[benjaminye]

@benjaminye Aren't we doing stratified sampling now?

Unfortunately this isn't as simple. I used huggingface's recommended from_generator() method to read in files directly into a Dataset object; instead of loading the files into a pandas DataFrame first (which isn't as memory & compute efficient).

Not loading into DataFrame precludes us from using scikit-learn's train_test_split() method; instead we have to use the method from huggingface dataset. However, that method expects the stratify_by column to be a ClassLabel (which is an integer representation). A hacky method is to cast it to ClassLabel and then cast it back into string Value, an avenue that I'm exploring.

Moreover, since stratified sampling only applies to classification tasks, I decided to think about this a bit more and release it in the next update.

[benjaminye]

@benjaminye Let's ensure the packages we use only contain permissive licenses, i.e, Apache 2.0 and MIT. We should remove any dependencies that have copyleft license.

I've checked all the new dependencies and they all have permissive licenses.

[RohitSaha]

Thanks for making the changes!