The algorithm is based on paper Federated Domain-Specific Knowledge Transfer on Large Language Models Using Synthetic Data, a novel framework that enables domain-specific knowledge transfer from LLMs to SLMs while preserving SLM data privacy.
If you publish work that uses FDKT, please cite FDKT as follows:
@article{li2024federated,
title={Federated Domain-Specific Knowledge Transfer on Large Language Models Using Synthetic Data},
author={Li, Haoran and Zhao, Xinyuan and Guo, Dadi and Gu, Hanlin and Zeng, Ziqian and Han, Yuxing and Song, Yangqiu and Fan, Lixin and Yang, Qiang},
journal={arXiv preprint arXiv:2405.14212},
year={2024}
}