how to adapt my new model?

[belle9217]

No description provided.

[felipemello1]

Hi @belle9217 , can you give more details of what you are trying to do?

[belle9217]

I have a model with a similar architecture to the LLaMA model, which has been modified based on LLaMA by altering the multi-head attention mechanism for QKV. now I want to use FSDP(deepspeed zero3) and torch.compile,and I found torchtune has the two functionalities.so I want to add my model to the torchtune framework. since I see torchtune has support Qwen、llama、phi, how can I adjust my modeling.py to use torchtune? is there any API or Interface？