Will this project support lora?

[IvoryTower800]

Thanks for giving me an easy way finetuning model on TPU. I tried finetune a full model using this project on kaggle's tpu and it works.

I'm wondering whether there is a schedule for supporting finetune a lora or qlora model? it's a more practical way for those only have access to limited computing resources like me...

[erfanzar]

yes in the upcoming future and after finishing DPO, SFT Trainers the Lora process will start in FJFormer (EasyDel Computation Backend) and ill Update BITs too

[IvoryTower800]

Awesome! I think lots of people are looking forward to it. Thank you!