[FAQ] Pretrained Model in the sense of TABPFN #210
Description
Dear Team,
I just have a qurstion regarding your model and what it could achieve.
Did you ever try a pretraining schema like the TABPFN team did to pretrain your model and ship a foundation model that could be finetuned or used out of the Box? I am not sure how your model specificly attends to rows and columns in comparison with the TABPFN but maybe it could be possible to copy the approach and use it with MAMBA. Also due to the different scaling in compute you could easily incooperate more data points, what seemes to be a limitiation at the moment.
Would this be something your planning on doing?
Best