Spherical - high throughput experiments

[mgoliyad]

We need to finalize technology stack for high throughput inference for BOLTZ2 and ESM2 models.

[mgoliyad]

Initial test was conducted using vLLM. Both models are not vLLM compatible. Next, I will explore approach using DeepSpeed

[mgoliyad]

vLLM results were documented in
https://drive.google.com/drive/folders/18vfHPDH74WrqBbVY86RPf014Qif7bK17?usp=share_link

[mgoliyad]

Successfully tested deepspeed command on single node (on anvil machine)For multi nodes configuration - torchrun command worked with sbatch script but not with RP (Aymen will check the RP sessions).

[mgoliyad]

Started putting all pieces together to run HTP using RP.
Srun command is not working for service tasks - "srun: Job 14411578 step creation still disabled, retrying (Requested nodes are busy)".
Andre is setting flux on anvil machine.