I have created an 8-bit and 2-bit quantised version but it keep loading original

[220940947]

Due diligence

• I have done my due diligence in trying to find the answer myself. #56

Topic

The paper

Question

I have created a 8-bit and 2-bit version but it keeps loading the original anyone else having this problem or is there a known solutions?

I have adjusted server.py so it should but it keeps falling back to the original model and not my smaller ones for better usage on more pc's.

Like it is not even looking for the model when starting the server. Even tho the path's are correct.

[nekhbet]

@220940947 any luck?