model.safetensors.index.json broken
#1
by
DJLevel3
- opened
Hello! Looks like model.safetensors.index.json is for the original, not the quantized version. Can you fix this? I'm working on doing it myself but I'm quite the noob, haha
Nice of you to assume that I know what I am doing... It works on my machine™ with the same set of files, which is also the case for other quants available on my profile. Are you trying something else, like using a funny backend, loading with transformers or whatever instead of EXL2, trying to finetune a quant, etc.? You can also detail your setup and environment.
Edit: I use TabbyAPI to load the model and Open Webui/SillyTavern etc. to interface.