Train and serve Mixture-of-Experts models that do not fit in VRAM: fused 4-bit experts, QLoRA, CPU/NVMe offload, and fast inference on consumer NVIDIA GPUs.
No known vulnerabilities in the latest version
Based on latest version 0.37.4. If you're running an older version, check OSV for your specific version.
[](/packages/experts4bit-qlora)
<a href="/packages/experts4bit-qlora"><img src="/api/badges/experts4bit-qlora?period=month" alt="PyPI Stats"></a>