quantization

Self-Hosting Mistral Large 4: Lessons Learned the Hard Way

When Mistral released the weights for Mistral Large 4 — a roughly 1-trillion-parameter model with about 52 billion active parameters — my inbox filled up with...