fuzzing.uk
Zenith Rifle
Software & ML on Apple Silicon. Blogging about model serving, MoEs, and squeezing big models onto small machines.

Run a 20B Ternary MoE in 0.63 GB of RAM

Run a 20B Ternary MoE in 0.63 GB of RAM