Kimi K3 with 2.8 trillion parameters runs locally on Apple Silicon: weight streaming, MLX and REAP pruning

🚀 A wild few days for local AI and open-weight models

Kimi K3 by Moonshot AI: the model’s numbers

The star of the show is Kimi K3 by Moonshot AI:

• 2.8 trillion total parameters
• Mixture of Experts (MoE) architecture
• 104 billion active parameters per token
• roughly 1.6 TB of open weights
• a one-million-token context window

After the full weights were released on July 27, the community started testing it on local hardware, above all Apple Silicon.

The specs come from Moonshot AI’s tech blog (July 17, 2026) and from the official model card on Hugging Face: 2.8T total parameters, 104B active per token, 16 active experts out of 896, a one-million-token context window.

Salvatore Sanfilippo’s experiments on Apple Silicon

Salvatore Sanfilippo (a hugely well-known Italian developer) is experimenting with streaming the weights from Hugging Face on a MacBook Pro M5 Max with 128 GB of unified memory.

The model doesn’t fit entirely in memory: the weights are read progressively from storage while it runs. It’s very slow, but it proves you can start exploring a 1.6 TB model even on a single machine! 🤯

Sanfilippo is also working on running it distributed across two 512 GB Mac Studios.

Sanfilippo documented the streaming experiment in a post on X in late July 2026; the distributed inference strategies across two 512 GB Mac Studios are described on his blog (May 2026).

MLX, quantization and streaming conversion

The community is also trying to adapt Kimi K3 to Apple Silicon and cut its memory requirements through MLX, quantization and pruning.

MLX is Apple’s framework designed to take advantage of the unified memory in its chips, letting CPU and GPU work on the same data without copying it between separate memories.

It doesn’t shrink the model, but it does let you run converted, quantized or pruned versions more efficiently.

Streaming conversion processes part of the weights at a time, saves them in MLX format and frees the memory, without having to load the entire model during the process.

REAP pruning: from 1.6 TB to roughly 350 GB

REAP pruning, on the other hand, really does shrink the model.

Kimi K3 contains hundreds of specialized blocks, called experts, but it activates only a few of them for each token. Pruning removes the ones considered less important.

An experimental configuration keeps 179 of 896 experts, shrinking the model to roughly 350 GB and letting it fit into the memory of a 512 GB Mac Studio.

The trade-off is significant: it’s no longer the full model and token generation stays slow.

The MLX port with streaming converter and REAP pruning comes from Pipe Network; the 179-of-896-experts configuration (~350 GB) is published on Hugging Face (July 2026).

The most interesting part: how fast the community moves

And this is the most interesting part.

Not just the arrival of a huge open-weight model, but the speed at which the community is building conversions, quantizations, distributed systems and pruning techniques to bring these models onto local hardware.

The limits: prosumer hardware and trade-offs on quality and speed

One caveat though: Apple Silicon and MLX are closing the gap with proprietary models, but an ordinary consumer PC won’t cut it.

Some quantized or pruned versions can run on prosumer workstations costing tens of thousands of euros, without requiring an enterprise cluster.

Major trade-offs remain: quantization and pruning can reduce quality, while fitting the weights into memory doesn’t mean you get fast inference.

Local AI hasn’t reached parity with the best cloud services yet. My advice, for anyone using AI to build software products, is to use Kimi Code with the K3 API and skip local LLMs.