Apple researchers have demonstrated local inference for a large open-weight language model across a cluster of Mac Studio computers, using the company's MLX framework and a low-latency connection over Thunderbolt 5. The test ran Muse Spark Open, an 816-billion-parameter mixture-of-experts model, on four machines and produced more than 36 tokens per second.
The work is intended to show how models too large for a single desktop can be divided among multiple Apple-silicon systems. In its native BF16 precision, Muse Spark Open would require more than 1.6 terabytes of memory for parameters alone. Quantizing it to NVFP4 reduces the model footprint to about 493GB, but additional memory needed during inference still prevents it from fitting on one machine.
Apple's setup splits the model's layers between computers through pipeline parallelism. One Mac Studio processes an initial set of layers and sends the intermediate result to the next machine, which continues the calculation. Because the layers remain sequential, generation speed can remain close to single-machine performance when communication overhead is low.
That communication uses an RDMA-over-Thunderbolt driver introduced with macOS 26.2. MLX version 0.30.1 and later exposes communication operations through a backend called jaccl. In the two-machine configuration described by Apple, the researchers connected the systems with two Thunderbolt cables to increase link bandwidth. MLX's launch utility then coordinated work across the nodes.
The researchers also benchmarked smaller models on individual Mac Studios. Mistral Medium 3.5, with 128 billion parameters, and Qwen 3.8, with 27 billion, fit on a machine configured with 512GB of unified memory. At a context length of 65,536 tokens, Apple reported memory use of up to 158GB for Mistral and 68GB for Qwen.
Comparisons between M5 Ultra and M3 Ultra systems showed different gains at different stages of inference. Apple measured time to first token as much as 4.6 times faster on M5 Ultra, attributing the improvement to dedicated matrix-multiplication hardware in the GPU. Generation throughput rose by as much as 1.6 times, alongside an increase in unified-memory bandwidth from 819GB per second to 1.2TB per second.
Apple reported peak power use of roughly 200 watts for an M3 Ultra Mac Studio and 300 watts for the M5 Ultra version during its tests. The results are vendor benchmarks rather than independent performance measurements, but they provide a reproducible outline of how MLX can scale local inference beyond one computer while keeping model data on systems controlled by the user.
