🔍 Read the full analysis: MPS In PyTorch 2.15: A Guide For AI Workloads on ThorstenMeyerAI.com
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
PyTorch 2.15 adds FP8 tensor support and performance changes to its Metal Performance Shaders backend for Apple silicon. Hugging Face reports large gains in selected model and video-decoding tests, but says results depend on workload and setup. The release requires macOS 15 or later and Python 3.11 or later.
PyTorch 2.15 adds FP8 tensor support and new performance paths to its Metal Performance Shaders (MPS) backend for Apple silicon, according to the PyTorch team at Hugging Face. The team reports faster decoding in tests of two mixture-of-experts models and lower peak memory in a Wan2.2 video-decoding workload, but cautions that results are specific to the models and configurations tested. The release requires macOS 15 or later and Python 3.11 or later.
In tests on an M3 Ultra using torch.compile, the Hugging Face team measured Qwen3.6-35B-A3B decoding at 61.3 tokens per second with PyTorch 2.15, compared with 9.4 tokens per second on 2.14. For LFM2.5-8B-A1B, the reported figures were 123.6 tokens per second on 2.15 and 23.6 on 2.14. These are measurements for the named models and setup, not a forecast for other Macs or workloads.
The release routes eligible single-row linear layers through specialized matrix-vector kernels and fuses some bias additions, targeting the repeated operations involved in language-model decoding. Hugging Face reports a 1.23× geometric-mean speedup across 96 BF16 layer shapes drawn from real models, and says vocabulary softmax was about eight times faster in its measurements. MPS also gains a native grouped matrix multiplication path for mixture-of-experts decoding, alongside changes to reduce GPU-to-CPU synchronization and speed up top-k routing.
PyTorch 2.15 supports torch.float8_e4m3fn tensors on MPS. FP8 values use one byte, half the storage of BF16 values, and the team says scaled matrix multiplication can use tensorwise or rowwise scaling. That storage difference does not by itself establish a specific end-to-end memory or accuracy gain; those depend on the application and its scaling choices. Other listed MPS work covers complex LU and triangular solves, QR decomposition, convolutions, padding, dropout, interpolation backward, and reductions.
Faster Local Inference on Macs
The update targets two practical constraints for running models locally: per-token response time and GPU memory use. Faster decoding could make supported language models more responsive on an Apple silicon Mac, while FP8’s smaller data representation may help some applications fit workloads into available memory. The Wan2.2 result, if it holds in comparable configurations, could also matter to video-generation users whose pipelines are limited by memory.
Hugging Face reports that tiled Wan2.2 VAE decoding peaked at about 8 GiB in PyTorch 2.15, compared with 270 GiB in its earlier test. It also reports a quality-related fix: a Qwen-Image-2.1 VAE encode-decode round trip scored 29.95 dB PSNR on 2.15, versus 10.06 dB on 2.14. These figures suggest that backend changes can affect both resource use and output fidelity, but they do not establish how the changes perform across other image or video pipelines.
For developers and users, the practical implication is to test the release against their own models and settings. A benchmark on one chip cannot establish equivalent results across Apple silicon generations, and faster kernels are useful only if the workload produces acceptable outputs and remains stable.
Apple Silicon Mac compatible AI model inference hardware
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
How MPS Performance Has Changed
MPS is PyTorch’s backend for using Apple’s Metal GPU framework. The Hugging Face team says that, at the start of 2026, some models on MPS encountered unsupported operations, performance losses, or unexpectedly high memory use. Its approach has included testing models from the Hugging Face Hub and working with PyTorch maintainers and community contributors on upstream fixes, including replacing some MPSGraph paths with dedicated Metal kernels.
PyTorch 2.15 builds on work in version 2.14, which had already introduced matrix-vector kernels for torch.mm. The new release connects eligible F.linear operations to those kernels, focusing on single-row inputs common during language-model decoding. The supplied material also previews possible work for version 2.16, including moving attention more fully onto Metal and further MPSGraph-to-Metal migrations. Those items are plans, not features confirmed as released in 2.15.
“The goal is to make your Mac a first-class place to run and train PyTorch models.”
— PyTorch team at Hugging Face
PyTorch compatible M1 M2 Mac GPU acceleration tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Benchmark Limits and Open Questions
The reported numbers come from tests shared by Hugging Face, and the supplied material does not include enough detail to independently reproduce every result. The tokens-per-second comparisons identify an M3 Ultra and torch.compile, but do not establish performance on other chips or with different software versions, model settings, or generation parameters. The team says results vary by workload.
It is also unclear how widely the Wan2.2 memory result applies across video resolutions, pipeline settings, or other systems. The Qwen-Image-2.1 result uses PSNR, but broader image-quality evaluations are not provided. FP8 tensor and scaled matrix multiplication support are described in the release; practical accuracy and memory effects will depend on how each application uses them. No independent benchmark or broad cross-device comparison is included in the supplied material.
FP8 tensor support for machine learning
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Test Workloads and Watch 2.16
Users considering the update should check compatibility before changing their environment: MPS requires macOS 15 or later, while PyTorch 2.15 requires Python 3.11 or later. The release material says MPS availability checks now explain when an older operating system or the absence of a Metal GPU prevents MPS from being used.
The next practical step is to compare speed, memory use, and output quality with the specific models and Mac configurations users rely on. Hugging Face has previewed further MPS work for PyTorch 2.16, including attention running more fully on Metal, additional MPSGraph migrations, and faster LLM decoding. Those remain planned developments; no release date or final feature details are established in the material provided.
video decoding hardware acceleration Mac
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
What is new for MPS in PyTorch 2.15?
PyTorch 2.15 adds FP8 tensor support on MPS, specialized matrix-vector paths for eligible linear layers, and a native grouped matrix multiplication path for mixture-of-experts decoding. It also includes other operation and backend changes.
Do the reported decoding speeds apply to every Apple silicon Mac?
No. The reported Qwen3.6-35B-A3B and LFM2.5-8B-A1B results were measured on an M3 Ultra with torch.compile. They do not establish performance for other chips, models, or software configurations.
What are the minimum requirements?
The supplied release material lists macOS 15 or later for MPS and Python 3.11 or later for PyTorch 2.15.
Does FP8 automatically halve an application’s memory use?
No. FP8 values use half the storage of BF16 values, but the overall memory effect depends on how an application stores and processes its tensors. Accuracy and performance also depend on the scaling choices and workload.
Are the PyTorch 2.16 improvements available now?
Not according to the supplied material. More complete Metal attention support, additional MPSGraph migrations, and faster decoding are described as planned or previewed work, not confirmed PyTorch 2.15 features.
Primary source: Hugging Face · via ThorstenMeyerAI.com
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
