Problems · Before you buy
What should I check before buying a laptop for running AI models?
Memory is the wall. A model either fits in the memory available to the accelerator or it does not run usefully at all, so check capacity first, then memory bandwidth, then sustained power, then everything else. No amount of processor grade rescues a machine whose memory is too small.
On Windows machines that means the graphics card's VRAM; on Apple Silicon it means unified memory, most of which the graphics side can address. The two are not interchangeable but the question is the same: how much, and how fast.
First, notice when it happens
The causes, ranked
1. Memory capacity, the hard wall
most commonHow to confirm it
Work backwards from the models you want to run. As a rough guide, a quantised model needs roughly its parameter count in gigabytes at 8-bit and about half that at 4-bit, plus headroom for context. 8GB is genuinely limiting; 16GB runs small models comfortably; 24GB and above opens the mid-sized ones.
The fix
Buy the largest memory you can afford, once. This is soldered on Apple Silicon and fixed on every laptop graphics card, so it is the least upgradeable decision in the machine.
2. Memory bandwidth, which sets the speed
most commonHow to confirm it
Token generation is bandwidth-bound rather than compute-bound for most local models. Check the bandwidth figure in gigabytes per second; it varies enormously between tiers and between the base and higher Apple Silicon chips.
The fix
Compare bandwidth explicitly when two machines offer the same memory size. The one with more bandwidth generates faster on the same model, and the gap is larger than most buyers expect.
3. Sustained power and thermals
commonHow to confirm it
A laptop graphics card configured at a low wattage performs far below the same chip at full wattage, and a thin chassis throttles during a long run. The badge is the same, the delivered performance is not.
The fix
Find the configured wattage in a review, and run a sustained test on the unit. Long generations and any training are sustained workloads, which is exactly where laptops disappoint against their specification.
4. Unified memory versus dedicated VRAM
commonHow to confirm it
Apple Silicon can address most of its unified memory for models, which lets a 32GB or 64GB machine run models that no consumer laptop graphics card can hold. Windows machines offer more raw compute per unit of money but far less addressable memory.
The fix
Choose by what you actually run: large models favour unified memory, heavy training on smaller models favours a high-wattage dedicated GPU with the software ecosystem to match.
5. Software and framework support
commonHow to confirm it
Check that your tools support the platform natively rather than through a translation layer. Support quality varies by framework and changes quickly.
The fix
Verify with your specific tools before buying, not after. This is where a theoretically capable machine becomes practically frustrating.
6. Storage for the model files themselves
occasionalHow to confirm it
Model weights are large and accumulate quickly. 1TB fills faster than people expect once several models coexist with everything else.
The fix
1TB minimum, and prefer a machine whose drive can be replaced or supplemented.
Measure it instead of guessing
Run the sustained test before committing, because a long generation is exactly the load that separates a machine's peak from its settled level. Combined with the memory and bandwidth figures from the specification page, it gives you the two numbers that actually predict local AI performance.
What software cannot tell you here
- We cannot measure VRAM, unified memory or bandwidth; those are specification-page figures.
- We cannot test a specific model's performance in a browser. What we can show is whether the machine holds power under a long load, which is the part reviews rarely publish.
Asked, in people’s own words
- How much VRAM do I need to run local AI models?
- Roughly the parameter count in gigabytes at 8-bit quantisation, about half at 4-bit, plus headroom for context. That makes 8GB genuinely limiting, 16GB comfortable for small models, and 24GB or more the entry point for mid-sized ones. Capacity decides what runs; bandwidth decides how fast.
- Is a MacBook good for local AI?
- For running large models, unusually good, because unified memory lets a well-specified machine hold models that no consumer laptop graphics card can fit. For heavy training and for frameworks with the deepest support, a high-wattage dedicated GPU still leads. The choice follows your workload rather than the brand.
- Can a laptop replace a desktop for AI work?
- For inference and experimentation, frequently yes. For sustained training, laptops lose on power and cooling: the same chip at a laptop's wattage delivers a fraction of the desktop card's throughput, and a thin chassis throttles during long runs. Many people buy a capable laptop for inference and rent compute for training.