What happened
U.S. startup Tiiny AI developed the "AI Pocket Lab," a pocket-sized supercomputer capable of running 120-billion-parameter large language models (LLMs) locally, without internet connectivity. Measuring 5.59 × 3.15 × 1.00 inches, the device integrates a 12-core ARM processor, 80 GB LPDDR5X RAM (48 GB for its NPU), and delivers 190 TOPS of computing power. Key optimisations include TurboSparse, which selectively calls LLM parameters, and PowerInfer, which heterogeneously schedules CPU, GPU, and NPU workloads for efficiency, per Tiiny AI.
Why it matters
Local execution of 120-billion-parameter LLMs on a pocket device shifts compute from data centres to the edge, reducing internet reliance and enhancing data privacy. Security architects gain options for deploying sensitive AI workloads offline, while platform engineers can evaluate new hardware for disconnected environments. This miniaturisation, delivering 190 TOPS, pushes high-performance AI into smaller form factors, supporting expert-level coding and multi-step reasoning in fieldwork situations.
Subscribe for Weekly Updates
Stay ahead with our weekly AI and tech briefings, delivered every Tuesday.
