Google released LiteRT.js on July 9, 2026, a JavaScript runtime that runs AI models directly inside the browser with no server round-trip. Google says it outperforms other web runtimes by up to 3x on CPU and GPU inference, and that hardware acceleration delivers a 5 to 60x speedup over plain CPU execution. It ships as the @litertjs/core npm package and positions itself as the modern successor to TensorFlow.js for developers who want on-device inference on the web.
What Happened
LiteRT is Google's on-device inference engine, formerly known as TFLite, used across Android and embedded devices. LiteRT.js brings that same engine to the browser through a JavaScript binding, so a web app can load a model and run it entirely on the user's machine. The pitch is three practical wins: privacy, because data never leaves the device; cost, because there is no inference server to pay for; and latency, because there is no network hop between a user action and the model's response.
This is an initial release, but it arrives with the pieces that make on-device web AI usable in production: a model converter, a quantization tool, and acceleration across CPU, GPU, and experimental NPU backends. For anyone who tried to ship browser inference with earlier tools and hit performance walls, the headline claim of up to 3x faster execution is the reason to look again.
LiteRT.js vs TensorFlow.js
The obvious comparison is TensorFlow.js, Google's earlier browser ML library. LiteRT.js is not a drop-in replacement, but it targets the same job with a newer engine and a different model pipeline.
| Dimension | TensorFlow.js | LiteRT.js |
|---|---|---|
| Engine | TensorFlow.js runtime | LiteRT (formerly TFLite) via WebAssembly |
| Model format | TF.js / Keras models | .tflite models, PyTorch via LiteRT Torch |
| GPU backend | WebGL | ML Drift on WebGPU |
| NPU support | None | WebNN (experimental, Chrome/Edge) |
| Quantization | Limited | AI Edge Quantizer |
| Reported speed | Baseline | Up to 3x faster on CPU and GPU |
The two details that matter most for a builder are the PyTorch path and the WebGPU backend. Being able to convert a PyTorch model with LiteRT Torch means you are not locked into the TensorFlow model ecosystem, and running the GPU backend on WebGPU rather than the older WebGL keeps LiteRT.js aligned with where browser graphics and compute are actually heading.
The Backends and What They Buy You
LiteRT.js exposes three acceleration targets, and the speedup you get depends on which one the user's browser and hardware support:
- CPU (XNNPACK): multi-threaded SIMD execution, the universal fallback that works everywhere.
- GPU (ML Drift on WebGPU): the main performance lever, where the 5 to 60x gains over CPU come from.
- NPU (WebNN): experimental support in Chrome and Edge that routes to a dedicated neural accelerator when one is present.
Because support varies by device, the realistic approach is to target GPU where available and fall back to CPU gracefully. The wide 5 to 60x range reflects exactly that: the gain depends on the model, the browser, and the silicon, so the only honest number is the one you measure on your own target devices.
What This Enables for Builders
On-device inference changes the economics of AI features in web apps. A background-removal tool, a real-time pose or hand tracker, an on-page transcription widget, or a small language model for smart suggestions can all run without a GPU server behind them. That removes per-request inference cost and the privacy burden of shipping user data to a backend, which is especially valuable for creative tools that handle images, audio, or webcam input.
It also fits a broader move toward local and in-browser AI. We have covered the same trend with small runtimes like the 7MB Ternlight embedding model in WebAssembly and with local AI workstations. LiteRT.js is the general-purpose runtime layer for that world: bring a converted model, pick a backend, and ship inference that lives on the user's device.
How to Try It
The path from a trained model to browser inference is short once the model is in the right format:
- Install the runtime with
npm i @litertjs/corein your web project. - Convert your model to
.tflite, using LiteRT Torch if you are starting from PyTorch, and optionally shrink it with the AI Edge Quantizer. - Load the LiteRT WebAssembly module and compile the model with your chosen accelerator (GPU via WebGPU where supported, CPU otherwise).
- Create input tensors, run inference, and read the output tensors back.
- Benchmark on your actual target devices, since the 3x and 5 to 60x figures are ceilings, not guarantees.
Google ships demos alongside the package and documents the full engine on the LiteRT site, which is the reference for supported ops and conversion details. If TensorFlow.js was your previous tool, the migration cost is mostly in the model-conversion step, not the inference code.
Frequently Asked Questions
What is LiteRT.js?
It is a JavaScript binding of Google's LiteRT (formerly TFLite) inference engine that runs AI models directly in the web browser, entirely on the user's device, with no inference server required.
How much faster is it than other web runtimes?
Google reports up to 3x faster CPU and GPU inference than other web runtimes, and a 5 to 60x speedup from GPU or NPU acceleration versus plain CPU execution. Actual gains depend on the model, browser, and hardware.
Can I use my PyTorch models?
Yes. LiteRT Torch converts PyTorch models to the .tflite format LiteRT.js runs, so you are not limited to TensorFlow-native models.
Which browsers and hardware does it support?
CPU execution via XNNPACK works broadly. GPU acceleration uses WebGPU through the ML Drift backend, and experimental NPU support uses the WebNN API in Chrome and Edge. Design for a CPU fallback where GPU or NPU is unavailable.
Should I switch from TensorFlow.js?
If you need better performance, WebGPU acceleration, PyTorch model support, or stronger quantization, LiteRT.js is the newer engine built for that. The main migration work is converting your model; the inference code is straightforward.
Is it production-ready?
It is an initial release available as @litertjs/core on npm with demos and documentation. The core engine is mature from its Android and embedded history, but treat the web binding and experimental NPU path as early and benchmark before shipping.
