Research · AI · On-Device AI · Spatial Computing · Research
Latent Compute: Devices That Learn What They Can Sense
Phones and laptops carry sensors and telemetry that operating systems routinely average away. Latent Compute is an H3M research direction asking whether a small on-device language model performs better with context from the device it runs on. The test is designed, with seven gates set before building, and has not yet been run.
· Ryan Brant
Every phone and laptop is full of sensors, emitters, and telemetry: motion, light, sound, radio, thermals, power. Most of that signal never reaches the software that could use it. Operating systems and apps routinely filter it, average it, or throw it away.
At the same time, small language models are moving onto devices. Latent Compute is an H3M research direction that connects the two, and it starts with a measurable question: does a small language model with context from the device it runs on outperform the same model without it?
This is a research direction, designed but not yet measured. Here is how it is meant to work, and how we will know whether it does.
Take inventory
On install, the runtime finds what the device can actually sense and characterises how reliable each source is. No two devices are the same, so the inventory is specific to the hardware it runs on.
Wrap each source
Raw sensor feeds arrive in different formats, rates, and units. Each family of sensors and telemetry is wrapped into a module with the same shape, so everything downstream can treat them consistently.
Learn the device
Readouts specific to each device are learned by self-supervision, without labels. The runtime learns to predict hidden stretches of signal, and in doing so learns the calibration of the particular device it lives on.
Offer stable signals
Local agents read the result through a single context interface of stable signals that don't depend on the hardware. A phone and a laptop return the same keys, so an on-device model or agent can use context without knowing what chip or sensor produced it.
Decide how you'll know before you build
The go/no-go test is specified before any platform work: the same small model, with and without runtime context, on the same hardware. Seven gates are set in advance, and passing all seven at once is the bar. None has been run yet.
That ordering is deliberate. It is easy to build an impressive runtime and then look for a reason it helps. Fixing the test first keeps the research honest.
Why it matters for spatial computing
Spatial computing depends on devices understanding their surroundings: where they are, how they are moving, what the room sounds and looks like. If small on-device models can draw on the full signal a device already carries, the next generation of AR and AI experiences can be more aware, more private, and less dependent on the cloud.
Decide how you'll know it works before you build it.