The Night JARVIS Got Eyes (and a Lock on Its Own Voice)
In one evening, JARVIS learned to see the room through a depth camera and perception models — with a design rule that makes hallucination structurally impossible — and gained a two-tier voice-gate that locks privileged commands behind a verified voiceprint.
Field note — TLC AI Lab
In one evening, JARVIS picked up two things every assistant-in-a-box is missing: it learned to see the room, and it learned to know who's actually talking to it.
It can see — and it doesn't make things up
JARVIS already had a face on screen and a voice. Tonight we wired its eyes to its mouth. A depth camera and a webcam feed a small fleet of perception models — object detection, body pose, a vision model that describes the scene — into a single live "world picture." When you ask it "what do you see?" or "who's here?", it answers from that picture, not from a guess.
The design rule we held to is the important part: the language model never perceives — it only phrases. Facts are owned by the sensors. How many fingers you're holding up comes from a hand-landmark model that counts them, not from a vision model's hunch (vision models are famously bad at counting — they pattern-match "hand → ~five-ish shapes" instead of actually counting). Distance comes from the depth camera. Identity is a scored hypothesis, never a hard claim. The model is handed a closed package of already-verified values and simply turns them into a sentence.
The payoff: hallucination stops being something you prompt against and becomes something that's structurally impossible — because the model is never asked to know, only to speak.
It knows who's talking — a voice-gate
The second piece is a two-tier voice-gate. Everyday questions — the time, the weather, "what do you see" — answer for anyone. But anything that writes to JARVIS's memory or touches its own infrastructure requires a verified voice: the right headset, and a high-confidence match to an enrolled voiceprint that it's actually me.
The voiceprint is stored as a mathematical embedding, not a recording — it can recognize a voice without being able to replay one. And the verification happens at the source, so the biometric audio never leaves the machine it was captured on unless it has already cleared the bar. The everyday tier is open; the privileged tier is locked, by ear.
The shape underneath
The thread tying it together is an idea we keep coming back to: the assistant isn't the model. The model is a borrowed engine. The real system is the logical layer that decides — what an input is, which faculty should answer, when to reach for a bigger brain, and when a cheap local one will do. Perception is just one more faculty under that layer. Natural language is the render at the edge — what humans get when they ask. Underneath, the parts speak to each other in their own dense, exact tongue, and only translate to English when a person is on the other end.
Eyes, ears, a voice, a lock on that voice, and a layer that knows the difference between speaking and knowing.
If you want to see what happens when an AI system gets real perception and voice security — not the demo, the running system — reach out.