De-Escalate: Expressive Humanoid Persona

MIT Reality Hack 2026 · Personoid / team “The Future is Chrome” · Samsung GalaxyXR → Booster K1 humanoid

Devpost (gallery + long-form submission)Code (Caerii fork)Original team repo

At MIT Reality Hack 2026 our team shipped De-Escalate: Expressive Humanoid Persona — a mixed-reality experience on Samsung GalaxyXR where you encounter a physical Booster K1 humanoid that can look and move like it is under emotional , and your job is to help calm it down using voice, hands, gaze, and UI. The project name on the marquee is playful; the engineering goal was serious: a credible pipeline from XR intent to real robot motion, with LLM-generated keyframe sequences, Gemini Live voice loops, stock emotional animations, and enough safety discipline that we could demo on hardware without fooling ourselves.

What we were trying to prove

Humanoid platforms are moving from novelty to infrastructure. If you want people to practice empathy, de-escalation, or bedside manner, a screen-based sim only gets you part of the way; weight, timing, and the sheer uncanniness of a real mechanism matter. We set out to show that a hackathon-scale team could still integrate a serious SDK, a modern XR client, and modern language/voice models without hiding behind canned clips — and that you could preview motion in 3D before committing torque on the floor.

End-to-end flow

GalaxyXR runs Unity 6000 with Android XR / OpenXR: passthrough, hands, gaze, and Marcel's XR UI foundation (with TextMesh Pro work from Daniel). User actions and utterances fan out to services: a FastAPI layer exposes generation and execution endpoints (natural-language prompts to structured keyframes, stock emotional clips, execution hooks), while separate Gemini Live paths cover robot-directed voice control and Josh's voice-concierge layer with mood visualization. Below Python sits the Booster Robotics SDK — DDS-backed real-time control, IK, and end-effector commands — talking to the K1. A React + Three.js front end (Vite) loads the robot meshes, previews , and sanity-checks workspace bounds before anyone hits “go” on hardware.

LLM keyframes and safety

The animation server accepts prompts, calls Gemini 2.0 Flash, OpenAI, or Anthropic with structured outputs, validates joint targets against a tight workspace envelope, and streams executable sequences. Stock “emotion” libraries (tens of keyframes per affect) gave us dependable demos when the network misbehaved. We treated the preview app as mandatory, not decorative: if a sequence looked wrong in Three.js, it did not graduate to the robot.

What made it hard

Three teams sharing one K1 battery; copying builds onto the robot under weak docs; Gemini Live over conference WiFi WebSockets; WSL networking pain for DDS on laptops (we often ran Python on the robot itself and ate latency); and a winter storm that ate polish time. None of that shows up in a clean architecture diagram, but it is the real schedule. The reason the repo has as much documentation as code is that the next integrator will hit the same walls.

Recognition

Judges awarded us Runner-up — Google DeepMind Prize for the overall integration of LLM control, voice, and multimodal XR around a physical humanoid, and a Reality Amplifier Prize for how we showed up for other teams during the event. Track-specific wording and screenshots live on Devpost.

Team

The Future is Chrome: Daniel Trujillo (GalaxyXR build, Android XR, TMP integration), Marcel Bornstein (XR UI system, art, audio), Josh Valenzuela (Gemini Live voice concierge and mood UI), Alif Jakir (Booster integration, Python keyframe stack, Unity service layer, Three.js preview, documentation), Jose E. Pimentel (Arduino / architecture support; App Lab bridge experimental). The public fork I maintain is Caerii/Personoid-MITRealityHack-2026; the original submission tree is danieljtrujillo/The-Future-is-Chrome-MIT-Reality-Hack-2026.

What I worked on

I owned the robot control vertical: extending the Booster Python examples into a keyframe animation service, wiring FastAPI routes and validation, building the Unity C# clients (HTTP, scene hooks, XR events), converting LLM postures into something the SDK accepts, integrating Gemini Live audio into robot actions, standing up the React/Three preview, and writing the architecture and safety notes so the next person does not have to rediscover DDS ports at 3 a.m.

Personal note

TODO: I still want to add a first-person essay here — what it felt like to share one robot with half the hall, negotiate merge conflicts against a charging timer, and trust teammates across totally different stacks — separate from the technical narrative above and the story on Devpost.

Alif Jakir