Apple has officially entered the race for general-purpose embodied artificial intelligence. In a landmark scientific paper and open-source release titled Perceptive Humanoid Motion from Simulation (arXiv:2611.06908), an Apple AI and ML research team led by Vladlen Koltun, Siyuan Hu, Abulikemu Abuduweili, and colleagues introduced HALO (Humanoid Autonomy, Learned in Simulation, running Onboard).
The system solves the central bottleneck holding back bipedal robots: the scarcity of paired real-world sensorimotor training data. Instead of spending thousands of hours teleoperating physical robots with cumbersome VR headsets or motion-capture suits, Apple trained HALO entirely inside 1,000 procedurally generated virtual homes. Deployed zero-shot on an off-the-shelf Unitree G1 humanoid robot equipped with a consumer Apple Mac mini M4 Pro strapped to its back, HALO achieved an unprecedented 91% success rate across 360 physical trials in unmapped real homes.
Bypassing the Teleoperation Bottleneck with Synthetic Physics
For years, commercial humanoid initiatives like Tesla Optimus, Figure AI, and Sanctuary AI have relied heavily on teleoperation: human operators wearing motion-tracking suits or VR controllers to guide robot arms through repetitive demonstrations. This approach scales linearly with expensive human labor, requiring at least one human hour to capture one hour of robot data.
Apple took an entirely different path: compute-driven synthetic data generation:
- 1,000 Procedural Virtual Homes: Using procedural simulation engines like Infinigen and Isaac Gym, the researchers generated 1,000 complete 3D houses spanning 48,213 individual rooms and realistic furniture layouts.
- 33 Geometric and Dynamic Constraints: An automated kinematic motion planner (Kimodo) proposed candidate actions (walking forward, turning, sitting, crawling), filtering out sliding, ground penetration, and kinematic singularities through 33 strict geometric checks.
- Aggressive Domain Randomization: Simulated cameras reproduced 8 distinct optical artifacts, including extreme lighting changes, motion blur, salt-and-pepper sensor noise, and color temperature shifts.
- Massive GPU Compute Fleet: Generating the dataset utilized 250 NVIDIA H100 GPUs, while training the motion generator consumed 64 NVIDIA B200 GPUs and tracker reinforcement learning ran on 4 H100 GPUs.
Without ever seeing a real physical residence during training, the policy acquired a diverse repertoire of complex motor skills: sitting on varied chairs and soft sofas, navigating toward designated objects, crawling on all fours under low office desks, sidestepping through narrow doors, and pushing heavy storage boxes across rooms.
Hierarchical Onboard Architecture: Apple Silicon Meets Embedded Edge
Rather than offloading neural network inference to remote cloud data centers, which introduces dangerous latency and fails whenever Wi-Fi drops, Apple designed HALO to execute 100% onboard the robot. The hardware package combines the bipedal chassis of the Unitree G1 with two complementary compute units:
- The Backpack Compute Unit (Apple Mac mini M4 Pro): A stock Mac mini equipped with an Apple M4 Pro chip and 64 GB of unified memory (delivering 273 GB/s of unified memory bandwidth) is mounted inside a custom 3D-printed enclosure on the robot’s spine. It processes inputs from an egocentric 120-degree RGB camera (OAK-1W).
- VLM Task Planner (~0.5 Hz): Runs an onboard vision-language foundation model (Qwen-based architecture) every 2 seconds to interpret human speech and parse multi-step workflows into atomic sub-goals.
- Motion Generator (~3 Hz): A Diffusion Transformer (DiT) decoder operating with flow matching and 15 Euler integration steps predicts 5 seconds of whole-body motion (150 frames across 36 kinematic parameters). The first 0.5 seconds (15 frames) are committed and fed back as clean state tokens.
- The Embedded Chassis Unit (NVIDIA Jetson Orin NX): Mounted inside the torso of the Unitree G1, this embedded SoC reads egocentric depth imagery from a downward-facing 55-degree sensor alongside joint proprioception.
- Motion Tracker (50 Hz): A neural policy trained via reinforcement learning in MuJoCo tracks the committed kinematic trajectory at 50 Hz, calculating exact torque and angle setpoints for 29 articulated joints.
Zero-Shot Physical Benchmarks: 91% Success Rate Across 360 Real Trials
To test whether synthetic training actually works in the messy physical world, Apple evaluated HALO across six completely unmapped real-world environments, ranging from modern apartments and office kitchens to outdoor wooden decks.
Across 360 separate physical trials, HALO registered an overall success rate of 91%:
- Chair and Sofa Sitting: 84% to 92% success rate, even when navigating around clutter to reach unfamiliar chairs.
- Under-Table Crawling: 94% success rate crawling beneath low desks and tables where conventional upright bipedal controllers cannot venture.
- Narrow Door Passage: Successfully sidestepping through doors opened just wide enough for the robot’s shoulders.
- Chained Long-Horizon Tasks: The VLM orchestrator autonomously executed composite missions: navigating a room, ducking under an obstacle, pushing aside an obstructing container, following a moving person, and seating itself on a sofa upon arrival.
By publishing the complete training pipeline, simulation assets, and deployment codebase on GitHub, Apple has delivered a major gift to academic and commercial robotics researchers. The demonstration proves that consumer-grade silicon and synthetic physics can replace millions of dollars of manual teleoperation, marking a pivotal turning point for autonomous humanoid deployment. To explore how the open-source Unitree G1 platform stacks up against industrial factory-floor humanoids, see our in-depth comparison ASUS Gallop vs Unitree G1.
