Sparse 3D Traversal, ETH Zürich’s PM-01 Humanoid Swings the Monkey Bars on Raw LiDAR

Humanoid robots can now handle monkey bars. Researchers at ETH Zürich’s Robotic Systems Lab showed an EngineAI PM-01 leaping onto a sparse overhead ladder, swinging hand over hand from one thin rung to the next, and dropping cleanly to the floor in a single unbroken sequence. The work, titled Learning Agile Perceptive Traversal of Sparse 3D Structures for Humanoids, treats the playground classic as a hard test of seeing thin, hanging geometry and then moving the whole body with split-second accuracy.

Fourteen of fifteen hardware trials successfully completed the entire jump-up, brachiation, and jump-down sequence across three distinct bar configurations. The bar heights varied from 1.69 to 1.75 meters, and the spacings ranged from 0.26 to 0.33 meters. Peak swing speeds reached 0.5 meters per second, which is comparable to what we’ve witnessed in humans using the same gear. Guess what? One of the runs even made it work on a rickety frame that began to tremble under the strain. Later, the same control system allowed a policy that had been trained separately to duck beneath some very narrow overhead slats, only 2 cm on each side, and then recover to a stable position.

A head-mounted RoboSense E1R solid-state LiDAR generates raw data, which is fed directly into the controller. This device can sweep a 120-by-90-degree field at 10 Hz and create a 192-by-144 pattern. We have an attention-based encoder called AME-2 that sets up on a 2D grid of these points, pools the global picture, and then focuses on the local features that are important the following time around. A GRU memory keeps track of where the bars are even when they move out of view, and we use joint position targets to deliver 50 Hz signals to the onboard PD loops. There’s even an auxiliary head that makes an educated guess about the centerline of the ladder, providing an additional piece of training data.

Their initial step was to have a “expert” policy understand each phase individually, with complete knowledge of where the bar ends were. They had three of these specialists set up: one for jumping onto the first rung, one for swinging, and one for lowering down. A phase scheduler enabled us to pass control from one expert to the next. Then they utilized distillation to combine all of these specialists into a single student who had to learn from scratch based on what it saw and felt. Later, they employed PPO refining to incorporate some imitation learning into the task reward while gradually turning off the imitation signals. This training was performed in the Isaac Lab with extensive domain randomization, as they experimented with bar radius, width, tilt, friction, mass, battery-voltage sag, actuator heat limits, and even a detailed lidar noise model with range error, edge bleed, dropouts, and frame freezes.

Instead of hands, it has passive stainless-steel hooks on the end of the robot, each having a 60-millimeter circular aperture to accommodate the test bars. The robot can hang its entire weight and is unconcerned about minor contact faults because it can release by merely twisting the wrist joint. They also have a symmetrical design that allows the robot to swing backward if necessary, which is a convenient feature. The simple shapes used to build the contact geometry allow us to keep the contact computation simple during simulation.

Using onboard sensing and taking into account the modeled hardware restrictions has significantly reduced the gap between simulation and real-world performance. Attention maps from both the sim and the real robot illuminate exactly the appropriate places, such as the next rung and the ground plane, at precisely the right times. Previous controllers that used map-based or elevation-mapped data struggled to keep up with thin overhead geometry, and voxel grids just took up too much memory.They also have a history of swinging machines that presume known bar placements, but their PM-01 is the first reported example of a general-purpose humanoid robot doing the entire jump-swing-land sequence using its own LiDAR.