Whole-Body Manipulation and Why Suits Beat Cameras

Lift a heavy pot from the back of a shelf. Your knees bend. Your back straightens. Your shoulders rotate. Your fingers adjust their grip as the weight shifts. That is one action, distributed across the body. The hand did not do it. The body did it, and the hand was the endpoint.

Almost every real household task recruits the whole body in this way. A robot that reasons about manipulation as a purely arm-and-hand problem is a robot that will look competent on a bench and fail the moment it has to operate in a real space.

A camera can see the pot moving. It cannot see that the operator shifted weight three seconds earlier to prevent a fall, or that the operator's grip force ramped up in anticipation of the object's mass before the object had moved at all. These preparatory motions are invisible to vision and essential to policy.

The problem compounds when the robot is embodied differently from the operator. A humanoid trained on hand-only data has no priors about how its own hips should shift under load. It will try to solve every task with the arm and fail on any load that exceeds the arm's capacity.

A full-body suit records the coordination as a single trajectory. The policy sees the pre-lift weight shift, the joint sequence during the lift, and the recovery posture afterwards. This is what humanoids, quadrupeds with arms, and mobile manipulators need to learn safe operation.

The training payoff is largest for tasks where the manipulation is coupled to locomotion: opening a fridge and stepping back, carrying groceries across a room, reaching to the top of a shelf. Arm-only data teaches the arm; whole-body data teaches the plan.

Whole-body data is useful even for robots that do not look like humans. A wheeled base with two arms benefits from knowing which arm the operator used and when, and how the operator's body oriented toward the task. A quadruped with a manipulator arm benefits from knowing how the operator's whole body counter-balanced during the reach. The retargeting is more work, but the signal is real.

Our whole-body sessions include 39 IMU nodes across the body, tri-modal video (headband, wrist, external), per-finger force from the glove, and foot-contact events from pressure-sensitive insoles when the task involves stance changes. The output is a single time-locked trajectory that a policy can consume as a coherent whole rather than a set of loosely related streams.

If your robot has more than one arm, or feet, or a mobile base, whole-body capture is not a nice-to-have. It is the data format that matches the physical problem. Anything less leaves capability on the table.