Aligned Multi-Stream Capture: The Wearable Advantage

The word alignment does a lot of work in modern robot learning and almost none of it is visible in the model architecture. It happens upstream, at the capture rig, and it determines whether the multimodal fusion techniques the field has been developing for the last few years can actually be applied to the resulting data. A wearable capture stack designed for alignment produces training data that a well-designed multimodal model can consume; a stack that treats alignment as an afterthought does not.

Alignment is the property that a physical event occurring in the world produces coincident, correctly-timestamped observations across every modality that could have observed it. When a fingertip contacts an object, the force signal shows the contact at the same physical instant that the video shows it and the pose stream shows the hand at the contact position. If the timestamps are off by tens of milliseconds, a small amount by wall-clock standards, the training signal is quietly poisoned. A model asked to learn the relationship between visual state and force outcome sees relationships that are physically impossible, and adjusts its representations accordingly.

Sub-millisecond alignment is the working target for serious multimodal training. Achieving it is not glamorous. It requires shared clock domains across sensors, hardware timestamp support at the sensor level rather than at the packet-arrival level, and careful handling of variable-rate streams so that resampling does not smear physical events into wall-clock ambiguity.

In a wearable capture rig, every sensor is close to every other sensor and to the operator whose actions are being captured. The physical distances are short. The wireless or wired links have low latency. The clock synchronization problem is small, and the mechanical mounting means that the pose sensors, force sensors, and cameras all move together as a single instrument.

In a teleop-warehouse setup, by contrast, sensors are spread across a leader console, a follower robot, an environment camera array, and often a network round-trip between them. Aligning these streams is a substantial systems problem and rarely fully solved. Aligned multi-stream data is producible at scale, but the marginal cost of alignment is high and the resulting datasets often ship with alignment caveats that downstream teams have to work around.

This is not a philosophical difference. It is a straightforward mechanical fact that shows up in the timestamps.

Every capture unit, glove, headband, suit, straps, runs on a shared clock domain synchronized at hardware level. Sensor packets carry hardware timestamps that predate any wireless or software-side latency. A capture-time monitor watches for alignment drift and flags sessions that exceed threshold before they enter the labeling pipeline. Delivered episodes ship with alignment metadata so that downstream consumers can verify the property themselves rather than trusting the label.

This is one of the parts of the product that customers do not usually notice until they have tried to work with datasets that lack it. Once a training run has had its multimodal fusion silently degraded by misaligned data, the value of alignment becomes obvious and non-negotiable. Most of our long-term dataset customers came to us the second time for this reason, not the first.

Aligned data enables the cross-modal fusion techniques that make multimodal training worth the name. Cross-attention across image and force tokens learns physically meaningful relationships because the tokens are actually of the same event. Auxiliary losses that predict one modality from another produce meaningful gradients rather than noise. Failure analysis at inference time can attribute a policy's mistake to a specific modality's misreading, which is diagnostically decisive for the next iteration.

None of these downstream capabilities are exotic. All of them assume alignment. Any capture rig that does not deliver alignment as a property of the raw data forces the downstream team to spend engineering effort recovering it, and recovery is always partial.

Alignment is a boring, unfashionable, and enormously consequential engineering property. It is the kind of thing that does not appear in the abstract of a paper and shows up decisively in the reproducibility of a training run. We optimized for it early because the alternative, chasing model-side workarounds for upstream data problems, is a treadmill the customer, not the supplier, ends up running.

For any team choosing a data supplier this year, the question worth asking is not whether the supplier's rig is multimodal. It is whether the modalities are aligned to a spec the team can actually train on, and whether the alignment is documented, monitored, and enforced at capture time. That question separates dataset vendors from data suppliers, and it is a good filter.