Most 'Multimodal' Robot Models Are Still Vision With Extras

Read the abstracts of the last twelve months of high-profile robot policy papers and 'multimodal' is nearly universal. Read the input specifications and the picture is different. Vision, usually two or three camera streams, carries almost all of the observation-side capacity. Force, when present, is a small side channel. Tactile, when present, is a research demonstration on a narrow task set. The word multimodal is doing more work in the marketing than in the model.

In a representative recent VLA, the observation input is dominated by image tokens from two or three camera views, plus a modest proprioception vector, plus a language instruction embedding. Force, when included at all, appears as a low-dimensional wrench with sampling and precision far below the visual stream's effective bandwidth. Tactile, when included, is either absent from the pretraining and added in fine-tuning, or present only in a subset of trajectories that the pretraining loss can barely attend to.

The result is a model whose gradients are dominated, throughout most of training, by visual reconstruction and vision-conditioned action prediction. Adding a force channel to the input does not change what the model learned to depend on. It changes what the model has available in principle. Those are different things.

The imbalance is a straightforward reflection of the data. Vision is abundant across every capture rig and every dataset the field aggregates. Force is scarce and unevenly calibrated. A pretraining loss faithfully reflects the input distribution: modalities that appear in most examples are trained on; modalities that appear in a few examples are effectively ignored.

Architectural details compound this. Loss weighting across heterogeneous inputs is not a solved problem, and when force is a rare and low-dimensional channel, standard normalization schemes push the effective loss weight toward zero. Even a well-intentioned team can end up with a nominally multimodal model that is functionally vision-only, because the gradient budget went where the data was.

Two symptoms are consistent across the space. First, policies degrade sharply when vision is compromised, poor lighting, unusual backgrounds, partial occlusion, even in scenarios where a human would rely on touch and finish the task without vision. A genuinely multimodal policy would tolerate visual degradation because the other modalities would compensate. A vision-dominated policy cannot.

Second, policies handle contact-critical tasks poorly relative to reach-and-place tasks with matched visual difficulty. This is the same pattern seen in the slip and compliance discussions, and it has the same underlying cause: the modality that carries the decisive information for those tasks is present in the model as a garnish rather than as a peer of the visual stream.

It would require training data in which force and vision are both present, both calibrated, both aligned, and both at the sampling rates the underlying phenomena demand. It would require training regimes in which the loss weighting reflects the operational importance of each modality rather than its cheapness. It would require evaluation benchmarks that reward performance under modality degradation rather than only under nominal conditions, so that a policy that leans harder on force at deployment can be identified as better.

None of these are exotic. They are choices any serious team can make. The reason they have not been made at scale is that the training data does not yet exist in the volume that would make the resulting models worth publishing. That is the data-supply problem the last five pieces have been circling.

For customers evaluating robot policies today, the practical implication is to ask sharp questions about the input mix and the training-data composition, not just about the model. A model that lists force as an input but was trained on a corpus that is one percent force by mass is not a multimodal model in any operational sense. It is a vision model that will accept a force tensor.

For teams building the next generation of foundation models, the implication is to invest in the data supply as aggressively as in the model architecture. The gains left on the table by imbalanced training are large and getting larger as the model side matures. The next piece describes what aligned multi-stream capture from wearables enables, and why it is where the balanced-multimodal era actually starts.