Dynamic Reflections: Probing Video Representations with Text Alignment
DeepMind found that video-text alignment scores, thought to be weak, actually improve dramatically when models are given more video frames and more captions at test time, without retraining.
It reframes a widely-cited 'weak alignment' finding as an artifact of test-time data scarcity rather than a fundamental limit, and offers a validated scaling law plus a new zero-shot way to probe video encoder quality without expensive downstream training.
Tyler Zhu · Tengda Han · Leonidas Guibas · Viorica Pătrăucean · Maks Ovsjanikov — meet the researchers →
Two readings, equal authority
How to choose: The paper’s words is verbatim — use it when you need to quote, or to judge how they write. Plain language is a paraphrase written for comprehension — use it when you want the idea fast. Neither is a summary of the other; they are two doors into the same room.
“The alignment of representations from different modalities has recently been shown to provide insights on the structural similarities and downstream capabilities of different encoders across diverse data types. While significant progress has been made in aligning images with text, the temporal nature of video data remains largely unexplored in this context. In this work, we conduct the first comprehensive study of video-text representation alignment, probing the capabilities of modern video and language encoders. Our findings reveal several key insights. First, we demonstrate that cross-modal alignment highly depends on the richness of both visual (static images vs. multi-frame videos) and text (single caption vs. a collection) data provided at test time, especially when using state-of-the-art video encoders. We propose parametric test-time scaling laws that capture this behavior and show remarkable predictive power against empirical observations. Secondly, we investigate the correlation between semantic alignment and performance on both semantic and non-semantic downstream tasks, providing initial evidence that strong alignment against text encoders may be linked to general-purpose video representation and understanding. Finally, we correlate temporal reasoning with cross-modal alignment providing a challenging test-bed for vision and language models. Overall, our work introduces video-text alignment as an informative zero-shot way to probe the representation power of different encoders for spatio-temporal data.”
DeepMind and Princeton researchers ran the first large-scale test of whether video and text representations line up with each other, the way image-text representations already do. They found that video-text alignment had looked weak mainly because past tests fed models too little data at once — giving a model more video frames and more captions per video pushed alignment scores up dramatically, and they fit a mathematical curve that predicts this improvement well. They also found that models with stronger video-text alignment tend to do better on separate downstream video tasks, and that video and text models represent time differently even when they eventually agree on which caption matches a video.
What this paper defines
Every definition below is the paper’s own sentence, with its locator. The plain gloss is a reading aid and is marked as one.
Platonic Representation Hypothesis (PRH)
“The PRH posits that as neural networks are scaled in terms of capacity, data diversity, and task variety, their learned internal representations converge toward a shared, universal statistical model of reality.”§2 Related Work
In plain terms: The idea that as AI models get bigger and are trained on more varied data, their internal representations of the world start to converge on the same underlying structure, regardless of what type of data they were trained on.
Mutual k-NN (MkNN) alignment metric
“The Mutual k-NN (MkNN) metric introduced in Huh et al. (2024) measures the agreement between the nearest neighbor structure in two different embedding spaces.”§3 Our Approach, Alignment metric
In plain terms: A score for how much two different models (e.g. a video model and a text model) agree on which examples are most similar to each other.
alignment probing
“Cross-modal alignment has also been assessed via “alignment probing,” which correlates emergent alignment potential with the representation’s clustering quality (k-NN performance) over linear separability (Zhang et al., 2025).”§2 Related Work
In plain terms: A way of checking cross-modal alignment by looking at how well a representation's neighbors cluster together, rather than whether a simple straight-line boundary can separate its categories.
test-time scaling law (saturation model)
“Here nf and nc are the number of video frames and text captions given to a specific pair of vision and text encoders (as in Fig. 1), S∞ represents the theoretical saturation score corresponding to ideal alignment of this vision/text model pair, whereas Cf , Cc , α and β are fitted scalar parameters.”§6 Video-Text Alignment and Data Dependence, Test-time Scaling Laws
In plain terms: A formula predicting how much a model pair's alignment score rises as you give it more frames and captions, leveling off at some maximum score particular to that pair of models.
Saturation Score (S∞)
“S∞ represents the theoretical maximum alignment score achievable by the specific pair of vision and text encoders, assuming access to infinite test-time information (nf → ∞, nc → ∞).”Appendix B.2
In plain terms: The best alignment score a given video-model/text-model pair could ever reach if you fed it an unlimited number of frames and captions.
temporal reorder negative (VideoComp)
“We use the “temporal reorder” examples which compare a caption of a video against a negative caption describing the same events in a different order.”§8 Temporal Analysis & Cross-Model Alignment, VideoComp
In plain terms: A test caption that describes the exact same events in a video but puts them in the wrong chronological order, used to check whether a model actually notices event order.
What they actually did
Each step is a synthesis. Open any step to see the paper’s own sentence it was derived from, with its locator — so nothing here floats free of the source.
- Adapt an existing image-text alignment metric (mutual k-NN) so it can be measured on multi-frame video and multi-caption text instead of just single images and single captions.
Trace this step to the paper
“At a high-level, our approach follows the methodology introduced in Huh et al. (2024), which uses a mutual k-NN metric to measure the similarity across different modalities. We adapt this approach to our setting, extending it to measure the multi-frame (or multi-clip) and multi-caption similarity of video-text pairs.”§3 Our Approach
- Build a test set of paired videos and their associated caption sets, to be embedded separately by video and text encoders.
Trace this step to the paper
“We assume a test set of N video-caption pairs: S = (V, C) = {(v1 , c1 ), . . . , (vN , cN )}.”§3 Our Approach, Dataset
- Compute the Mutual k-NN alignment score between the two embedding spaces, choosing whichever pair of intermediate layers (one from the vision encoder, one from the text encoder) gives the highest alignment.
Trace this step to the paper
“we also follow previous work and optimize over the choice of intermediate layers for both encoders, and pick the pair of layers that maximizes the alignment score.”§3 Our Approach, Alignment metric
- Vary the amount of visual information given at test time by sampling an increasing number of frames per video, from a single frame up to dozens of frames.
Trace this step to the paper
“For a video vi and a given video encoder Evid , which natively processes clips with no frames, we extract the indices of nf frames through uniform linear interpolation to study the effect of using an increasing number of frames nf at test-time.”§3 Our Approach, Incorporating multi-instance data
- Vary the amount of textual information given at test time by concatenating an increasing number of a video's captions into a single string before encoding.
Trace this step to the paper
“we can use anywhere from one caption to the full set of captions. We concatenate the set of selected captions into a single string and use text-based encoders (including LLM-based ones as discussed below) to extract their intermediate features.”§3 Our Approach, Incorporating multi-instance data
- Evaluate 85 vision models/variants and 30 language models/variants (121 models total) across two video-caption datasets, VATEX and the Perception Encoder Video Datasets (PVD).
Trace this step to the paper
“We introduce a robust evaluation framework to probe and compare the capabilities of 121 modern video and language models.”§1 Introduction
- Fit a parametric saturation-based scaling law relating alignment score to the number of frames and captions, for each vision/text encoder pair.
Trace this step to the paper
“We tested several formulations, but found that a saturation-based model (Eq. 2) provided the best fit by a significant margin”§6, Test-time Scaling Laws
- Correlate each self-supervised video model's text-alignment score against its own measured accuracy on separate downstream semantic and non-semantic video tasks (action classification, point/box tracking, camera pose, depth estimation).
Trace this step to the paper
“we consider the best performing video models reported in Carreira et al. (2024) and correlate their vision-text feature alignment against Gemma 2-9b-it on the VATEX dataset to their accuracy on video analysis tasks”§7 Cross-Modal Alignment and Downstream Performance
- Test temporal sensitivity using the synthetic Test of Time dataset, comparing how each modality ranks logically related caption variants as nearest neighbors.
Trace this step to the paper
“We use video-text alignment on this dataset to understand how sensitive text and vision models are to this temporal reordering.”§8 Temporal Analysis & Cross-Model Alignment, Test of Time
- Test robustness to temporal shuffling using VideoComp's 'temporal reorder' negative captions, comparing alignment against the true positive captions versus against reordered negatives describing the same events out of sequence.
Trace this step to the paper
“To test the sensitivity to such negatives, we first compute the standard alignment using the text embeddings of positive captions. Then, we recompute neighbors in the text space, by considering the negative of each caption”§8 Temporal Analysis & Cross-Model Alignment, VideoComp
- Synthesize additional diverse captions for the PVD dataset (which lacks multiple natural captions per video) by prompting an LLM to split one detailed caption into ten shorter, non-redundant captions.
Trace this step to the paper
“we synthesized multiple captions for videos in the PVD dataset using LLMs, starting from the detailed caption provided in the original dataset. Importantly, we encouraged the model to only use details present in the original caption, and not hallucinate new possible descriptions. We relied on Gemini 2.5 Pro.”Appendix A.3
Exactly what was run, and how
| Model | Developer | Temp | Effort / reasoning | Deployment | Other settings |
|---|---|---|---|---|---|
| VideoMAEv2 | Wang et al. (open-source; evaluated by Google DeepMind/Princeton) | not reported | not reported | local weights | Used purely as a frozen self-supervised video feature extractor (no text supervision during its own training); intermediate-layer features are used for alignment, not for text generation. |
| DINOv2 | Oquab et al. (Meta; open-source, evaluated here) | not reported | not reported | local weights | Used as a frozen self-supervised image encoder, applied both to single frames and averaged across frames for the 'image model on video' setting. |
| Gemma 2 (9B-it and other sizes) | Google DeepMind | not reported | not reported | local weights | Used as a frozen text encoder: captions are encoded and the representation is averaged over the token sequence, not used for text generation. |
| Gemini 2.5 Pro | Google DeepMind | not reported | not reported | unstated | Used generatively, not as an encoder: prompted to split one detailed PVD caption into ten shorter, non-redundant captions, restricted to details already present in the source caption. |
Source for VideoMAEv2 settings
“For example, VideoMAEv2 has no exposure to textual supervision, yet it learns powerful spatiotemporal features that achieve top performance on downstream action understanding tasks (Wang et al., 2023).”§2 Related Work
Source for DINOv2 settings
“The vision encoders used to validate these claims are typically state-of-the-art self-supervised models like DINOv2 (Oquab et al., 2023).”§2 Related Work
Source for Gemma 2 (9B-it and other sizes) settings
“We also consider recent unimodal language models from the Gemma 2 series (Team et al., 2024b) and evaluate their potential as text encoders.”§4 Data and Models
Source for Gemini 2.5 Pro settings
“We relied on Gemini 2.5 Pro.”Appendix A.3
What they reported — and what they left out
The paper studies 121 encoder models (85 vision, 30 language) primarily as frozen feature extractors rather than generative systems: it reports each model's architecture family, size variants, and training paradigm (self-supervised vs. text-aligned), and states in the Reproducibility Statement that all models are hosted on HuggingFace or open-sourced by their authors, but it does not report decoding temperature, sampling settings, or system prompts for any encoder, nor generation-hyperparameter details for Gemini 2.5 Pro, the one generative LLM used (to synthesize PVD captions).
The numbers they report
Multi-frame, multi-caption test-time data can raise video-text alignment scores far above previously reported single-image, single-caption scores.
alignment approaching 0.4 in some cases
See it in the paper
“the alignment score can be improved significantly, achieving close to 0.4 in some cases, without modifying the underlying trained models.”§1 Introduction
Among pure image-and-text models studied in prior work, the best non-Gemma text encoder paired with the best image model reproduces a previously reported, comparatively low alignment score.
0.18 (DINOv2 + best non-Gemma text encoder)
See it in the paper
“the alignment score between the best image model DINOv2 and the best text encoder outside of the Gemma family is 0.18”§5 Video-Text Alignment Results
Simply switching to a stronger (Gemma-2 family) text encoder raises the best image-text alignment score.
~0.206
See it in the paper
“Simply using a powerful text encoder already increases the best image-text alignment score to approximately 0.206.”§5 Video-Text Alignment Results
Basic temporal averaging of frame features with strong image models reaches a notably higher alignment than single-frame image-text alignment.
~0.223
See it in the paper
“basic temporal averaging across multiple frames with powerful image models exhibits remarkably high video-text alignment reaching alignment of approximately 0.223.”§5 Video-Text Alignment Results
A native, self-supervised video model achieves the single highest text-alignment score among all vision encoders tested, beating the strongest image encoder.
See it in the paper
“the highest alignment is achieved with a self-supervised VideoMAEv2 model.”§5 Video-Text Alignment Results
The saturation-based scaling law fits the observed alignment-vs-data-quantity relationship extremely well for both a video and an image encoder.
R2 = 0.9791 (VideoMAEv2); R2 = 0.9964 (DINOv2)
See it in the paper
“This formulation achieves remarkably high coefficients of determination for both VideoMAEv2 (R2 = 0.9791) and DINOv2 (R2 = 0.9964)”§6, Test-time Scaling Laws
The fitted scaling-law parameters show VideoMAEv2 relies on additional frame count far more heavily than DINOv2 does, while both rely on caption count similarly.
Cf = 0.15 (VideoMAEv2) vs. Cf = 0.05 (DINOv2), nearly 3x
See it in the paper
“the frame coefficient (Cf ) for VideoMAEv2 is nearly triple that of DINOv2, while the caption coefficients (Cc ) remain comparable, highlighting the video model’s stronger ability to leverage temporal information from additional frames to improve alignment.”§6, Test-time Scaling Laws
Increasing the number of captions per video from one to ten substantially improves alignment on average.
60% average improvement, based on a linear fit
See it in the paper
“going from 1 to 10 captions improves alignment by 60% on average.”§6, Test-time Scaling Laws
Cross-modal (video-text) alignment correlates positively with downstream performance on both semantic tasks and several non-semantic perception tasks, for self-supervised video models.
See it in the paper
“there is a strong positive correlation between the cross-modal alignment scores and semantic tasks performance such as action classification on SSv2 and Kinetics. Interestingly, there is also a significant correlation between the alignment score and the accuracy on non-semantic perception tasks such as camera pose estimation, depth prediction, and object tracking.”§7 Cross-Modal Alignment and Downstream Performance
One downstream task, point tracking, is a clear exception where alignment barely predicts performance.
See it in the paper
“The point tracking task represents a notable exception, with weak correlation between text alignment and downstream performance.”§7 Cross-Modal Alignment and Downstream Performance
On a synthetic temporal-ordering test, once given enough neighbors, nearly all models achieve near-perfect video-text alignment, but they disagree sharply on ranking among the top few neighbors.
See it in the paper
“We find that when we take k = 3 neighbors, many models get near perfect alignment as expected, since each example has 3 distinctly closer neighbors”§8 Temporal Analysis & Cross-Model Alignment, Test of Time
Language models tend to treat two captions with the same words in a different order as nearest neighbors, behaving more like a bag-of-words matcher than a temporally-sensitive reader, at least at the shallow layers examined.
See it in the paper
“LLMs measure closeness more akin to a bag-of-words than being temporally sensitive, at least at the shallower layers from where we are extracting the features.”§8 Temporal Analysis & Cross-Model Alignment, Test of Time
On the harder, real-world VideoComp temporal-reorder test, alignment against negative (temporally-shuffled) captions is only slightly lower than against true positive captions, but models with the strongest original alignment lose the most against negatives.
See it in the paper
“the alignment is lower with the negative captions than the positive captions, but not significantly. Notably, models with larger alignment suffer more of a drop, suggesting that these models may be learning temporally-aware structures which are somewhat perturbed by the negative.”§8 Temporal Analysis & Cross-Model Alignment, VideoComp
Using all ten VATEX captions instead of one produces a consistent, well-fit linear improvement in alignment across models.
best-fit line y = 1.6x - 0.018, r = 0.998
See it in the paper
“The line of best fit is y = 1.6x − 0.018 with r = 0.998, indicating that multi-caption data consistently improves alignment.”Appendix A.1
On the PVD dataset, synthesizing even a handful of additional captions already beats using the single original detailed caption, with returns leveling off after about six synthesized captions.
improvement from 3 captions onward; saturates around 6 captions
See it in the paper
“using even just 3 captions leads to better alignment compared to the baseline score. This saturates around 6 captions for most models, pointing to potential redundancy between the synthesized captions beyond this number.”Appendix A.3
What they assert, beside what they showed
Left is the claim in the paper’s own words. Right is the data offered for it. Where the two do not fully meet, a gold band names the distance.
The weak video/image-text alignment scores reported in prior work were largely an artifact of the limited data (a single frame, a single caption) given at test time, not a fundamental ceiling on alignment.
“we demonstrate that by considering multiple video frames instead of a single image, as well as a diverse set of captions instead of a single annotation, the alignment score can be improved significantly, achieving close to 0.4 in some cases, without modifying the underlying trained models.”
“our best results significantly exceed those reported in Huh et al. (2024), highlighting that the paucity of annotations in both visual space (images vs. videos) and text space (single caption vs. multiple descriptions) can help to explain the limited alignment observed in prior work.”
Figure 12 caption, Appendix A.1Native video encoders like VideoMAEv2 are intrinsically better than static image encoders like DINOv2 at exploiting additional temporal (frame) information.
“We demonstrate that alignment scores dramatically improve – doubling in some cases – simply by utilizing multiple video frames and diverse caption sets at inference, without any retraining. We quantify this phenomenon with a novel and accurate (R2 > 0.98) saturation-based scaling law, which quantitatively confirms that native video models (like VideoMAEv2) are intrinsically better at using temporal information than static encoders (like DINOv2).”
“the frame coefficient (Cf ) for VideoMAEv2 is nearly triple that of DINOv2, while the caption coefficients (Cc ) remain comparable, highlighting the video model’s stronger ability to leverage temporal information from additional frames to improve alignment.”
§6, Test-time Scaling LawsVideo-text alignment can serve as an informative zero-shot proxy metric for a video encoder's downstream representation quality, as an alternative to expensive supervised probing.
“Overall, these results suggest that video-text alignment could potentially be used as a powerful zeroshot metric for probing the quality of video representations as an alternative or complementary to more expensive evaluation techniques that require repeatedly training multiple cross-modal decoders during self-supervised video model development.”
“there is a strong positive correlation between the cross-modal alignment scores and semantic tasks performance such as action classification on SSv2 and Kinetics.”
§7 Cross-Modal Alignment and Downstream PerformanceVideo and text models eventually 'agree' on which captions match a video but encode temporal order information differently along the way.
“Video and text eventually are aligned, but encode temporal information differently.”
“We find that the language models tend to rank the neighbor which has the same words but ordered differently first... Thus, in this case, LLMs measure closeness more akin to a bag-of-words than being temporally sensitive, at least at the shallower layers from where we are extracting the features.”
Figure 5 caption / §8 Temporal Analysis & Cross-Model Alignment, Test of TimeVideo models that show larger overall alignment are more likely to have learned temporally-aware representations, evidenced by their bigger drop in alignment against temporally-reordered negative captions.
“models with larger alignment suffer more of a drop, suggesting that these models may be learning temporally-aware structures which are somewhat perturbed by the negative.”
“the alignment is lower with the negative captions than the positive captions, but not significantly.”
§8 Temporal Analysis & Cross-Model Alignment, VideoCompSome video models (e.g., DINOv2) are versatile enough to span multiple distinct video-video alignment clusters, which indicates general-purpose representation quality.
“we hypothesize that such alignment against multiple models provides a strong indicator of the versatility of a given vision model.”
“some models (such as DINOv2) are able to span multiple clusters”
§8 Temporal Analysis & Cross-Model Alignment, Cross-model alignmentHow they frame it, and what they want next
Their framing
The authors frame their central result — that prior 'weak' video/image-text alignment was largely a test-time data-scarcity artifact — as answering an open question explicitly posed in the field's foundational Platonic Representation Hypothesis paper. They present the scaling-law fit and the downstream-task correlations as further validating that alignment is a meaningful, general property of these encoders, while directly naming specific places (point tracking, generative video models, fine-grained temporal ordering) where the story is incomplete or still an open question, rather than glossing over them.
Register: The paper states its central claims plainly and backs them with specific numbers (explicit R² values, correlation coefficients, percentage improvements), but consistently marks the edges of what it has shown: it labels one interpretive claim a 'hypothesis' left for future work, names a specific task (point tracking) where its own story doesn't hold, and explicitly states that generative video models' alignment to text 'is currently quite weak' as an open problem rather than minimizing it.
Where they hedge
“This can potentially be due to the highly local nature of the point tracking task, and also suggests room for improvement in terms of truly general purpose video encoders.”§7 Cross-Modal Alignment and Downstream Performance
“However, there is still room for further improvement in their temporal awareness.”§8 Temporal Analysis & Cross-Model Alignment, VideoComp
“we hypothesize that such alignment against multiple models provides a strong indicator of the versatility of a given vision model. We leave a complete investigation of this phenomenon as interesting future work.”§8 Temporal Analysis & Cross-Model Alignment, Cross-model alignment
“Finally, while generative models are a promising direction for video models, it remains an open question of how we can best harness their latent representations for understanding, as currently their alignment to text is quite weak.”§9 Conclusion, Limitations, and Future Work
“While these assumptions are consistent with prior work, a more careful investigation would shed better light on the exact ideal form”Appendix B.4.2
What they say it means
- Because earlier low alignment scores reflected impoverished test-time data rather than a ceiling on encoder quality, future evaluations of video/image encoders should scale up test-time visual and textual data before concluding an encoder aligns poorly with language.
the paper’s words
“This result highlights, for the first time, that large improvements in alignment can be achieved through efforts at test time, which is complementary to the training-time resources (model size, amount of training data, etc.) considered in prior work.”§1 Introduction
- Because alignment correlates with performance on multiple downstream video tasks, teams developing self-supervised video encoders could use a cheap zero-shot alignment score to guide development instead of repeatedly training expensive supervised probes.
the paper’s words
“vision-text alignment can be used as an informative zero-shot metric to guide video model development.”§9 Conclusion, Limitations, and Future Work
- Predictive test-time scaling laws could inform practical data-collection decisions, such as how many caption annotations are worth collecting per video.
the paper’s words
“We note that such predictive models for test-time scaling can be useful to design strategies for multimodal data acquisition (e.g., when collecting multiple high-quality annotations of video data, which can be costly), as well as to compare the ability of different encoders to incorporate diverse data modalities.”§6, Test-time Scaling Laws
What they call for next
- Future work should determine how to best harness generative video models' latent representations for understanding, since their alignment to text is currently weak.
the paper’s words
“Finally, while generative models are a promising direction for video models, it remains an open question of how we can best harness their latent representations for understanding, as currently their alignment to text is quite weak.”§9 Conclusion, Limitations, and Future Work
- Future work should fully investigate why some video models span multiple video-video alignment clusters while others do not.
the paper’s words
“We leave a complete investigation of this phenomenon as interesting future work.”§8 Temporal Analysis & Cross-Model Alignment, Cross-model alignment
- Future work should build a more careful theoretical model of cross-modal feature interactions to pin down the exact form of the scaling law, beyond the independence assumptions used here.
the paper’s words
“a more careful investigation would shed better light on the exact ideal form (potentially involving a more careful model of the interactions between modalities, as suggested by this model) as interesting future work.”Appendix B.4.2
Limitations they state
“The point tracking task represents a notable exception, with weak correlation between text alignment and downstream performance.”§7 Cross-Modal Alignment and Downstream Performance
“At the same time, our analysis sheds light on some limitations of existing pure video foundation models, showing that many video models are outperformed by image models applied frame-by-frame.”§9 Conclusion, Limitations, and Future Work
“Finally, while generative models are a promising direction for video models, it remains an open question of how we can best harness their latent representations for understanding, as currently their alignment to text is quite weak.”§9 Conclusion, Limitations, and Future Work
“However, there is still room for further improvement in their temporal awareness.”§8 Temporal Analysis & Cross-Model Alignment, VideoComp
“We leave a complete investigation of this phenomenon as interesting future work.”§8 Temporal Analysis & Cross-Model Alignment, Cross-model alignment
Moves worth stealing
Opens by directly quoting a named open question left unresolved in the field's foundational paper, then structures the paper's whole contribution as an answer to it.
“the maximum theoretical value for the alignment metric is 1. Is a score of 0.16 indicative of strong alignment [...] or does it signify poor alignment with major differences left to explain? We leave this as an open question.”
Adds a full theoretical-derivation appendix deriving the paper's own empirically-fit scaling law from two independent first-principles models, rather than treating a good curve fit as its own justification.
“In this document, our goal is to provide a theoretical justification for these scaling laws.”
States a favorable-looking result and immediately flags, in the very next clause, the confound that could partly explain it away, rather than letting a reader discover the confound later.
“A notable exception is the Perception Encoder Core Giant image model applied on all the frames of the video, which obtains the best alignment score, but is also contrastively trained on images, videos, and text.”
States the exact number of models tested up front as a mark of evaluation breadth, then substantiates it with a fully itemized appendix list rather than leaving 'extensive experiments' unquantified.
“We introduce a robust evaluation framework to probe and compare the capabilities of 121 modern video and language models.”
States a weak or null-ish result plainly as a genuine limitation of the field's current models rather than reframing it as a success of the paper's own method.
“as currently their alignment to text is quite weak.”
Where else this leads
Same territory
- From AGI to ASI Google DeepMind
scaling laws
Published alongside it
The nearest publications in time, across all three labs.
- ProEval: Proactive Failure Discovery and Efficient Performance Estimation for Generative AI Evaluation Google DeepMind
2026-04-25 - Where the goblins came from OpenAI
2026-04-29 - Did US Worker Retraining Reduce Participant Automation Exposure? Google DeepMind
2026-05-06 - HeadVis Anthropic
2026-05-15
What this page was built from
Working from a full plain-text extraction of the ICLR 2026 camera-ready PDF (27 pages, including the appendix's theoretical derivations and full model lists), per the corpus manifest's 'full' text grade; multi-panel figures with dense axis labels and per-point annotations (e.g., Figures 4, 5, 6, 11, 12) are represented here via their captions and surrounding prose rather than point-by-point transcription, since the PDF-to-text extraction interleaves those labels out of visual reading order.