🔘 About this critique
Can separately trained models arrive at compatible representations of the same scene? This preprint investigates how the choice of training views affects that agreement and why predicting the next token may be an inefficient route to alignment with vision.
This editorial assessment concerns arXiv version 1, submitted on 29 October 2026. It examines the theoretical assumptions, experimental comparisons and released numerical tables. Selected averages, ratios and percentage-point differences have been recalculated from those tables. These checks do not reproduce model training, certify the mathematical proofs or independently validate the underlying embeddings. The work is assessed as a preprint, not as a peer-reviewed journal article. Exposure describes the consequences of overstating a specific inference, rather than an overall score for the study.
The recommendation is to read the paper critically, with particular attention to what its comparisons with language models actually measure.
In the experiments, a text encoder is trained to bring different descriptions of the same image into agreement in representation space. These descriptions provide the “good views” used throughout the paper. The study compares this Joint-Embedding Predictive Architecture (JEPA) with predicting successive tokens and tests whether the resulting text representations align with those of separately trained vision encoders.
Critique
| Examined version | arXiv:2610.39302v1 · 29 October 2026 · DOI 10.48550/arXiv.2610.39302 · preprint · CC BY 4.0. |
| Materials examined | Complete paper, including Appendices A–C and Tables 1–18; the authors’ public numerical tables and reproduction documentation. The repository was consulted at commit a2b9949fa907d4fbe562742d4b1ffa7205ab7427 on 11 November 2026. |
| Focus | Representation alignment, comparisons with autoregressive language models, training-data provenance, sample efficiency and the scope of applications to world models. |
Unit 1 — The theorem establishes sufficient conditions within a specified model
Article data. Theorem 1 assumes a finite world, injective observations, a common distribution of positive pairs, population optima and minimisers unique up to an orthogonal transformation. Proposition 1 proves the uniqueness requirement for a whitened, second-order objective under a spectral gap. Section 2.1 leaves uniqueness for LeJEPA’s full-distribution regularisation as an assumption. The experiments use an identity predictor; the analysis explicitly excludes learned predictors conditioned on actions. Section 7 acknowledges that no experiment satisfies all the theorem’s assumptions exactly.
Reading. The result explains when independently trained modalities solve equivalent representation problems. It provides a useful sufficient condition, rather than an unconditional account of how arbitrary models understand reality. Real observations can omit information, and practical optimisation need not reach the population optimum. These are recognised boundaries of the result. In the context of world models, the paper contributes to representation learning; it does not establish a theorem for action prediction, planning or autonomous interaction.
Critical conclusion. The mathematical contribution should be read with its assumptions attached. The experiments support the proposed training principle beyond the idealised setting, but empirical agreement is not a verification that every practical JEPA inherits the theorem’s guarantee.
| Claim type | Theoretical scope and application to world models |
| Exposure | High. A conditional representation theorem could otherwise be interpreted as a general proof of world understanding or reliable planning. |
Unit 2 — The advantage over language models concerns particular representation tasks
Article data. On WIT-1024 with DINOv2 ViT-B, the 109-million-parameter text JEPA obtains a mutual 10-nearest-neighbour score of 0.184, compared with 0.111 for the authors’ next-token model using the same encoder size and corpus, 0.154 for Qwen2.5-72B and 0.151 for Llama-3-70B. The protocol takes the maximum over layer pairs. Tables 16–18 extend the comparison to another alignment measure, COCO and several vision encoders. The pretrained language-model comparisons concern the named base models, rather than all available LLMs.
| Text model | WIT mutual 10-NN | Comparison |
|---|---|---|
| Good-view JEPA, 109M | 0.184 | Three runs at the retained regularisation setting |
| Next-token model, 109M | 0.111 | Same encoder size, corpus and 900M samples |
| Bad-view JEPA, 109M | 0.088 | Single run reported in version 1 |
| Qwen2.5-72B | 0.154 | Pretrained base language model |
| Llama-3-70B | 0.151 | Pretrained base language model |
| CLIP B/32 text encoder | 0.124 | Reference encoder trained with image–text pairs |
Selected values from Table 3. All scores in this column measure neighbourhood agreement with DINOv2 ViT-B on 1,024 WIT pairs, using the maximum over layer pairs; higher is better. They are not percentages of correctly answered questions. The pretrained models do not have matched training histories. M and B denote millions and billions of parameters.
Reading. Agreement between neighbourhoods in embedding spaces measures a specific geometric relationship. It is not an accuracy score for reasoning, factual knowledge or language generation. The matched-corpus experiments are particularly informative about the training recipe, although objectives, views, attention and pooling differ, and hyperparameters were tuned only for the good-view JEPA. Comparisons with pretrained LLMs additionally differ in data and training history. Proposition 4 rewrites next-token prediction in a generalised JEPA form, but the authors explicitly describe the subsequent nuisance argument as heuristic, not a corollary of their spectral theorem.
Critical conclusion. The evidence supports an efficient route to representations aligned with vision and useful for retrieval. It does not establish that a 109M encoder replaces a 72B language model across its capabilities, or that next-token prediction is generally ineffective. Model size alone is not a measure of matched capability or total system cost.
| Claim type | Benchmark interpretation and comparator scope |
| Exposure | High. A strong alignment result could be converted into a claim of general superiority over LLMs. |
Unit 3 — The 42-fold comparison has specific denominators and checkpoints
Article data. Section 4.2 compares 21.6 million training samples for a good-view JEPA with 900 million for next-token prediction, yielding WIT scores of 0.109 and 0.111. Their ratio is approximately 41.7. A good-view sample reads three texts, so the released counts imply about 13.9 times fewer text tokens, consistent with the paper’s rounded factor of 14. The runs use different batch sizes and learning rates. The authors state that no earlier next-token checkpoint was scored for this comparison.
Reading. The arithmetic supports the reported comparison between these runs. It does not identify the earliest point at which next-token prediction could reach the target, nor a minimum compute budget for either objective. Samples, tokens, unique source images, floating-point operations and elapsed training time are different quantities. The cost of producing recaptions and training the models that supply them is also outside this sample ratio.
Critical conclusion. The result is compelling evidence of sample efficiency in the reported alignment setting. It should not be presented as a demonstrated 42-fold reduction in compute, energy, money or time, or as a universal efficiency constant for JEPAs.
| Claim type | Efficiency measurement and denominator choice |
| Exposure | High. Substituting compute or deployment cost for training samples would materially change the result. |
Unit 4 — Separate encoder training still uses information that links modalities
Article data. Good views require knowing which descriptions refer to the same scene. At scale, each Recap-DataComp image supplies an alt-text and two recaptions generated by LLaVA, built on LLaMA-3-8B with a CLIP vision encoder. Table 7 reports WIT alignment of 0.107 with those three texts, compared with 0.060 for an alt-text paired with a text-only paraphrase using the texts of 2.24 million images. The matched-objective controls receive the same texts. The authors also identify at least 43 overlapping WIT captions and report changes no larger than 0.002 when those pairs are removed for the nine models checked.
Reading. The text encoder itself does not receive pixels, and no cross-modal loss jointly trains the two encoders. Nevertheless, grouping by image and recaptions transmit information derived from image–text relationships. This qualifies a claim of complete independence from multimodal supervision, while leaving the same-text comparison between objectives informative. The reported overlap analysis reduces a particular contamination concern; it does not demonstrate that every source of overlap or inherited knowledge has been excluded.
Critical conclusion. The scalable result combines a representation objective with a structured source of training views. Both deserve credit. Its comparison with ordinary language pretraining should acknowledge the provenance and construction of those views rather than attributing the entire advantage to the loss function alone.
| Claim type | Training-data provenance and attribution of gains |
| Exposure | Medium. Indirect multimodal information could disappear from an account that describes the encoder as learning from text alone. |
Unit 5 — Seed variation and benchmark-informed choices qualify the headline scores
Article data. The three largest good-view runs score 0.191, 0.181 and 0.179 on WIT, reproducing the reported mean of 0.184. Another run with twice the effective SIGReg weight scores 0.121. Appendix C states that the first run’s weight was retained after observing this result on WIT. Including all four gives approximately 0.168, still above the named LLMs with DINOv2 ViT-B. In the pair-free experiment, ImageNet-100 top-5 accuracy is 64.4% for the JEPA and 47.6% for the rerun Qwen3-8B: a difference of 16.8 percentage points. The JEPA’s 15 runs comprise three training seeds and five alignment-method seeds per trained model.
Reading. The advantage is not confined to one successful run, but WIT also informed a training choice and is therefore not a wholly untouched final evaluation. The additional benchmarks and controls are valuable checks. For pair-free alignment, repeated alignments of one trained encoder do not constitute additional independent training replications. The appendix acknowledges that its significance tests treat the 15 runs as independent and separately reports complete separation across the three training seeds for the two stronger classification comparisons.
Critical conclusion. The reported gains warrant attention, with the selection history and nested replication made explicit. The 16.8-point difference is a descriptive result for ImageNet-100 top-5 under that protocol; it should not be generalised to overall language-model accuracy. Further evaluation should reserve independent model-selection and test data and account for the two sources of run variation.
| Claim type | Selection effects and statistical unit of replication |
| Exposure | Medium. Benchmark-informed choices or nested runs could make a performance advantage appear more independent or precisely established than it is. |
Unit 6 — The captioning demonstration combines JEPAs with an LLM and paired alignment
Article data. The strongest reported COCO captioner reaches CIDEr 0.838 using frozen DINOv2 and DINOv3 image encoders, a text JEPA, a Qwen2.5-1.5B decoder fine-tuned on text embeddings, and consensus over 32 sampled captions plus the greedy candidate. Its linear map is fitted on approximately 30,000 Flickr image–caption pairs. The separate zero-pair Flickr captioner reaches CIDEr 0.158 ± 0.042, compared with approximately 0.430 using the paired map. Pair-free classification in Section 5.2 is a different evaluation.
Reading. These are distinct demonstrations of how learned representations can be connected. The stronger captioner provides evidence for combining a JEPA representation with language generation. Its result cannot be assigned to the 109M encoder alone or described as requiring no paired information anywhere in the system. The paper carefully separates training a neural network on paired examples from fitting a linear map on those pairs; that distinction must survive in the summary.
Critical conclusion. The practical contribution is a modular route from separately trained encoders to useful captioning, including a more demanding experiment without paired alignment data. It supports complementarity between JEPA representations and LLM decoders, with different performance and supervision requirements for the paired and pair-free variants.
| Claim type | System composition and supervision requirements |
| Exposure | High. Combining results from different variants would exaggerate what the small encoder or the zero-pair system achieves. |
Unit 7 — The scaling law is useful within the regime in which it was tested
Article data. The authors fit an empirical law to 45 grid cells and report predictions made before training 85 additional runs. Its reported RMSE is 0.0093, compared with a seed-to-seed difference of 0.0096. At 320 million samples, beyond the fitted range, the model predicts retrieval scores of 0.455 and 0.389 for two good-view settings whose observed scores are 0.418 and 0.348. Appendix C states that the fitted exponents are poorly identified and that the law has no term for repeated data.
Reading. Prospective prediction of new runs is stronger evidence than fit alone. The documented extrapolation failures also establish a practical boundary. This law summarises one model family and training recipe; it is not derived from the representation theorem and does not establish how alignment will scale indefinitely or across all modalities.
Critical conclusion. The law is a useful experimental guide within its evaluated regime. Its coefficients and favourable prediction error should be reported alongside the domain of validation and the observed failures beyond it.
| Claim type | Predictive validation and extrapolation |
| Exposure | Medium. A locally successful empirical relationship could be mistaken for a universal law of representation learning. |
Overall assessment
The paper offers a substantial contribution to understanding how training views shape representations. Its conditional theorem, matched-corpus experiments, released numerical records and tests across several vision encoders give a coherent basis for preferring views that preserve shared scene information when alignment is the objective. The most persuasive comparison with traditional LLMs is therefore specific: much smaller text JEPAs can achieve stronger agreement with visual representations under the evaluated protocols. That result does not establish general linguistic or reasoning superiority, nor does it remove the contribution of recaptioning, model selection or paired information elsewhere in the system. The captioning experiments demonstrate a productive combination of JEPA representations and an LLM decoder. For world-model research, the work supplies a useful principle for learning compatible representations while leaving action-conditioned prediction, planning and broader operational capabilities to be established separately.
Recommended for readers studying representation learning, multimodal alignment, world-model architectures and the design of fair comparisons between predictive objectives.
🔘 Paper page: https://arxiv.org/abs/2610.39302v1
Abstract
Whether models trained on different modalities converge to the same representation of the world is central to AI. The Platonic Representation Hypothesis (PRH) conjectures that they do as they scale, but without a proof it offers neither a guarantee nor a recipe. We turn the hypothesis into a theorem. First, we prove a Platonic Representation Theorem: Joint-Embedding Predictive Architectures (JEPAs) trained separately on any number of modalities learn the same representation up to one rotation whenever their positive pairs are two observations of one scene–good views–and their misalignment is bounded otherwise. Second, we show that next-token prediction is a JEPA with bad views, parts of a single document, which explains why language models align with vision only through scale. Third, we validate the theory from 145k captions to the texts of 940M web images, and uncover a scaling law of alignment that predicts unseen runs within seed noise. Good views need 42 times fewer training samples than next-token prediction on the same corpus to reach the same alignment, and a 109M-parameter text JEPA aligns with DINOv2 and CLIP’s image encoders better than every language model we evaluated, up to 72B parameters, and than CLIP’s text encoder: next-token prediction is a remarkably inefficient route to platonic representations. As a showcase, one rotation turns separately trained encoders into an image captioner (CIDEr .838 on COCO). Even that rotation needs no pair: on the latest pair-free benchmark, our text JEPA beats every zero-shot accuracy reported with language models up to 8B parameters, e.g., 64% against at most 56% top-5 on ImageNet-100. Positive views, and not scale alone, are thus the lever to build platonic representations.
Abstract by Randall Balestriero, Yann LeCun, reproduced from arXiv:2610.39302v1 under the Creative Commons Attribution 4.0 International licence. Wording is retained; formatting has been standardised. The editorial assessment above qualifies the scope of the claims.
Authors
Randall Balestriero, Yann LeCun
Liked this post? Follow this blog to get more.