How Vision-Language Models Learned to Reason About Space (10 Papers, One Thread)

<blockquote> This is a cross-post. The original (with diagrams, code, and cheat sheets for every chapter) lives on my blog: 👉 <a href="https://direcf.github.io/posts/spatial-vlm-evolution/" rel="noopener noreferrer">The Evolution of Spatial VLMs</a> </blockquote> Vision-language models can describe a warehouse photo in fluent prose — and then fail to answer "how many meters is the forklift from the shelf?" They see but don't perceive. Over 2024–2026 a line of research closed that gap step by step, and the arc is remarkably coherent once you read it through a single lens: The representational mismatch between discrete language tokens and continuous 3D geometry — and the successive attempts to close it. I wrote a 10-chapter course tracing that arc paper by paper. Here's the map, with a one-line "why this paper had to exist" for each. Full chapter (analogies + mermaid diagrams + runnable code) is linked on each. <h3> The 10-paper arc </h3> <ol> <li><a href="https://direcf.github.io/posts/spatial-vlm-evolution/chapter-01.html" rel="noopener noreferrer">BLINK: See but Not Perceive</a> — The diagnosis. Humans 95.7% vs GPT-4V 51.3% on tasks people solve "in the blink of an eye." VLMs read images as a bag of semantic concepts, not geometry.</li> <li><a href="https://direcf.github.io/posts/spatial-vlm-evolution/chapter-02.html" rel="noopener noreferrer">SpatialVLM: Data Is the Answer</a> — Reframe spatial blindness as a data problem, not an architecture one. Lift 2D web images to metric 3D and synthesize 2B spatial VQA pairs.</li> <li><a href="https://direcf.github.io/posts/spatial-vlm-evolution/chapter-03.html" rel="noopener noreferrer">SpatialRGPT — Point at a Region, Plant the Depth</a> — Add region grounding + a gracefully-degrading depth plugin. The first VLM to point at "this object" and reason in metric 3D.</li> <li><a href="https://direcf.github.io/posts/spatial-vlm-evolution/chapter-04.html" rel="noopener noreferrer">SpatialBot — Learning to Read Depth</a> — Make depth queryable: a Depth API the model can call instead of guessing.</li> <li><a href="https://direcf.github.io/posts/spatial-vlm-evolution/chapter-05.html" rel="noopener noreferrer">Thinking in Space</a> — From single images to video: does the model build a cognitive map and remember space?</li> <li><a href="https://direcf.github.io/posts/spatial-vlm-evolution/chapter-06.html" rel="noopener noreferrer">OmniSpatial</a> — Draw the map of the ability itself: a comprehensive taxonomy of spatial reasoning.</li> <li><a href="https://direcf.github.io/posts/spatial-vlm-evolution/chapter-07.html" rel="noopener noreferrer">Perspective-Aware Reasoning (APC)</a> — "If I stood over there…" — mental imagery to break egocentric bias.</li> <li><a href="https://direcf.github.io/posts/spatial-vlm-evolution/chapter-08.html" rel="noopener noreferrer">Depth-native VLMs — DepthVLM vs DepthLM</a> — What if the VLM is the depth sensor? Two takes on baking depth in natively.</li> <li><a href="https://direcf.github.io/posts/spatial-vlm-evolution/chapter-09.html" rel="noopener noreferrer">Verifiable RL for Spatial Reasoning</a> — Spatial skill isn't only more labels; a verifiable reward can activate latent ability. SVQA-R1 → Smooth Operator (50K samples rivaling 4.5M supervised).</li> <li><a href="https://direcf.github.io/posts/spatial-vlm-evolution/chapter-10.html" rel="noopener noreferrer">Grounded 3D-Aware — Reconstruct While You Reason</a> — The synthesis: GR3D promotes grounding into a generation-time mechanism; VLM-3R folds 3D reconstruction into instruction tuning. Plus the six frontiers still