From the robot’s view, is the red cone left or right of the blue cube?
During the shorter rigid turn, does the red tip pass over yellow?
If red continues straight, will it touch blue or yellow first?
Remove orange; then topple red to the right. Will blue fall?
As Vision-Language Models (VLMs) advance, they are increasingly expected to perform spatiotemporal reasoning over motion, actions, and view changes beyond what the input images show. Video World Models (VWMs) provide a natural unified engine for visual imagination by generating task-relevant visual content as additional evidence. However, existing approaches typically invoke a VWM through fixed pipelines centered on a predefined form of visual imagination, such as novel-view synthesis. To support flexible visual imagination for diverse reasoning tasks, we introduce CoWorld Reasoner, an agentic multimodal reasoning framework that uses a VWM as a unified visual reasoning engine. Its Video World Model-Driven Harness exposes multiple visual operations through tools and uses skills to guide the VLM in deciding whether to imagine, selecting an operation, constructing a generation request, and interpreting the resulting visual evidence. We further introduce VLM-VWM Harness Co-Evolution, which jointly revises tools, skills, and execution code based on execution feedback while keeping both model weights fixed. Across seven benchmarks spanning spatial intelligence, temporal and causal reasoning, and perceptual reasoning, CoWorld Reasoner outperforms direct reasoning with the same VLM. The strongest variant improves on all seven benchmarks, including relative accuracy gains of 11.8% on MMSI and 13.0% on ERQA, and outperforms all five evolving baselines using the same backbone across five benchmarks.
Across seven benchmarks, CoWorld Reasoner improves over direct reasoning with the same VLM.
CoWorld Reasoner achieves the highest aggregate accuracy among the compared evolving methods.
| Method | MMSI | SAT | Spin | SL-MCQ | SL-Open | ERQA | Causal | PB | Avg. |
|---|---|---|---|---|---|---|---|---|---|
| Base Models | |||||||||
| GPT-5-mini | 34.10 | 75.40 | — | 54.29 | 40.93 | — | 45.44 | — | — |
| GPT-5 | 41.90 | 80.10 | 78.50 | — | — | — | 54.17 | — | — |
| Gemini-2.5-Pro | 36.90 | 76.70 | 75.20 | 50.07 | 33.61 | 55.80 | 50.55 | — | — |
| Qwen3-VL-8B | 31.10 | 52.70 | 67.40 | — | — | 40.10 | 43.53 | — | — |
| Fine-tuned Models | |||||||||
| MiMo-Embodied | 30.20 | 78.67 | — | — | — | 46.75 | — | — | — |
| RoboBrain2.0 | 28.50 | 86.67 | — | — | — | 46.00 | — | — | — |
| Evolving Agents | |||||||||
| GPT-5.6 | 38.35 | 88.33 | 82.24 | 56.64 | 36.21 | 51.75 | 60.72 | 50.84 | 58.29 |
| + HSI | 35.05 | 87.67 | 73.46 | 55.71 | 29.57 | 57.25 | 58.06 | 48.92 | 54.39 |
| + ReasoningBank | 36.80 | 86.67 | 81.69 | 56.57 | 30.29 | 59.00 | 60.65 | 50.07 | 57.30 |
| + Meta-Harness | 37.42 | 88.33 | 83.31 | 55.86 | 30.57 | 60.00 | 60.07 | 49.83 | 57.63 |
| + ExpeL | 37.42 | 89.00 | 82.36 | 56.64 | 29.57 | 57.00 | 61.22 | 49.12 | 57.25 |
| + GEPA | 38.87 | 87.67 | 79.92 | 54.86 | 28.21 | 59.75 | 62.51 | 47.61 | 56.25 |
| + CoWorld Reasoner | 42.89 | 89.67 | 83.76 | 57.36 | 37.64 | 58.50 | 60.79 | 51.35 | 59.69 |
| Gain (pp) | +4.54 | +1.33 | +1.51 | +0.71 | +1.43 | +6.75 | +0.07 | +0.51 | +1.40 |
Gains span all four MMSI categories and reach 6.56 points for positional relationships. SAT perspective taking reaches 84.85%, the highest among the compared methods; goal reasoning also improves.
| Method | MMSI PR | MMSI Attr. | MMSI Mot. | MMSI MSR | MMSI All | SAT Ego | SAT Obj | SAT Act. | SAT Goal | SAT Pers. | SAT All |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Specialists and Agents | |||||||||||
| o1 | — | — | — | — | — | 78.30 | 82.60 | 73.00 | 73.50 | 69.70 | 74.60 |
| GPT-4.1 | 30.65 | 30.77 | 34.67 | 28.79 | 30.90 | 95.70 | 73.90 | 78.30 | 88.20 | 39.40 | 74.00 |
| GPT-5.4 | — | — | — | — | 37.50 | — | — | — | — | — | — |
| Qwen3-VL-8B | 33.72 | 26.15 | 28.00 | 29.80 | 31.10 | — | — | — | — | — | 52.70 |
| ViewFusion | 39.66 | 35.38 | 36.67 | 23.23 | 35.40 | — | — | — | — | — | — |
| OpenSpatial | — | — | — | — | 41.90 | — | — | — | — | — | — |
| World2VLM | — | — | — | — | — | 100.00 | 65.22 | 67.57 | 79.41 | 57.57 | 72.67 |
| MiMo-Embodied | — | — | — | — | 30.20 | — | — | — | — | — | 78.67 |
| RoboBrain2.0 | 26.44 | 33.85 | 30.00 | 29.29 | 28.50 | — | — | — | — | — | 86.67 |
| Spatial-Interactor | — | — | — | — | 32.20 | — | — | — | — | — | 74.00 |
| SenseNova-SI | 52.87 | 40.77 | 32.67 | 27.78 | 43.30 | — | — | — | — | — | — |
| AVIC | 31.23 | 44.62 | 35.33 | 32.32 | 33.80 | 100.00 | 78.20 | 83.70 | 85.20 | 54.50 | 79.30 |
| AVIC-R | — | — | — | — | — | 91.30 | 86.90 | 83.70 | 85.20 | 57.50 | 80.00 |
| MindJourney | — | — | — | — | — | 95.70 | 82.60 | 83.80 | 88.20 | 75.80 | 84.70 |
| SandboxVLM | — | — | — | — | — | 100.00 | 82.60 | 79.40 | 92.90 | 70.00 | 84.10 |
| Evolving Agents | |||||||||||
| GPT-5.6 | 35.59 | 42.40 | 40.00 | 41.62 | 38.35 | 100.00 | 95.65 | 89.19 | 83.82 | 78.79 | 88.33 |
| + HSI | 30.22 | 42.40 | 38.62 | 40.10 | 35.05 | 100.00 | 91.30 | 91.89 | 82.35 | 77.27 | 87.67 |
| + ReasoningBank | 33.60 | 43.20 | 40.69 | 38.07 | 36.80 | 97.83 | 91.30 | 87.84 | 83.82 | 77.27 | 86.67 |
| + Meta-Harness | 34.19 | 45.60 | 36.55 | 41.12 | 37.42 | 91.30 | 97.83 | 93.24 | 80.88 | 81.82 | 88.33 |
| + ExpeL | 32.41 | 48.80 | 37.24 | 43.15 | 37.42 | 100.00 | 97.83 | 89.19 | 83.82 | 80.30 | 89.00 |
| + GEPA | 36.18 | 44.00 | 39.31 | 42.13 | 38.87 | 93.48 | 93.48 | 89.19 | 85.29 | 80.30 | 87.67 |
| + CoWorld Reasoner | 42.15 | 46.40 | 43.45 | 42.13 | 42.89 | 100.00 | 93.48 | 89.19 | 85.29 | 84.85 | 89.67 |
| Gain (pp) | +6.56 | +4.00 | +3.45 | +0.51 | +4.54 | +0.00 | −2.17 | +0.00 | +1.47 | +6.06 | +1.33 |
Generated evidence contributes most on causal occlusion and collision cases. The agent adapts its skill and tool use to the task, while direct perception often needs no generation; the stronger H3 world model also yields larger gains than LTX across the evaluated benchmarks.
The harness evolves on development examples, transfers to Grok-4.5 with one additional evolution round, and recovers errors made by direct GPT-5.6 reasoning.
Please cite our paper as:
@misc{cheng2026coworldreasoner,
title = {CoWorld Reasoner: Agentic Multimodal Reasoning with the Evolving Video World Model-Driven Harness},
author = {Dongjie Cheng and Ruifeng Yuan and Ran Wei and Ning Song and Yongqi Li and Wenjie Li},
year = {2026},
eprint = {2610.21910},
archivePrefix = {arXiv},
primaryClass = {cs.AI},
url = {https://arxiv.org/abs/2610.21910}
}
We thank the authors and maintainers of the benchmarks and video world models used in this project.
Third-party models, datasets, and dependencies retain their own licenses.