Perceive, Predict, and Plan: a Unified World Action Model for Egocentric Dexterous Manipulation
Jointly learning 3D perception, action-conditioned video prediction, and goal-directed planning from egocentric interactions.
Vision · Learning · Robotics
Zirui Wu.
Seeing the world.
Learning to act in it.
I work on 3D vision and world models, with a focus on understanding and generating interactions with the physical world.
01 Research
From Rays to Projections: Better Inputs for Feed-Forward View Synthesis
Point cloud projections as inputs for consistent feed-forward view synthesis and self-supervised learning.
Hover to view original ↔Tap to compare with original ↔
3D Gaussian Inverse Rendering with Approximated Global Illumination
Real-time global illumination approximation using indirect emitters in screen space.
DualMap: Online Open-Vocabulary Semantic Mapping for Natural Language Navigation in Dynamic Changing Scenes
Understanding and navigating changing environments through natural language queries.
MARS: An Instance-aware, Modular and Realistic Simulator for Autonomous Driving
A modular neural simulator for realistic autonomous driving scenes.
Robotic localization
LATITUDE: Robotic Global Localization with Truncated Dynamic Low-pass Filter in City-scale NeRF
A two-stage pipeline for global localization in a city-scale neural map.
02 About
Background & serviceI am pursuing a PhD in Robotics and Autonomous Systems at HKUST (Guangzhou), co-supervised by Martin R. Oswald and Jie Song.
I received my BSc in Computer Science and Technology from Beijing Institute of Technology.
Curriculum vitae ↗Teaching
2024 / 2026
Learning for 3D Vision
Head Teaching Assistant · Fall
Lecturer: Jie Song
Build a CroCo transformer from scratch ↗2025
Calculus I
Teaching Assistant · Fall
Reviewing
CVPR · ICCV · ECCV · ICLR · 3DV · ICRA · IROS
RA-L · RAM · Pattern Recognition
03 Beyond research
A few other things
On the field
Behind the plate.
I captained BIT’s Kunpeng Baseball Team in 2021–2022. Catcher, no. 28—a nod to Buster Posey.
Behind the bar
Another kind of brewing.
In 2020–2022, I managed a campus coffee shop serving free coffee. Our setup: a Rancilio Classe 5. My signature drink: a herbal-tea latte with flavored cream.