Pusan National University Researchers Combine AI Motion Experts for Dynamic 3D Scene Reconstruction

A research team led by Professor Kyeongbo Kong at Pusan National University has developed a pair of artificial intelligence frameworks designed to improve the three-dimensional reconstruction of complex, dynamic environments. By combining multiple specialized motion representations through a Mixture-of-Experts architecture, the new methods address longstanding trade-offs in dynamic scene modeling, as first reported by TechXplore.

Accurately reconstructing dynamic 3D scenes from visual data is essential for next-generation technology applications, including autonomous driving, virtual reality, robotics, digital twins, and spatial computing. Developers widely rely on Dynamic Gaussian Splatting techniques to model movement. However, real-world scenes feature diverse and unpredictable dynamics, and existing dynamic representations typically excel under specific conditions while struggling to generalize across heterogeneous real-world motion.

To solve this constraint, the research team created two complementary Mixture-of-Experts systems, designated as MoE-GS and MoDE. The MoE-GS framework trains several dynamic Gaussian models independently and then adaptively blends their outputs using a learned expert routing mechanism. This approach enables the framework to select and combine the optimal expert representation for specific spatial regions and distinct time frames.

The second architecture, MoDE, takes a joint optimization approach using a unified spatial framework. Instead of optimizing separate dynamic models in isolation, MoDE integrates multiple deformation experts into a single, shared Gaussian representation during training. Both systems rely on leveraging specialized dynamic models in tandem rather than forcing a single representation to calculate all environmental movements.

"We conducted a systematic analysis and have come to the understanding that no existing Dynamic Gaussian Splatting method consistently performs best across diverse scenarios," Kong said in a statement. "Motivated by this finding, we introduce MoE-GS, the first framework that adaptively combines multiple specialized dynamic Gaussian models through a Mixture-of-Experts architecture instead of relying on a single representation."

Experimental findings demonstrated that merging complementary motion experts produces higher scene reconstruction fidelity in complex scenarios containing multi-modal dynamic motion. The adaptive routing mechanism dynamically allocates visual rendering tasks to the expert model best suited for each zone and frame, maintaining flexibility without compromising visual rendering quality.

The development team highlighted that as artificial intelligence systems progress toward interacting directly with real-world physical environments, accurately modeling complex spatial dynamics will become critical. The framework offers practical potential for physical AI models, world models, autonomous systems, and interactive spatial computing platforms operating in non-static surroundings.

"Our findings suggest that combining multiple specialized motion representations can be an effective way to handle heterogeneous dynamics that are difficult for a single representation to model consistently," Kong added. The complete findings, authored by In-Hwan Jin and colleagues, were published in the IEEE Transactions on Pattern Analysis and Machine Intelligence under the title "On the Design of Mixture-of-Experts for Dynamic Gaussian Splatting."