Max Planck and CMU Researchers Develop Smartphone-Based Motion Capture System

Computer vision researchers from the Max Planck Institute for Intelligent Systems (MPI-IS) and Carnegie Mellon University have developed a technology that allows standard camera setups—including everyday smartphones—to record precise multi-person motion capture without requiring specialized suits or physical tracking dots.

Traditional motion capture in film and visual effects relies on physical markers attached to actors in controlled studio environments. That legacy process remains capital-intensive, labor-heavy, and subject to tracking failures when subjects obscure one another during close physical contact, such as dancing or martial arts. While existing commercial markerless systems eliminate physical hardware on actors, they typically deliver lower precision and higher costs.

The new framework, named Markerless Accurate Multi-person Motion Acquisition (MAMMA), was unveiled at the CVPR 2026 conference in Denver, as reported by TechXplore (https://techxplore.com/news/2026-09-virtual-markers-smartphone-cameras-reconstruct.html). MAMMA analyzes multi-camera video feeds, identifies 512 virtual tracking markers directly on the skin of subjects, and aligns those points with a standard human body model known as SMPL-X to reconstruct detailed physical poses.

The researchers demonstrated that MAMMA can operate using off-the-shelf consumer hardware, such as a network of iPhones, while matching the precision of studio marker systems. Benchmark tests showed the discrepancy in pose accuracy between MAMMA and traditional suit-and-marker setups was less than one millimeter, while processing movement data nearly three times faster by eliminating manual data cleanup.

Hanz Cuevas Velasquez, a research scientist in the Perceiving Systems Department at MPI-IS and lead author of the research, emphasized that the software achieves high-precision alignment while operating on accessible consumer hardware rather than specialized studio gear.

The development team anticipates applications spanning clinical medicine, sports science, virtual reality, and video game production. Anastasios Yiannakidis, a research engineer at MPI-IS, highlighted that the markerless capability enables physical therapists and biomechanics researchers to analyze patient movement in non-clinical environments without cumbersome hardware.

The system also targets independent game development and media creation by lowering production barriers for high-fidelity character animation. Tsvetelina Alexiadis, Data Team Lead on the project, noted that MAMMA enables smaller studios to record complex, contact-heavy animation sequences like wrestling or dancing without operating a dedicated motion capture studio.

The research team has made its dataset and underlying software tools open to the academic community to encourage further research into multi-person motion tracking and biomechanical modeling.