Engineers at Disney Research facility have developed a technology that tailors a facial capture system to the characteristics of a specific actor's expressions while significantly minimizing the time and effort it would normally require.
Previously, to achieve a similar outcome in graphical terms, a large set of training data in the form of stills and images will be captured normally ranging from hundreds to thousands.
The researchers showed off this technology at the International Conference on 3D Vision in Palo Alto, California on Tuesday.
"Real-time markerless facial performance capture has been growing in popularity for film and video game production, thanks to advances in machine learning," said Markus Gross, vice president for Disney Research, while speaking to Eureka Alert. "By reducing the amount of facial imagery necessary to train these systems, our team has taken a big step toward increasing the flexibility and efficiency of this approach."
"It takes a lot of labor to capture not only a full spectrum of facial expressions but to do so under a variety of lighting conditions and from different camera angles," said Kenny Mitchell, senior research scientist. "Our idea was that if we could strategically capture an actor's expressions under certain conditions, we could synthesize all of the training data for a target scenario and save a lot of time."
The face capture technology uses only a tiny number of recordings and synthetically generates data to train the system. A series of a uniformly-lit multi-camera rig (which only one camera is active, while the rest are for lighting) helps to create an adjustable 3D model of the actor's face which then can be used to create motions as required.
It is a simple setup with an extremely impressive output, and it could open various types of avenues for studios and movies that cannot afford high-end motion capturing at the moment, while also being accurate and time-saving at the same time. The technology needs further refining by Disney Research to increase the accuracy level even further.