Volumetric Capture with Simon Windsor - Dimension Films & Hammerhead VR

I was at the Broadcast and Video Exhibition at London Excel and had the pleasure of hearing first hand from one of the leading companies in Volumetric Capture solutions for entertainment, art and culture. Simon Windsor, Co Founder of Hammerhead and Dimensions and their Managing Director would share his industry knowledge of the process of Volumetric Capture as well as provide individual cases of their work in the field with companies and clients such as WWE and Anthony Joshua.

Dimension Studios is an Immersive Media Company, specialising in Volumetric Capture, which they assert is the next generation of storytelling. Working in conjunction with the first mixed reality Microsoft studio globally, the immersive content studio and game development team focuses on the entertainment space and commercial applications. Dimension captures life in wonderful volumetric detail, providing a step change in the realism that can be achieved for the creation of virtual humans and environments. Inject new life into stories, games and experiences for both immersive and 2D media.

“At Dimension, the Volumetric capture Studios with Digital Catapult and Microsoft Partnership is leading the way in creating Virtual Worlds & Virtual Humans with their Extended Reality storytelling experimentation. Their film ABE, a paragon of storytelling narrative design is a short film faithfully recreated in VR where the audience steps into the VR world and into the role of the victim, a psychological horror, showing how to translate an existing 2D script into a VR world. A state-of-the-art volumetric video stage is used for performance capture, and a separate scanning stage for 3D character and head capture. Whether creating volumetric video or 3D images, they can record shape and detail from every angle, for realistic content that can be used across multiple platforms.”

Their 2016 project Syren, was a full VR Game, Available on steamVR, PSVR, HTC and showed more and more a realistic world, much achieved through the powerful UE4 game Engine.

Using Photogrammetry use, Dimension’s project’s often mirror the world, combined with realistic humans traditional avatars such as their mixed Witherspoon worlds and Timeslice Films, pioneered the Bullet time scene from the Matrix using similar capture methods.

Inside the first Volumetric capture studio, with backing from digital catapult, the Microsoft Stage uses a green background and combination of two stages for more ways to capture virtual humans. This technique has also been used to bring famous icons into the cortual world such as Anthony Joshua.

The combination of volumetric images, volumetric video, 3D video and 10) cameras with 10 overhead create a wire frame and mesh, where a video texture can be applied and the coupe can be graded and soon brought into real time games engines, as an mp4 or obj file.

Using both IR and RGB allows the capture of depth information more accurately and can be captured at up to 90 FBS, with 10 GB per second data being compressed to 10 MB, much like Netflix. 

The recent award winning Bohemian Rhapsody used Volumetric Capture techniques, in order to use virtual camera movements through the crowds.

In regards to the performer being captured, we gain al like like replica from a linear performance with one can be it from any angle. It’s is also a verbatim one to one performance of the artist of talent. A clever mesh technology is used to focus on areas needing to be captured more realistically such as the surface of the face, hands, branding. A target to protect the detail around the core area and the technology automatically locks on to the faces and even world well with highly visual areas such as tattoos. 

Being able to rotate and view and navigate around the virtual character, they’re is no uncanny valley, a human connection and a more believable experience. Optically capturing a true representation of a person, you can truly feel their emotional and physical presence and heightens the connection to them. The volumetric capture method can also be used in touch screen and mobile devices, but as free view point media, where 2D images become 3D and one can rotate the image.

Volumetric storytelling, an interactive and immersive experimental medium is a highly engaging new method of entertainment and education, where the immersion doesn’t break, a paragon being the Anthony Joshua experience, which was so visceral and you can truly sense the physical appearance of the athlete.  

The talk focused on how space becomes our area to enjoy the immersive medium and how communication and storytelling will be transformed allowing for new way to interact and work together. With a free viewpoint, creative freedom, realism and connection with virtual humans for virtual worlds, Simon Windsor emphasised the potential of volumetric capture for good and as a component part of a story, where characters are placed in the forefront and a more game like environment in the background, blending the two seamlessly. By looping volumetric video captures, the performance experience can also last inherently longer or even everlasting. The  replacement of props to resolve the lack of detail in fine objects such as guitar scenes are effectively changed in post production. 

It was very interesting to hear Simon Windsor’s thoughts on the trajectory on where media is going and the rise of the visual precedented world and how there is a growing need to capture virtual realistic humans and create a connection. Using real time capture and a real time engine, real time movement can be captured with motion capture data which drives the character action. This is also useful in education and teaching to provide holograms and virtual teachers. The future may even involve this medium for personal memories and in the final question, I would ask fore more information on the ethical considerations and challenges faced by Simon Windsor and Dimension Studios in their work in Volumetric Capture.