BGH1: Creating a Volumetric Studio

Founded in 2015 by Brandon Zamel and Steven-Charles Jaffe, Springbok Entertainment is a unique production company that sits at the cross section of storytelling and emerging technologies. Zamel previously ran digital interactive content for Johnny Depp’s production company while Jaffe comes from the feature film and television world, producing several successful films such as Ghost, Star Trek VI: the Undiscovered Country, Strange Days, and more.

For the past few years, Springbok Entertainment has been producing AR (Augmented Reality) and VR (Virtual Reality) projects for fine art installations, philanthropic work, and narrative film. Recently they were nominated for a special Emmy Award for The 100%, which tells the true-life story of professional ballerina Maggie Kudirka, who was diagnosed with metastatic breast cancer at the age of 23. The VR film is an immersive experience that brings viewers alongside Kudirka’s rise as a ballerina. The 100% was also the first project to be shot at the Metastage in Los Angeles, which uses Microsoft's volumetric capture technology. “The film premiered at the Tribeca Film Festival, where we won the Tribeca X Award for Best VR experience,” explained Zamel. “We accomplished a lot of firsts with the technology at that point.”

Now, Springbok Entertainment is investing more time and resources in building a volumetric video pipeline. The LA-based company recently employed Panasonic LUMIX BGH1 cameras to create volumetric content. They had previously worked with Panasonic through their sister non-profit company, Springbok Cares, which has worked in the cancer research space using VR technology. From that experience, they decided to incorporate Panasonic gear as a core component for their unique volumetric workflow.

A still from The 100%, Springbok Entertainment's VR film that tells the true-life story of professional ballerina Maggie Kudirka. (All photos courtesy of Springbok Entertainment)

Volumetric video is created by capturing a live action subject using multiple cameras, and employs photogrammetry or 3D scanning to turn every frame into a 3D model. For most applications, the cameras will be set up in a circle, or semi-circle, pointed inwards on a green screen stage. “The cameras are basically triangulating all the details and points to create a full 3D model of that subject,” explained Springbok Technical Director Sam Maliszewski. “You can do anything with that model – you can put it into a feature film as a background extra, or VFX shot, or use the scans in a video game. There are several different techniques to combine that with LiDAR, depth cameras, or just doing straight photogrammetry.”

Springbok Entertainment is currently using 12 BGH1s for volumetric capture, either fully wrapped around a subject, or in a 180-degree semi-circle. The BGH1s are capturing high resolution textures and are accompanied with depth cameras, which handle the scanning of the actual geometry. “We’re using the BGH1s mainly due to their ability to be programmed and powered over network,” explained Maliszewski. “Having that accessibility in combining them with other cameras or devices, the BGH1 is the perfect camera for this work.”

According to Maliszewski, their camera ring is not tied to a specific location and can be reconfigured or recalibrated if they want to point the cameras in a specific direction instead of a full 360-degrees. They’re also able to reconstruct without a green screen. “I'm genlocking all of the cameras and getting them to be frame accurate,” he revealed. “If in a single frame between camera, a subject moves just a ¼-inch, it’s not going to align properly, and you won’t be able to reconstruct. We’re able to take that genlock signal and sync that with the other depth cameras.”

Springbok Entertainment team is currently using 12 BGH1 cameras for volumetric capture, either fully wrapped around a subject, or in a 180-degree semi-circle

The BGH1s are capturing 4K/30p H.265 files in-camera. Since they are using the texture for projection and not reconstruction, Maliszewski feels H.265 is a viable format. They’re also capturing in V-Log to capture as much dynamic range as possible. Also, depending on the pipeline, they will dial in different levels of sharpness and/or contrast. “The cameras match up freakishly well right out of the box,” said Maliszewski. “There’s very little color matching that we need to do.”

For volumetric capture, it’s important to light very flat and because the cameras need to be precisely calibrated, Maliszewski keeps the moving parts to a minimum. The team uses wide-angle Laowa 9mm f/2.8 Zero-D prime lenses that match the depth cameras in terms of field of view. “Once the focus is set, it's set,” he explained. “There’s no changing of zoom or swapping of lenses because while the tolerances are good, switching between lenses could mean that something can be off by a pixel.”

Depending on the project, Springbok Entertainment primarily works within Unity, a real-time 3D gaming engine that has transitioned into other industries, including film and automotive. According to Zamel, working in Unity looks like a video game. “If we’re in a narrative structure, we have our script, and then it's essentially an assembly of all those different components until we get to where we want to go,” revealed Zamel. “If we're doing a more traditional piece, then it looks more like a VFX pipeline. It really depends on what the project calls for.”

To monitor and control settings to the BGH1s, the team used the LUMIX Tether for Multicam application.

“Because there's a significant amount of processing that needs to be done to all the footage,” said Maliszewski, “we only process the footage that we're going to use. There's a lot of pre-vis in advance to figure out the blocking, making sure everything works right there. Before we process the footage, we'll do rough cuts with flat 2D images to figure out which take we want, or if it’s the right timing. We want to know how everything is going to play out.”

According to Maliszewski, syncing multiple BGH1s and controlling them as one has enabled a workflow that did not exist before. Before working with the BGH1s, he would be working with either a webcam or some sort of computer-based Ethernet camera, which did not capture a cinematic look, or provide wide dynamic range. “Other box cameras at the closest price point give you multiple sync between cameras, but they don't give you back control, or the ease of use of by powering over Ethernet while running fewer cables. It feels like Panasonic came over to me and said, ‘Hey Sam, here’s your camera.’”

Many of the tools that originated in the video gaming industry are becoming fundamental tools for film and TV production and major tech companies are investing time and resources into augmented and mixed reality. Zamel wants Springbok Entertainment to be at the forefront of new technology to create a better experience, or tell a better story. At the same time, he is cautious not to overstep and rely on this technology, which could come at the cost of the experience.

Springbok Entertainment's camera ring with BGH1s genlocked in order for them to be frame accurate.

“The technology can't be the feature, the technology needs to be there to support the storyteller,” explained Zamel. “In relation to this conversation, what Panasonic has enabled in our workflow is the ability to incorporate a type of capture that we are huge believers in. Not just for the stories we want to tell, but where we believe the industry is going as a whole.”

According to Maliszewski, a big part of the investments in AR/VR technology involves getting real objects, people, and performances that are both believable and low cost in terms of data and rendering. “This is something that is niche right now,” said Maliszewski, “but as these technologies are progressing, it’s going to become a vital foundation for how content will be viewed in the future.

“It’s not going to replace movie theaters or television in terms of passive entertainment,” he continued. “But when it gets to a point where you need to see a virtual person, or hologram, right in front of you, this is the way to do that. I’m excited to see how that progresses in the same way that the DSLR was for the independent filmmaker.”

The Netflix-approved BGH1 contains a 10.2-megapixel Live MOS Sensor with Dual Native ISO technology. You can also output via HDMI and 3G-SDI simultaneously and includes timecode and genlock BNC ports, and more.

For more information on Springbok Entertainment, visit their official website here.