02 September 2008

Quantum Camera Revisited

In January of 2007 I posted about what I considered "Quantum Camera Components". I made reference to speech recognition technology, dynamic time warping, and depth perception as key elements of the so-called Quantum Camera of the future. I am happy to report that now, only a little more than a year and a half later, major elements of that system have evolved which are beyond what I was expecting. And the real fireworks is that the brain of the Quantum Camera has already been put to use in a beautiful music video for Radiohead "House of Cards". Here is text from Google explaining:

"No cameras or lights were used. Instead two technologies were used to capture 3D images: Geometric Informatics and Velodyne LIDAR. Geometric Informatics scanning systems produce structured light to capture 3D images at close proximity, while a Velodyne Lidar system that uses multiple lasers is used to capture large environments such as landscapes. In this video, 64 lasers rotating and shooting in a 360 degree radius 900 times per minute produced all the exterior scenes."

The data the LIDAR scanner produces in Radiohead's video is limited, ostensibly, by some arbitrary factors. I guess this includes such things as:

- scanner-head rotations per second
- number of lasers
- various settings concerning thresholds of signal to noise.

It is beautiful how the LIDAR imagery is distorted and skewed through analog processes. The video has something reminiscent of the Rutt Etra video synthesizer, not only for its visual aesthetics but also in relation to how the property of depth is a factor in the data-visualization.

Yet it is possible to imagine that in the not-to-distant future, a more evolved scanning system would be able to reconstruct a photographically accurate four-dimensional reality. In this light, the team behind this Radiohead video have achieved a remarkable milestone in the history of moving images and cinema. If we are already seeing LIDAR visualizations in this year of 2008, it should be within the next couple years that the full impact of this new medium will reach its visual potential. I can't wait!

Meanwhile on the speech recognition front, Cambridge-based company Everyzing has technology which, if its scalable for the entire searchable web should be able to help on the Hypercubist sound stage. While not conceived for use in production, it would probably be a matter of logistics and inspiration to get the computers needed to crunch the transciption data in close to real time.

As you watch the video above, try to imagine replacing key words of Thomas Wilde's monologue as per the following rubric. It won't work in every sentence, but it might help you see my perspective on the future of this technology and how it relates to cinematic workflow.

"major search portal" - major movie studio
"large destination website with multimedia" - major motion picture
"if these videos are on the web" - if these takes are in the material we shot
"search economy" - editing/tesseracting timeline/hyperspace
"users expect fine grain control online" - editors/tesseractors expect fine grain control when editing

This brings me to the dynamic time stretching issues, and to a recent post I made about Microsoft's Video Synth project. At the time I merely archived the video without comment. What I liked about the technology was its ability to use geolocation data to create visually meaningful relationships from various different photographs.

The potential to combine a technology like Video Synth with a LIDAR scanning system strikes me as the next logical step. This next generation Quantum Camera could allow the true atomization of moving images and usher in an entire new era of cinematic imagery. Instead of Frames, Hypercubes; instead of Continuity, Quantinuity; instead of editing, Tesseracting. This will be a wild ride, and the evidence suggests we've only just begun.

Wow. Just contemplating the Petabytes of data this new cinema will generate gives me a headache.

Thanks to Brandon Rosenbluth for dropping the hint about "House of Cards".

1 comment:

Gabriel Shalom said...


"House Of Cards" Clip Created Exclusively Using Geometric Informatics and Velodyne Lidar Technology

World Premiere Today at Google

Data Made Available To Enable Fans To Create Their Own Short Clips

In keeping with their decision not to make conventional promotional music videos for any of the tracks on In Rainbows, Radiohead used absolutely no cameras in the making of its new "House Of Cards" clip, which can be viewed at Google.com

Instead, "House Of Cards", directed by James Frost of Zoo Films, was created utilizing two technologies: Geometric Informatics and Velodyne Lidar. The Geometric Informatics scanning system employs structured light to capture detailed 3D images at close proximity, and was used to render the performances of Radiohead's Thom Yorke, the female lead and several partygoers. The Velodyne Lidar system uses multiple lasers to capture large environments in 3D, in this case 64 lasers rotating and shooting in a 360 degree radius 900 times per minute, capturing all of the exterior scenes and wide party shots. Geometric processed their own data while 510 Systems processed the Velodyne Lidar data. The data was then manipulated by Union Editorial and the Syndicate to create the final result.

Google will premiere the video today at http://code.google.com/radiohead Additionally, the band will be making available the data used to create the video for fans to manipulate into their own unique short clips. A short documentary detailing the unique process used to create the video will also be available via this link, as will 3D renderings of selected scenes.

Of Radiohead's decision to eschew cameras for the "video," Thom Yorke commented, "I always like the idea of using technology in a way that it wasn't meant to be used, the struggle to get your head round what you can do with it. I liked the idea of making a video of human beings and real life and time without using any cameras, just lasers, so there are just mathematical points--and how strangely emotional it ended up being."