Hypercubist Manifesto – Pecha Kucha Berlin, 31 August 2010

Soon Your Computer Will See You Always

This little demo of some webcam background removal and face tracking trickery from Chris Harrison is auspicious of things to come for the future of viewing moving images. As long as our viewing devices are flat we'll have these slightly compromised aesthetics. Reminiscent of the previously posted DIY Nintendo Wii 3D Tracking Hack.

DIY Nintendo Wii 3D Tracking Hack

Seems to me after watching this video that widespread 3D holographic effects are just around the corner. I wonder how difficult it would be to get this working with most webcams in most laptops, or if the necessary sensor technology could be embedded easily.

Thanks to Rigo for showing me this video :)

Cubism vs. Hypercubism

I thought I would take a stab at a concise definition of Hypercubism, a word I use quite often and have until now perhaps not defined so exactly. For the sake of reference, here is an excerpt of Wikipedia's definition for Cubism:

In cubist artworks, objects are broken up, analyzed, and re-assembled in an abstracted form — instead of depicting objects from one viewpoint, the artist depicts the subject from a multitude of viewpoints to represent the subject in a greater context. Often the surfaces intersect at seemingly random angles, removing a coherent sense of depth. The background and object planes interpenetrate one another to create the shallow ambiguous space, one of cubism's distinct characteristics.

In contrast we might consider this summation of Hypercubism:

In hypercubist artworks, objects are particlized, analyzed and synthesized in a realistic form — instead of depicting all objects from one temporal perspective, the artist (or artists) depict the subject from a multitude of temporal perspectives to represent the subject in a greater temporality. Often the surfaces of intersect seamlessly, creating a coherent four-dimensional spacetime illusion. The background and object planes are always distinct to create deep concrete space, one of hypercubism's distinct characteristics

Besides being a bit of encyclopedic revisionism, these two definitions set up a useful theoretical dichotomy between Cubism and Hypercubism. Simply put:

Cubism shows multiple spaces in the same time while Hypercubism shows multiple times in the same space.

Microsoft's C-Dragon and Video Synth

I thought I would resurrect this post and talk a little bit more about what I think the implications are for Hypercubist Cinema with technologies like C-Dragon and Photosynth.

These technologies illustrate perhaps the most fundamental aspect of my theory of hypercubism, namely, an aesthetics in which multiple times are visible in the same space.

If the set of a fictional movie were photographed using such technology in concert with some of the multi-camera object-oriented methods and/or scanners, I imagine a robust hybrid system could emerge which would realistically texture map the photos onto detailed clouds of spatial data. The implications for the editing (read: tesseracting) afterward would be tremendous, allowing granularization of every object and every word, every facial expression or movement.

I predict that the difficulty will be in recording sound in such an environment. In a way, we might see a second era of silent films with the emergence of early hypercubist systems. My hunch is that commercial pressure will force new innovations in multi-track recording to the degree that individual sound sources in the same acoustic space will be able to be mixed independently of one another as if they'd been recorded in separate isolation booths. This theme of hypercubist synchronous sound deserves its own post in the future.

PS: sorry for the long BMW advertisement at the end of this video; BMW sponsor the TED talks...

MIT Creates Center For Future Storytelling

MIT's new Center for Future Storytelling is an amazing and encouraging development. Several of the projects they plan to take on are addressing key questions I have posed throughout the history of this blog.

3D Morphable Model Face Animation

What is amazing is that this video was posted to YouTube in 2006.

Eyeliner 3D

Is Eyeliner 3D really holography or just smoke and mirrors?
And does it even matter?

Given my background and formal education, as well as my admitted nostalgic love of certain masterpieces of celluloid cinema, it is challenging for me to conceive of my work in terms of depth. I have always been inclined to see 2-dimensional pictures in my head when imagining a story for a film. And I guess that's just it; proto-quantum cinema and/or live cinema are not really films at all; they may be "features" or even "feature-length" at times, but I am feeling more and more that the proto-quantum cinema will find its deepest roots in theater. Stumbling across technologies like Eyeliner 3D confirm this suspicion.

So while it may not be possible in my lifetime to realize true holographic projection (and certainly not nanopixels), several layers (perhaps as many in late-stage 32-bit video games) may be around the corner. The promise that these virtual layers of inszenierung may have for proto-quantum cinema may in fact cross the minimum thresholds of depth reconstruction necessary to truly define a new artform.

So do we really need absolute depth resolution? Or will foreground, midground and background (with some additional layers) suffice?

The Tyranny of The Frame

I have been thinking about Greenaway's "Tyranny of the Frame". It is pefectly acceptable to me that we should have a frame, whatever the format of the rectangle. I also have nothing against those artists who like to project on round things or whose fetish it is to go beyond our peripheral vision in 360° panorama.


The real tyranny of the frame of celluloid cinema is that it is in two dimensions. Above is an animated projection of a rotating tesseract. Why should the video frame be flat?

The camera of the last 100+ years of cinematic history is essentially a glorified eyeball with sophisticated spectacles.

The Quantum Camera will attach these free-floating, bespectacled eyeballs to brains. Brains capable of perceiving reality more like the way our nervous system works. When I was at University of Maryland I had a class about visual communication where the professor had us read a book about visual perception. Humans have depth perception. We can tell the foreground from the background without any trouble, and if something that was moving ceases to move, we can still distinguish it as a separate entity. We have had multi-track audio recording equipment for years. We need multi-depth video cameras. Cameras that record the background and the foreground to separate layers of video. Goodbye keying and matting, hello alpha channels!

But it needs to go further than just what keying can accomplish. Layers are still a 2D concept. Containers (I admit I borrow the word from conversations I have had with Philipp) are a much more appropriate model. Here is a film still from Lost In Translation:


The quantum camera would ideally see (at least) these containers:
(Restaurant (Charlotte) (Table (Food) ) (Steam) (Bob) )

Of course our eyes can discriminate an incredible level of detail:
(Bob (Costume (Sweater (Shirt) ) (Wristwatch) (Pants) )

And we can even infer things which we cannot see, such as socks, underwear and shoes.

It may also be necessary to "teach" the quantum camera in order to get it to learn to recognize these containers and their IDs. With the above example, I can imagine that it would work that first you would show the camera the empty table and benches and ID it Restaurant. Then you could put the food on the table and ID it Food. Then a threshold knob would be adjusted to catch the steam and ID it Steam. Lastly Bob and Charlotte would each be added to the composition and IDed respectively. Or perhaps you could use a combination of RFIDs and threshold settings for brightness, depth and movement on the camera.

When I was at Bard I attended a screening of Let's Get Lost by Bruce Weber (see this site for video clips). The screening was presented by Bard alumni Jeff Preiss, an accomplished cinematographer. With a successful career in commercial advertising, Jeff knew some secrets of the image industry. He mentioned that he had heard of major corporations developing cameras that photograph all surfaces of physical reality in the hope of creating photographic 3D space.

What are they waiting for?