BY LEV MANOVICH
Cinema, the Art of the Index [1]
Thus far, most discussions of cinema in the digital age have focused on the possibilities of interactive narrative. It is not hard to understand why: since the majority of viewers and critics equate cinema with storytelling, digital media is understood as something that will let cinema tell its stories in a new way. Yet as exciting as the ideas of a viewer participating in a story, choosing different paths through the narrative space, and interacting with characters may be, they only address one aspect of cinema which is neither unique nor, as many will argue, essential to it: narrative.
The challenge which digital media poses to cinema extends far beyond the issue of narrative. Digital media redefines the very identity of cinema. In a symposium that took place in Hollywood in the spring of 1996, one of the participants provocatively referred to movies as “flatties” and to human actors as “organics” and “soft fuzzies.”[2] As these terms accurately suggest, what used to be cinema’s defining characteristics have become just the default options, with many others available. When one can “enter” a virtual three-dimensional space, viewing flat images projected on the screen is hardly the only option. When, given enough time and money, almost everything can be simulated in a computer, filming physical reality is just one possibility.
This “crisis” of cinema’s identity also affects the terms and the categories used to theorize cinema’s past. French film theorist Christian Metz wrote in the 1970s that “Most films shot today, good or bad, original or not, ‘commercial’ or not, have as a common characteristic that they tell a story; in this measure they all belong to one and the same genre, which is, rather, a sort of ‘super-genre’ [‘sur-genre’]” (402). In identifying fictional films as a “super-genre” of 20th-century cinema, Metz did not bother to mention another characteristic of this genre because at that time it was too obvious: fictional films are live-action films, i.e. they largely consist of unmodified photographic recordings of real events which took place in real physical space. Today, in the age of computer simulation and digital compositing, invoking this characteristic becomes crucial in defining the specificity of 20th-century cinema. From the perspective of a future historian of visual culture, the differences between classical Hollywood films, European art films, and avant-garde films (apart from abstract ones) may appear less significant than this common feature: that they relied on lens-based recordings of reality. This essay is concerned with the effect of the so-called digital revolution on cinema as defined by its “super-genre” of fictional live-action film.[3]
During cinema’s history, a whole repertoire of techniques (lighting, art direction, the use of different film stocks and lenses, etc.) was developed to modify the basic record obtained by a film apparatus. And yet behind even the most stylized cinematic images we can discern the bluntness, the sterility, the banality of early 19th-century photographs. No matter how complex its stylistic innovations, the cinema has found its base in these deposits of reality, these samples obtained by a methodical and prosaic process. Cinema emerged out of the same impulse that engendered naturalism, court stenography, and wax museums. Cinema is the art of the index; it is an attempt to make art out of a footprint.
Even for Andrei Tarkovsky, film-painter par excellence, cinema’s identity lay in its ability to record reality. Once, during a public discussion in Moscow in the 1970s, he was asked the question as to whether he was interested in making abstract films. He replied that there can be no such thing. Cinema’s most basic gesture is to open the shutter and to start the film rolling, recording whatever happens to be in front of the lens. For Tarkovsky, an abstract cinema is thus impossible.
But what happens to cinema’s indexical identity if it is now possible to generate photorealistic scenes entirely in a computer using 3-D computer animation; to modify individual frames or whole scenes with the help a digital paint program; to cut, bend, stretch and stitch digitized film images into something which has perfect photographic credibility, although it was never actually filmed?
This essay will address the meaning of these changes in the filmmaking process from the point of view of the larger cultural history of the moving image. Seen in this context, the manual construction of images in digital cinema represents a return to 19th-century pre-cinematic practices, when images were hand-painted and hand-animated. At the turn of the 20th century, cinema was to delegate these manual techniques to animation and define itself as a recording medium. As cinema enters the digital age, these techniques are again becoming commonplace in the filmmaking process. Consequently, cinema can no longer be clearly distinguished from animation. It is no longer an indexical media technology but, rather, a sub-genre of painting.
This argument will be developed—in three stages. I will first follow a historical trajectory from 19th-century techniques for creating moving images to 20th-century cinema and animation. Next I will arrive at a definition of digital cinema by abstracting the common features and interface metaphors of a variety of computer software and hardware that are currently replacing traditional film technology. Seen together, these features and metaphors suggest a distinct logic of a digital moving image. This logic subordinates the photographic and the cinematic to the painterly and the graphic, destroying cinema’s identity as a media art. Finally, I will examine different production contexts that already use digital moving images—Hollywood films, music videos, CD-ROM games and artworks—in order to see if and how this logic has begun to manifest itself.
A Brief Archaeology of Moving Pictures
As signified by its original names (kinetoscope, cinematograph, moving pictures), cinema was understood, from its birth, as the art of motion, the art that finally succeeded in creating a convincing illusion of dynamic reality. If we approach cinema in this way (rather than the art of audio-visual narrative, or the art of a projected image, or the art of collective spectatorship, etc.), we can see it superseding previous techniques for creating and displaying moving images.
These earlier techniques shared a number of common characteristics. First, they all relied on hand-painted or hand-drawn images. The magic lantern slides were painted at least until the 1850s; so were the images used in the Phenakistiscope, the Thaumatrope, the Zoetrope, the Praxinoscope, the Choreutoscope and numerous other 19th-century pre-cinematic devices. Even Muybridge’s celebrated Zoopraxiscope lectures of the 1880s featured not actual photographs but colored drawings painted after the photographs (Musser 49-50).
Not only were the images created manually, they were also manually animated. In Robertson’s Phantasmagoria, which premiered in 1799, magic lantern operators moved behind the screen in order to make projected images appear to advance and withdraw (Musser 25). More often, an exhibitor used only his hands, rather than his whole body, to put the images into motion. One animation technique involved using mechanical slides consisting of a number of layers. An exhibitor would slide the layers to animate the image (Ceram 44-45). Another technique was to slowly move a long slide containing separate images in front of a magic lantern lens. 19th-century optical toys enjoyed in private homes also required manual action to create movement—twirling the strings of the Thaumatrope, rotating the Zoetrope’s cylinder, turning the Viviscope’s handle.
It was not until the last decade of the 19th century that the automatic generation of images and their automatic projection were finally combined. A mechanical eye was coupled with a mechanical heart; photography met the motor. As a result, cinema—a very particular regime of the visible—was born. Irregularity, non-uniformity, the accident and other traces of the human body, which previously inevitably accompanied moving image exhibitions, were replaced by the uniformity of machine vision.[4] A machine that, like a conveyer belt, was now spitting out images, all sharing the same appearance, all the same size, all moving at the same speed, like a line of marching soldiers.
Cinema also eliminated the discrete character of both space and movement in moving images. Before cinema, the moving element was visually separated from the static background as with a mechanical slide show or Reynaud’s Praxinoscope Theater (1892) (Robinson 12). The movement itself was limited in range and affected only a clearly defined figure rather than the whole image. Thus, typical actions would include a bouncing ball, a raised hand or eyes, a butterfly moving back and forth over the heads of fascinated children—simple vectors charted across still fields.
Cinema’s most immediate predecessors share something else. As the 19th-century obsession with movement intensified, devices that could animate more than just a few images became increasingly popular. All of them—the Zoetrope, the Phonoscope, the Tachyscope, the Kinetoscope—were based on loops, sequences of images featuring complete actions which can be played repeatedly. The Thaumatrope (1825), in which a disk with two different images painted on each face was rapidly rotated by twirling a string attached to it, was in its essence a loop in its most minimal form: two elements replacing one another in succession. In the Zoetrope (1867) and its numerous variations, approximately a dozen images were arranged around the perimeter of a circle.[5] The Mutoscope, popular in America throughout the 1890s, increased the duration of the loop by placing a larger number of images radially on an axle (Ceram 140). Even Edison’s Kinetoscope (1892-1896), the first modern cinematic machine to employ film, continued to arrange images in a loop (Musser 78). 50 feet of film translated to an approximately 20-second long presentation—a genre whose potential development was cut short when cinema adopted a much longer narrative form.
From Animation to Cinema
Once the cinema was stabilized as a technology, it cut all references to its origins in artifice. Everything which characterized moving pictures before the 20th century—the manual construction of images, loop actions, the discrete nature of space and movement—was delegated to cinema’s bastard relative, its supplement, its shadow—animation. 20th-century animation became a depository for 19th-century moving-image techniques left behind by cinema.
The opposition between the styles of animation and cinema defined the culture of the moving image in the 20th century. Animation foregrounds its artificial character, openly admitting that its images are mere representations. Its visual language is more aligned to the graphic than to the photographic. It is discrete and self-consciously discontinuous: crudely rendered characters moving against a stationary and detailed background; sparsely and irregularly sampled motion (in contrast to the uniform sampling of motion by a film camera—recall Jean-Luc Godard’s definition of cinema as “truth 24 frames per second”), and finally space constructed from separate image layers.
In contrast, cinema works hard to erase any traces of its own production process, including any indication that the images we see could have been constructed rather than recorded. It denies that the reality it shows often does not exist outside of the film image, the image which was arrived at by photographing an already impossible space, itself put together with the use of models, mirrors, and matte paintings, and which was then combined with other images through optical printing. It pretends to be a simple recording of an already existing reality—both to a viewer and to itself.[6] Cinema’s public image stressed the aura of reality “captured” on film, thus implying that cinema was about photographing what existed before the camera, rather than “creating the ‘never-was’” of special effects.[7] Rear projection and blue-screen photography, matte paintings and glass shots, mirrors and miniatures, push development, optical effects and other techniques which allowed filmmakers to construct and alter the moving images, and thus could reveal that cinema was not really different from animation, were pushed to cinema’s periphery by its practitioners, historians, and critics.[8] Today, with the shift to digital media, these marginalized techniques move to the center.
What is Digital Cinema?
A visible sign of this shift is the new role that computer-generated special effects have come to play in Hollywood industry in the last few years. Many recent blockbusters have been driven by special effects, feeding on their popularity. Hollywood has even created a new mini-genre of “The Making of” videos and books, which reveal how special effects are created.
I will use special effects from a few recent Hollywood films for illustrations of some of the possibilities of digital filmmaking. Until recently, Hollywood studios were the only ones who had the money to pay for digital tools and for the labor involved in producing digital effects. However, the shift to digital media affects not just Hollywood, but filmmaking as a whole. As traditional film technology is universally being replaced by digital technology, the logic of the filmmaking process is being redefined. What I describe below are the new principles of digital filmmaking, which are equally valid for individual or collective film productions, regardless of whether they are using the most expensive professional hardware and software or amateur equivalents. Consider, then, the following principles of digital filmmaking:
- Rather than filming physical reality it is now possible to generate film-like scenes directly in a computer with the help of 3-D computer animation. Therefore, live-action footage is displaced from its role as the only possible material from which the finished film is constructed.
- Once live-action footage is digitized (or directly recorded in a digital format), it loses its privileged indexical relationship to pro-filmic reality. The computer does not distinguish between an image obtained through the photographic lens, an image created in a paint program, or an image synthesized in a 3-D graphics package, since they are made from the same material—pixels. And pixels, regardless of their origin, can be easily altered, substituted one for another, and so on. Live-action footage is reduced to just another graphic, no different from images that were created manually.[9]
- If live-action footage was left intact in traditional filmmaking, now it functions as raw material for further compositing, animating, and morphing. As a result, while retaining visual realism unique to the photographic process, film obtains the plasticity that was previously only possible in painting or animation. To use the suggestive title of a popular morphing software, digital filmmakers work with “elastic reality.” For example, the opening shot of Forrest Gump (Robert Zemeckis 1994; special effects by Industrial Light and Magic) tracks an unusually long and extremely intricate flight of a feather. To create the shot, the real feather was filmed against a blue background in different positions; this material was then animated and composited against shots of a landscape.[10] The result: a new kind of realism, which can be described as “something that looks as if it is intended to look exactly as if it could have happened, although it really could not.”
- Previously, editing and special effects were strictly separate activities. An editor worked on ordering sequences of images together; any intervention within an image was handled by special-effects specialists. The computer collapses this distinction. The manipulation of individual images via a paint program or algorithmic image processing becomes as easy as arranging sequences of images in time. Both simply involve “cut and paste.” As this basic computer command exemplifies, modification of digital images (or other digitized data) is not sensitive to distinctions of time and space or of differences of scale. Thus, re-ordering sequences of images in time, compositing them together in space, modifying parts of an individual image, and changing individual pixels become the same operation, conceptually and practically.
- Given the preceding principles, we can define digital film in this way:
digital film = live action material + painting + image processing + compositing + 2-D computer animation + 3-D computer animation
Live-action material can be recorded either on film or video or directly in a digital format.[11] Painting, image processing, and computer animation refer to the processes of modifying already existent images as well as creating new ones. In fact, the very distinction between creation and modification, so clear in film-based media (shooting versus darkroom processes in photography, production versus post-production in cinema) no longer applies to digital cinema, since each image, regardless of its origin, goes through a number of programs before making it to the final film.[12]
Let us summarize the principles discussed thus far. Live action footage is now only raw material to be manipulated by hand: animated, combined with 3-D computer-generated scenes, and painted over. The final images are constructed manually from different elements, and all the elements are either created entirely from scratch or modified by hand.
We can finally answer the question “What is digital cinema?” Digital cinema is a particular case of animation that uses live-action footage as one of its many elements.
This can be re-read in view of the history of the moving image sketched earlier. Manual construction and animation of images gave birth to cinema and slipped into the margins, only to re-appear as the foundation of digital cinema. The history of the moving image thus comes full circle. Born from animation, cinema pushed animation to its boundary, only to become one particular case of animation in the end.
The relationship between “normal” filmmaking and special effects is similarly reversed. Special effects, which involved human intervention into machine-recorded footage and which were therefore delegated to cinema’s periphery throughout its history, become the norm of digital filmmaking.
The same applies for the relationship between production and post-production. Cinema traditionally involved arranging physical reality to be filmed though the use of sets, models, art direction, cinematography, etc. Occasional manipulation of recorded film (for instance, through optical printing) was negligible compared to the extensive manipulation of reality in front of a camera. In digital filmmaking, shot footage is no longer the final point but just raw material to be manipulated in a computer where the real construction of a scene will take place. In short, the production becomes just the first stage of post-production.
The following examples illustrate this shift from re-arranging reality to re-arranging its images. From the analog era: for a scene in Zabriskie Point (1970), Michelangelo Antonioni, trying to achieve a particularly saturated color, ordered a field of grass to be painted. From the digital era: to create the launch sequence in Apollo 13 (Ron Howard 1995; special effects by Digital Domain), the crew shot footage at the original location of the launch at Cape Canaveral. The artists at Digital Domain scanned the film and altered it on computer workstations, removing recent building construction, adding grass to the launch pad and painting the skies to make them more dramatic. This altered film was then mapped onto 3-D planes to create a virtual set that was animated to match a 180-degree dolly movement of a camera following a rising rocket (see Robertson 20).
The last example brings us to yet another conceptualization of digital cinema—as painting. In his book-length study of digital photography, William J.T. Mitchell focuses our attention on what he calls the inherent mutability of a digital image:
The essential characteristic of digital information is that it can be manipulated easily and very rapidly by computer. It is simply a matter of substituting new digits for old. . . . Computational tools for transforming, combining, altering, and analyzing images are as essential to the digital artist as brushes and pigments to a painter. (7)
As Mitchell points out, this inherent mutability erases the difference between a photograph and a painting. Since a film is a series of photographs, it is appropriate to extend Mitchell’s argument to digital film. With an artist being able to easily manipulate digitized footage either as a whole or frame by frame, a film in a general sense becomes a series of paintings.[13]
Hand-painting digitized film frames, made possible by a computer, is probably the most dramatic example of the new status of cinema. No longer strictly locked in the photographic, it opens itself towards the painterly. It is also the most obvious example of the return of cinema to its 19th-century origins—in this case, to hand-crafted images of magic lantern slides, the Phenakistiscope, the Zoetrope.
We usually think of computerization as automation, but here the result is the reverse: what was previously automatically recorded by a camera now has to be painted one frame at a time. But not just a dozen images, as in the 19th century, but thousands and thousands. We can draw another parallel with the practice, common in the early days of silent cinema, of manually tinting film frames in different colors according to a scene’s mood (see Robinson 165). Today, some of the most visually sophisticated digital effects are often achieved using the same simple method: painstakingly altering by hand thousands of frames. The frames are painted over either to create mattes (hand-drawn matte extraction) or to directly change the images, as in Forrest Gump, where President Kennedy was made to speak new sentences by altering the shape of his lips, one frame at a time.[14] In principle, given enough time and money, one can create what will be the ultimate digital film: 90 minutes, i.e. 129,600 frames, completely painted by hand from scratch, but indistinguishable in appearance from live photography.[15]
Multimedia as “Primitive” Digital Cinema
3-D animation, compositing, mapping, paint retouching: in commercial cinema, these radical new techniques are mostly used to solve technical problems while traditional cinematic language is preserved unchanged. Frames are hand-painted to remove wires that supported an actor during shooting; a flock of birds is added to a landscape; a city street is filled with crowds of simulated extras. Although most Hollywood releases now involve digitally manipulated scenes, the use of computers is always carefully hidden.[16]
Commercial narrative cinema still continues to hold on to the classical realist style where images function as unretouched photographic records of events that took place in front of the camera.[17] Cinema refuses to give up its unique cinema effect, an effect which, according to Metz’s penetrating analysis made in the 1970s, depends upon narrative form, the reality effect, and cinema’s architectural arrangement all working together.
Towards the end of his essay, Metz wonders whether in the future non-narrative films may become more numerous; if this happens, he suggests that cinema will no longer need to manufacture its reality effect. Electronic and digital media have already brought about this transformation. Beginning in the 1980s, new cinematic forms have emerged that are not linear narratives, that are exhibited on a television or a computer screen, rather than in a movie theater—and that simultaneously give up cinematic realism.
What are these forms? First of all, there is the music video. Probably not by accident, the genre of music video came into existence exactly at the time when electronic video effects devices were entering editing studios. Importantly, just as music videos often incorporate narratives within them, but are not linear narratives from start to finish, they rely on film (or video) images, but change them beyond the norms of traditional cinematic realism. The manipulation of images through hand-painting and image processing, hidden in Hollywood cinema, is brought into the open on a television screen. Similarly, the construction of an image from heterogeneous sources is not subordinated to the goal of photorealism but functions as an aesthetic strategy. The genre of music video has been a laboratory for exploring numerous new possibilities of manipulating photographic images made possible by computers—the numerous points which exist in the space between the 2-D and the 3-D, cinematography and painting, photographic realism and collage. In short, it is a living and constantly expanding textbook for digital cinema (see Shaviro, “Splitting the Atom” in this volume).
A detailed analysis of the evolution of music video imagery (or, more generally, broadcast graphics in the electronic age) deserves a separate treatment, and I will not try to take it up here. Instead, I will discuss another new cinematic non-narrative form, CD-ROM games, which, in contrast to music video, relied on the computer for storage and distribution from the very beginning. And, unlike music video designers who were consciously pushing traditional film or video images into something new, the designers of CD-ROMs arrived at a new visual language unintentionally while attempting to emulate traditional cinema.
In the late 1980s, Apple began to promote the concept of computer multimedia; and in 1991 it released QuickTime software to enable an ordinary personal computer to play movies. However, for the next few years the computer did not perform its new role very well. First, CD-ROMs could not hold anything close to the length of a standard theatrical film. Secondly, the computer would not smoothly play a movie larger than the size of a stamp. Finally, the movies had to be compressed, degrading their visual appearance. Only in the case of still images was the computer able to display photographic-type detail at full screen size.
Because of these particular hardware limitations, the designers of CD-ROMs had to invent a different kind of cinematic language in which a range of strategies, such as discrete motion, loops, and superimposition, previously used in 19th-century moving-image presentations, in 20th-century animation, and in the avant-garde tradition of graphic cinema, were applied to photographic or synthetic images. This language synthesized cinematic illusionism and the aesthetics of graphic collage, with its characteristic heterogeneity and discontinuity. The photographic and the graphic, divorced when cinema and animation went their separate ways, met again on a computer screen.
The graphic also met the cinematic. The designers of CD-ROMs were aware of the techniques of 20th-century cinematography and film editing, but they had to adopt these techniques both to an interactive format and to hardware limitations. As a result, the techniques of modern cinema and of 19th-century moving images have merged in a new hybrid language.
We can trace the development of this language by analyzing a few well-known CD-ROM titles. The bestselling game Myst (Broderbund, 1993) unfolds its narrative strictly through still images, a practice which takes us back to magic lantern shows (and to Chris Marker’s La Jetée).[18] But in other ways Myst relies on the techniques of 20th-century cinema. For instance, the CD-ROM uses simulated camera turns to switch from one image to the next. It also employs the basic technique of film editing to subjectively speed up or slow down time. In the course of the game, the user moves around a fictional island by clicking on a mouse. Each click advances a virtual camera forward, revealing a new view of a 3-D environment. When the user begins to descend into the underground chambers, the spatial distance between the points of view of each two consecutive views decreases sharply. If earlier the user was able to cross a whole island with just a few clicks, now it takes a dozen clicks to get to the bottom of the stairs! In other words, just as in traditional cinema, Myst slows down time to create suspense and tension.
In Myst, miniature animations are sometimes embedded within the still images. In the next bestselling CD-ROM 7th Guest (Virgin Games, 1993), the user is presented with video clips of live actors superimposed over static backgrounds created with 3-D computer graphics. The clips are looped, and the moving human figures clearly stand out against the backgrounds. Both of these features connect the visual language of 7th Guest to 19th-century pre-cinematic devices and 20th-century cartoons rather than to cinematic verisimilitude. But like Myst, 7th Guest also evokes distinctly modern cinematic codes. The environment where all action takes place (an interior of a house) is rendered using a wide-angle lens; to move from one view to the next, a camera follows a complex curve, as though mounted on a virtual dolly.
Next, consider the CD-ROM Johnny Mnemonic (Sony Imagesoft, 1995). Produced to complement the fiction film of the same title, marketed not as a “game” but as an “interactive movie,” and featuring full-screen video throughout, it comes closer to cinematic realism than the previous CD-ROMs—yet it is still quite distinct from it. With all action shot against a green screen and then composited with graphic backgrounds, its visual style exists within a space between cinema and collage.
It would not be entirely inappropriate to read this short history of the digital moving image as a teleological development which replays the emergence of cinema a hundred years earlier. Indeed, as computers’ speed keeps increasing, the CD-ROM designers have been able to go from a slide show format to the superimposition of small moving elements over static backgrounds and finally to full-frame moving images. This evolution repeats the 19th-century progression: from sequences of still images (magic lantern slide presentations) to moving characters over static backgrounds (for instance, in Reynaud’s Praxinoscope Theater) to full motion (the Lumières’ cinematograph). Moreover, the introduction of QuickTime in 1991 can be compared to the introduction of the Kinetoscope in 1892: both were used to present short loops, both featured the images approximately two by three inches in size, both called for private viewing rather than collective exhibition. Finally, the Lumières’ first film screenings of 1895, which shocked their audiences with huge moving images, found their parallel in CD-ROM titles of 1995, where the moving image finally fills the entire computer screen. Thus, exactly a hundred years after cinema was officially “born,” it was reinvented on a computer screen.
But this is only one reading. We no longer think of the history of cinema as a linear march towards only one possible language, or as a progression towards more and more accurate verisimilitude. Rather, we have come to see its history as a succession of distinct and equally expressive languages, each with its own aesthetic variables, each new language closing off some of the possibilities of the previous one—a cultural logic not dissimilar to Kuhn’s analysis of scientific paradigms. Similarly, instead of dismissing visual strategies of early multimedia titles as a result of technological limitations, we may want to think of them as an alternative to traditional cinematic illusionism, as a beginning of digital cinema’s new language.
For the computer/entertainment industry, these strategies represent only a temporary limitation, an annoying drawback that needs to be overcome. This is one important difference between the situation at the end of the nineteenth and the end of the twentieth centuries: if cinema was developing towards the still open horizon of many possibilities, the development of commercial multimedia, and of corresponding computer hardware (compression boards, storage formats such as Digital Video Disk), is driven by a clearly defined goal: the exact duplication of cinematic realism. So if a computer screen, more and more, emulates a cinema screen, this not an accident but a result of conscious planning.
The Loop and Spatial Montage
A number of artists, however, have approached these strategies not as limitations but as a source of new cinematic possibilities. As an example, I will discuss the use of the loop and of montage in Jean-Louis Boissier’s Flora petrinsularis (1993) and in my own Little Movies (1994-).[19]
As already mentioned, all 19th-century pre-cinematic devices, up to Edison’s Kinetoscope, were based on short loops. As “the seventh art” began to mature, it banished the loop to the low-art realms of the instructional film, the pornographic peep-show, and the animated cartoon. In contrast, narrative cinema has avoided repetitions; like modern Western fictional forms in general, it put forward a notion of human existence as a linear progression through numerous unique events.
Cinema’s birth from a loop form was reenacted at least once during its history. In one of the sequences of the revolutionary Soviet montage film, A Man with a Movie Camera (1929), Dziga Vertov shows us a cameraman standing in the back of a moving automobile. As he is being carried forward by an automobile, he cranks the handle of his camera. A loop, a repetition, created by the circular movement of the handle, gives birth to a progression of events—a very basic narrative which is also quintessentially modern: a camera moving through space recording whatever is in its way. In what seems to be a reference to cinema’s primal scene, these shots are intercut with the shots of a moving train. Vertov even re-stages the terror which the Lumières’ film supposedly provoked in its audience; he positions his camera right along the train track so the train runs over our point of view a number of times, crushing us again and again.
Early digital movies share the same limitations of storage as 19th-century pre-cinematic devices. This is probably why the loop playback function was built into the QuickTime interface, thus giving it the same weight as the VCR-style “play forward” function. So, in contrast to films and videotapes, QuickTime movies are supposed to be played forward, backward, or looped.
Can the loop be a new narrative form appropriate for the computer age? It is relevant to recall that the loop gave birth not only to cinema but also to computer programming. Programming involves altering the linear flow of data through control structures, such as “if/then” and “repeat/while”; the loop is the most elementary of these control structures. If we strip the computer from its usual interface and follow the execution of a typical computer program, the computer will reveal itself to be another version of Ford’s factory, with a loop as its conveyer belt.
Flora petrinsularis realizes some of the possibilities contained in the loop form, suggesting a new temporal aesthetics for digital cinema. The CD-ROM, which is based on Rousseau’s Confessions, opens with a white screen, containing a numbered list. Clicking on each item leads us to a screen containing two frames, positioned side by side. Both frames show the same video loop but are slightly offset from each other in time. Thus, the images appearing in the left frame reappear in a moment on the right and vice versa, as though an invisible wave is running through the screen. This wave soon becomes materialized: when we click on one of the frames we are taken to a new screen showing a loop of a rhythmically vibrating water surface. As each mouse click reveals another loop, the viewer becomes an editor, but not in a traditional sense. Rather than constructing a singular narrative sequence and discarding material which is not used, here the viewer brings to the forefront, one by one, numerous layers of looped actions which seem to be taking place all at once, a multitude of separate but co-existing temporalities. The viewer is not cutting but re-shuffling. In a reversal of Vertov’s sequence where a loop generated a narrative, the viewer’s attempt to create a story in Flora petrinsularis leads to a loop.
The loop that structures Flora petrinsularis on a number of levels becomes a metaphor for human desire that can never achieve resolution. It can be also read as a comment on cinematic realism. What are the minimal conditions necessary to create the impression of reality? As Boissier demonstrates, in the case of a field of grass, a close-up of a plant or a stream, just a few looped frames become sufficient to produce the illusion of life and of linear time.
Stephen Neale describes how early film demonstrated its authenticity by representing moving nature: “What was lacking [in photographs] was the wind, the very index of real, natural movement. Hence the obsessive contemporary fascination, not just with movement, not just with scale, but also with waves and sea spray, with smoke and spray” (52). What for early cinema was its biggest pride and achievement—a faithful documentation of nature’s movement—becomes for Boissier a subject of ironic and melancholic simulation. As the few frames are looped over and over, we see blades of grass shifting slightly back and forth, rhythmically responding to the blow of non-existent wind that is almost approximated by the noise of a computer reading data from a CD-ROM.
Something else is being simulated here as well, perhaps unintentionally. As you watch the CD-ROM, the computer periodically staggers, unable to maintain a consistent data rate. As a result, the images on the screen move in uneven bursts, slowing and speeding up with human-like irregularity. It is as though they are brought to life not by a digital machine but by a human operator, cranking the handle of the Zoetrope a century and a half ago.
Little Movies is my own project about the aesthetics of digital cinema, and a eulogy to its earliest form—QuickTime. Beginning with the well-known supposition that every new medium relies on the content of previous media, Little Movies features key moments in the history of cinema as its logical subject.
As the time passes, the medium becomes the message, that is, the “look” more than the content of any media technology of the past is what lingers on. Little Movies reads digital media of the 1990s from a hypothetical future, foregrounding its basic properties: the pixel, the computer screen, the scanlines. As described earlier, in the early 1890s the public patronized Kinetoscope parlors where peep-hole machines presented them with the latest marvel—tiny moving photographs arranged in short loops. And exactly a hundred years later, we are equally fascinated with tiny QuickTime movies—the precursor of digital cinema still to come. Drawing a parallel between these two historical moments, Little Movies are explicitly modeled after Kinetoscope films: they are also short loops.
Like Boissier, I am also interested in exploring alternatives to cinematic montage, in my case replacing its traditional sequential mode with a spatial one. Ford’s assembly line relied on the separation of the production process into a set of repetitive, sequential, and simple activities. The same principle made computer programming possible: a computer program breaks a task into a series of elemental operations to be executed one at a time. Cinema followed this principle as well: it replaced all other modes of narration with a sequential narrative, an assembly line of shots that appear on the screen one at a time. A sequential narrative turned out to be particularly incompatible with a spatialized narrative, which played a prominent role in European visual culture for centuries. From Giotto’s fresco cycle at Cappella degli Scrovegni in Padua to Courbet’s A Burial at Ornans, artists presented a multitude of separate events (which were sometimes even separated by time) within a single composition. In contrast to cinema’s narratives, here all the “shots” were accessible to a viewer at one.
Cinema has elaborated complex techniques of montage between different images replacing each other in time; but the possibility of what can be called “spatial montage” between simultaneously co-existing images was not explored. In Little Movies I begin to explore this direction in order to open up again the tradition of spatialized narrative suppressed by cinema. In one of the movies I develop the narrative through a number of short video clips, all much smaller in size than the computer screen. This allows me to place a number of clips on the screen at once. Sometimes all the clips are paused, and only one clip is playing; at other times two or three different clips play at once. As the narrative activates different parts of the screen, montage in time gives way to montage in space. Or rather, we can say that montage acquires a new spatial dimension. In addition to montage dimensions already explored by cinema (differences in images’ content, composition, movement) we now have a new dimension: the position of the images in space in relation to each other. In addition, since images do not replace each other (as in cinema) but remain on the screen throughout the movie, each new image is juxtaposed not just with one image which preceded it, but with all the other images present on the screen.
The logic of replacement, characteristic of cinema, gives way to the logic of addition and co-existence. Time becomes spatialized, distributed over the surface of the screen. Nothing is forgotten, nothing is erased. Just as we use computers to accumulate endless texts, messages, notes, and data (and just as a person, going through life, accumulates more and more memories, with the past slowly acquiring more weight than the future), “spatial montage” accumulates events and images as it progresses through its narrative. In contrast to cinema’s screen, which primarily functioned as a record of perception, here the computer screen functions as a record of memory.
By making images different in size and by having them appear and disappear in different parts of the screen without any obvious order, I want to present the computer screen as a space of endless possibilities. Rather than being a surface that passively accepts projected images of reality recorded by a camera, the computer screen becomes an active generator of moving-image events. It already contains numerous images and numerous narrative paths; all that remains is to reveal some of them.
Conclusion: From Kino-Eye to Kino-Brush
In the 20th century, cinema has played two roles at once. As a media technology, cinema’s role was to capture and to store visible reality. The difficulty of modifying images once they were recorded was exactly what gave cinema its value as a document, assuring its authenticity. The same rigidity of the film image has defined the limits of cinema as I defined it earlier, i.e. the super-genre of live action narrative. Although it includes within itself a variety of styles—the result of the efforts of many directors, designers, and cinematographers—these styles share a strong family resemblance. They are all children of the recording process that uses lenses, regular sampling of time, and photographic media. They are all children of a machine vision.
The mutability of digital data impairs the value of cinematic recordings as documents of reality. In retrospect, we can see that 20th-century cinema’s regime of visual realism, the result of automatically recording visual reality, was only an exception, an isolated accident in the history of visual representation which has always involved, and now again involves, the manual construction of images. Cinema becomes a particular branch of painting—painting in time. No longer a kino-eye, but a kino-brush.[20]
The privileged role played by the manual construction of images in digital cinema is one example of a larger trend: the return of pre-cinematic moving images techniques. Marginalized by the 20th-century institution of live-action narrative cinema that relegated them to the realms of animation and special effects, these techniques reemerge as the foundation of digital filmmaking. What was supplemental to cinema becomes its norm; what was at its boundaries comes into the center. Digital media return to us the repressed of the cinema.
As the examples discussed in this essay suggest, the directions that were closed off at the turn of the century, when cinema came to dominate the modern moving-image culture, are now again beginning to be explored. Moving-image culture is being redefined once again; cinematic realism is being displaced from being its dominant mode to become only one option among many.
Works Cited
Artintact 1. Karlsruhe: ZKM/Center for Art and Media, 1994. CD-ROM.
Aumont, Jacques, Alain Bergala, Michel Marie, and Marc Vernet. Aesthetics of Film. Trans. Richard Neupert. Austin: U of Texas P, 1992. Print.
Billups, Scott. “Casting from Forest Lawn (Future of Performers).” Annual Artists Rights Digital Technology Symposium. Directors Guild of America, Los Angeles. 16 Feb. 1996. Address.
Bordwell, David, and Kristin Thompson. Film Art: An Introduction. 4th ed. New York: McGraw-Hill, 1993. Print.
Bukatman, Scott. “The Artificial Infinite: On Special Effects and the Sublime.” Visual Display: Culture Beyond Appearances. Eds. Lynne Cooke and Peter Wollen. Seattle: Bay P, 1995. 255-89. Print.
Ceram, C.W. Archaeology of the Cinema. New York: Harcourt, 1965. Print.
Huhtamo, Erkki. “Encapsulated Bodies in Motion: Simulators and the Quest for Total Immersion.” Critical Issues in Electronic Media. Ed. Simon Penny. Albany: SUNY P, 1995. 159-86. Print.
Kuhn, Thomas S. The Structure of Scientific Revolutions. 2nd. ed. Chicago: U of Chicago P, 1970. Print.
Leish, Kenneth W. Cinema. New York: Newsweek Books, 1974. Print.
Lunenfeld, Peter. “Art Post-History: Digital Photography and Electronic Semiotics.” Photography after Photography: Memory and Representation in the Digital Age. Eds. Hubertus von Amelunxen, Stefan Inglhaut, and Florian Rötzer. Sydney: G+B Arts, 1996. Print.
Manovich, Lev. “Cinema and Digital Media.” Perspektiven der Medienkunst/Media Art Perspectives. Eds. Jeffrey Shaw and Hans Peter Schwarz. Ostfildern: Cantz, 1996. Print.
—. “Engineering Vision: from Constructivism to the Computer.” Diss. U of Rochester, 1993. Web. <https://www.academia.edu/2800644/The_Engineering_of_Vision_from_Constructivism_to_Computers>.
—. “To Lie and to Act: Potemkin’s Villages, Cinema and Telepresence.” Mythos Information—Welcome to the Wired World: Ars Electronica 95. Eds. Karl Gebel and Peter Weibel. Vienna: Springer, 1995. 343-48. Print.
—. “What is Digital Cinema?” Telepolis, 21 May 1996. Web. <http://www.heise.de/tp/artikel/6/6110/1.html>. Rpt. in The Digital Dialectic: New Essays on New Media. Ed. Peter Lunenfeld. Cambridge: MIT P, 2000. 173-92. Rpt. in Visual Culture: Critical Concepts in Media and Cultural Studies. Eds. Joanne Morra and Marquard Smith. New York: Routledge, 2006. 221-38.
Marker, Chris. La Jetée: Ciné-roman. New York: Zone, 1992. Print.
Metz, Christian. “The Fiction Film and its Spectator: A Metapsychological Study.” Apparatus: Cinematographic Apparatus—Selected Writings. Ed. Theresa Hak Kyung Cha. New York: Tanam P, 1980. 373-409. Print.
Mitchell, William J. T. The Reconfigured Eye: Visual Truth in the Post-Photographic Era. Cambridge: MIT P, 1992. Print.
Monaco, James. How to Read a Film. Rev. ed. New York: Oxford UP, 1981. Print.
Musser, Charles. The Emergence of Cinema: The American Screen to 1907. Berkeley: U of California P, 1990. Print.
Neale, Stephen. Cinema and Technology. Bloomington: Indiana UP, 1985. Print.
Perisi, Paula. “The New Hollywood Silicon Stars.” Wired Dec. 1995: 142+. Print.
Petrovich, Lucy, et al., eds. SIGGRAPH ‘94 Visual Proceedings. New York: ACM SIGGRAPH, 1994. Print.
Robertson, Barbara. “Digital Magic: Apollo 13.” Computer Graphics World Aug. 1995: 20. Print.
Robinson, David. From Peep Show to Palace: the Birth of American Film. New York: Columbia UP, 1996. Print.
Russett, Robert, and Cecile Starr. Experimental Animation. New York: Van Nostrand Reinhold, 1976. Print.
Samuelson, David. Motion Picture Camera Techniques. London: Focal P, 1978. Print.
Sitney, P. Adams. Visionary Film: The American Avant-Garde, 1943-1978. 2nd ed. Oxford: Oxford UP, 1979. Print.
Sobchack, Vivian. Screening Space: The American Science Fiction Film, 2nd ed. New York: Ungar, 1987. Print.
Vertov, Dziga. “Artistic Drama and Kino-Eye.” 1924. Kino-Eye: The Writings of Dziga Vertov. Ed. Annette Michelson. Trans. Kevin O’Brien. Berkeley: U of California P, 1984. 47-49. Print.
Notes
This essay was first published in 1996, in the German online magazine Telepolis, and has been reprinted, in modified form, in two book publications (see works cited). The version reprinted here, however, has appeared only on Manovich’s website.
[1] This is the third in a series of essays on digital cinema. See my “Cinema and Digital Media” and “To Lie and to Act: Potemkin’s Villages, Cinema and Telepresence.” This essay has greatly benefited from the suggestions and criticisms of Natalie Bookchin, Peter Lunenfeld, Norman Klein, and Vivian Sobchack. I also would like to acknowledge the pioneering work of Erkki Huhtamo on the connections between early cinema and digital media which stimulated my own interest in this topic. See, for instance, his “Encapsulated Bodies in Motion.”
[2] The remarks were made by Scott Billups, a major figure in bringing Hollywood and Silicon Valley together by way of the American Film Institute’s Apple Laboratory and Advanced Technologies Programs in the late 1980s and early 1990s (see Billups; also Perisi).
[3] Cinema as defined by its “super-genre” of fictional live action film belongs to media arts, which, in contrast to traditional arts, rely on recordings of reality as their basis. Another term which is not as popular as “media arts” but is perhaps more precise is “recording arts.” For the use of this term, see Monaco 7.
[4] The birth of cinema in the 1890s is accompanied by an interesting transformation: while the body as the generator of moving pictures disappears, it simultaneously becomes their new subject. Indeed, one of the key themes of early films produced by Edison is a human body in motion: a man sneezing, the famous bodybuilder Sandow flexing his muscles, an athlete performing somersaults, a woman dancing. Films of boxing matches play a key role in the commercial development of the Kinetoscope. See Musser 72-79; Robinson 44-48.
[5] This arrangement was previously used in magic lantern projections; it is described in the second edition of Althanasius Kircher’s Ars magna (1671). See Musser 21-22.
[6] The extent of this lie is made clear by the films of Andy Warhol from the first part of the 1960s—perhaps the only real attempt to create cinema without a language.
[7] I have borrowed this definition of special effects from Samuelson.
[8] The following examples illustrate this disavowal of special effects; other examples can be easily found. The first example is from popular discourse on cinema. A section entitled “Making the Movies” in Leish’s Cinema contains short stories from the history of the movie industry. The heroes of these stories are actors, directors, and producers; special effects artists are mentioned only once. The second example is from an academic source: the authors of the authoritative Aesthetics of Film state that “[t]he goal of our book is to summarize from a synthetic and didactic perspective the diverse theoretical attempts at examining these empirical notions [terms from the lexicon of film technicians], including ideas like frame vs. shot, terms from production crews’ vocabularies, the notion of identification produced by critical vocabulary, etc.” (Aumont et al. 7). The fact that the text never mentions special effects techniques reflects the general lack of any historical or theoretical interest in the topic by film scholars. Bordwell and Thompson’s Film Art: An Introduction, which is used as a standard textbook in undergraduate film classes is a little better as it devotes three of its five hundred pages to special effects. Finally, a relevant piece of statistics: a library at the University of California, San Diego contains 4273 titles catalogued under the subject “motion pictures” and only 16 tiles under “special effects cinematography.” For the few important works addressing the larger cultural significance of special effects by film theoreticians see Sobchack; Bukatman. Norman Klein is currently working on a history of special effects environments.
[9] For a discussion of the subsumption of the photographic to the graphic, see Lunenfeld.
[10] For a complete list of people at ILM who worked on this film, see the SIGGRAPH ‘94 Visual Proceedings (Petrovich et al. 19).
[11] In this respect 1995 can be called the last year of digital media. At the 1995 National Association of Broadcasters convention Avid showed a working model of a digital video camera which records not on a video cassette but directly onto a hard drive. Once digital cameras become widely used, we will no longer have any reason to talk about digital media since the process of digitization will be eliminated.
[12] Here is another, even more radical definition: digital film = f (x, y, t). This definition would be greeted with joy by the proponents of abstract animation. Since the computer breaks down every frame into pixels, a complete film can be defined as a function which, given the horizontal, vertical, and time location of each pixel, returns its color. This is actually how a computer represents a film, a representation which has a surprising affinity with a certain well-known avant-garde vision of cinema! For a computer, a film is an abstract arrangement of colors changing in time, rather than something structured by “shots,” “narrative,” “actors,” and so on.
[13] The full advantage of mapping time into 2-D space, already present in Edison’s first cinema apparatus, is now realized: one can modify events in time by literally painting on a sequence of frames, treating them as a single image.
[14] See “Industrial Light & Magic alters history with MATADOR,” promotional material by Parallax Software, SIGGRAPH ‘95 Conference, Los Angeles, August 1995.
[15] The reader who followed my analysis of the new possibilities of digital cinema may wonder why I have stressed the parallels between digital cinema and the pre-cinematic techniques of the 19th century but did not mention 20th-century avant-garde filmmaking. Did not the avant-garde filmmakers already explore many of these new possibilities? To take the notion of cinema as painting, Len Lye, one of the pioneers of abstract animation, was painting directly on film as early as 1935; he was followed by Norman McLaren and Stan Brakhage, the latter extensively covering shot footage with dots, scratches, splattered paint, smears, and lines in an attempt to turn his films into equivalents of Abstract Expressionist painting. More generally, one of the major impulses in all of avant-garde filmmaking, from Léger to Godard, was to combine the cinematic, the painterly, and the graphic—by using live-action footage and animation within one film or even a single frame, by altering this footage in a variety of ways, or by juxtaposing printed texts and filmed images.
I explore the notion that the avant-garde anticipated digital aesthetics in my “Engineering Vision: from Constructivism to the Computer”; here I would like to bring up one point particularly relevant for this essay. When the avant-garde filmmakers collaged multiple images within a single frame, or painted and scratched film, or revolted against the indexical identity of cinema in other ways, they were working against “normal” filmmaking procedures and the intended uses of film technology. (Film stock was not designed to be painted on.) Thus they operated on the periphery of commercial cinema not only aesthetically but also technically.
One general effect of the digital revolution is that avant-garde aesthetic strategies became embedded in the commands and interface metaphors of computer software. In short, the avant-garde became materialized in a computer. Digital cinema technology is a case in point. The avant-garde strategy of collage reemerged as a “cut and paste” command, the most basic operation one can perform on digital data. The idea of painting on film became embedded in paint functions of film-editing software. The avant-garde move to combine animation, printed texts, and live-action footage is repeated in the convergence of animation, title generation, paint, compositing, and editing systems into single all-in-one packages. Finally, another move to combine a number of film images together within one frame (for instance, in Léger’s 1924 Ballet Mécanique or in Vertov’s 1929 A Man with a Movie Camera) also become legitimized by technology, since all editing software, including Photoshop, Premiere, After Effects, Flame, and Cineon, by default assumes that a digital image consists of a number of separate image layers. All in all, what used to be exceptions for traditional cinema became the normal, intended techniques of digital filmmaking, embedded in technology design itself. For the experiments in painting on film by Lye, McLaren, and Brakhage, see Russett and Starr 65-71; 117-128; also Sitney 230; 136-227.
[16] Reporting in the December 1995 issue of Wired, Paula Perisi writes: “A decade ago, only an intrepid few, led by George Lucas’s Industrial Light and Magic, were doing high-quality digital work. Now computer imaging is considered an indispensable production tool for all films, from the smallest drama to the largest visual extravaganza” (144).
[17] Therefore, one way in which the fantastic is justified in contemporary Hollywood cinema is through the introduction of various non-human characters such as aliens, mutants, and robots. We never notice the pure arbitrariness of their colorful and mutating bodies, the beams of energy emulating from their eyes, the whirlpools of particles emulating from their wings, because they are made perceptually consistent with the set, i.e. they look like something which could have existed in a three-dimensional space and therefore could have been photographed.
[18] This 28-minute film, made in 1962, is composed of still frames narrativized in time, and concludes with a very short live action sequence. For documentation, see Marker.
[19] Flora petrinsularis is included in the compilation CD-ROM Artintact 1. Little Movies are available online at <http://jupiter.ucsd.edu/~manovich/little-movies>.
[20] It was Dziga Vertov who coined the term “kino-eye” in the 1920s to describe the cinematic apparatus’s ability “to record and organize the individual characteristics of life’s phenomena into a whole, an essence, a conclusion” (47). For Vertov, the presentation of film “facts,” based as they were on materialist evidence, defined the very nature of the cinema.
Lev Manovich is the author of seven books including Software Takes Command (Bloomsbury Academic, 2013), Soft Cinema: Navigating the Database (The MIT Press, 2005), and The Language of New Media (The MIT Press, 2001), which was described as “the most suggestive and broad ranging media history since Marshall McLuhan.” Manovich is a Professor at The Graduate Center, CUNY, and a Director of the Software Studies Initiative that works on the analysis and visualization of big visual cultural data. In 2013 he appeared on the list of “25 People Shaping the Future of Design.” In 2014 he was included in the list of “50 Most Interesting People Building the Future” (The Verge).
Lev Manovich, “What is Digital Cinema?” in Denson and Leyda (eds), Post-Cinema: Theorizing 21st-Century Film (Falmer: REFRAME Books, 2016). Web. <https://reframe.sussex.ac.uk/post-cinema/1-1-manovich/>. ISBN 978-0-9931996-2-2 (online)