This is the fourth and last of a series of posts on the SIGGRAPH 2011 course program; each describing several of the courses that will be presented at the conference. Links to previous posts in the series: Part 1, Part 2, Part 3.
Cinematography: The Visuals & the Story
Cinematography is the art of communicating a story via camera and lighting choices. As a game developer, I find it fascinating for several reasons. One is that it is such a well-established art; over a century old, and based upon many of the principles of still older arts such as photography and painting. The maturity of the field can be seen in the way that the practice is codified – there are clear roles in film production, everyone knows what a director of photography, first camera assistant, etc. do from film to film. The field’s most prominent professional organization, the American Society of Cinematographers, was created in 1919 and its magazine American Cinematographer has been discussing tips and tricks of the trade since 1920. It’s an interesting contrast to game development – an extremely young discipline where most of the fundamentals are still being figured out.
Another reason I’m interested in cinematography is its relevance to game visuals; the primary problem (turning three-dimensional scenes into compelling screen images that carry a narrative) is the same. While issues of camera placement may be less relevant for some game genres (e.g. first person shooters), lighting, color, and scene composition considerations are relevant for almost any game.
The third reason is that most game developers (including myself until fairly recently) are either unaware of this vast wealth of relevant knowledge, or are indifferent to it. CG animated features have made great strides by incorporating principles of live-action cinematography; not many videogames are doing the same.
For these reasons, I’m glad to see a SIGGRAPH course covering cinematographic fundamentals. The speaker, Bruce Block, has had a lot of experience working in film (albeit not in the camera department) and has written a well-regarded and influential book (The Visual Story) about how visual structure is used to present story in film.
Storytelling With Color
The way in which color choices are applied throughout production is another area where I think games have a lot to learn from film. In film, the colors of almost every costume and piece of set decoration are part of a conscious choice to drive the narrative, establish a mood, or support character development. This was brought home to me last year when I visited Pixar and saw the “color script” for Toy Story 3 – a wall covered by postcard sized sketches, one for each shot in the film. Each rough sketch blocked out the shapes and colors in the shot, and when they were put together, you could clearly see how the carefully chosen color palette helped drive the story and emotional tone of the movie. Two of the Toy Story 3 color script images can be seen here, and the entire color script for a different Pixar film (Up) can be seen here.
This course will cover exactly these kinds of color choices, and will be presented by Kathy Altieri (Production Designer, Dreamworks Animation) and Dave Walvoord (Digital FX Supervisor, Dreamworks Animation). Kathy’s career in TV and film spans three decades; after working on backgrounds for multiple animated TV shows as well as classic animated feature films such as The Little Mermaid, Aladdin and The Lion King, she moved to Dreamworks, where she was Art Director on The Prince of Egypt and Production Designer on Spirit: Stallion of the Cimarron, Over the Hedge, and How to Train Your Dragon. Dave has 15 years of experience in VFX and CG feature animation, working at Blue Sky on films such as Fight Club and Ice Age before joining Dreamworks, where he was CG Supervisor on Shark Tale, Over the Hedge and Kung Fu Panda and Digital FX Supervisor on Kung Fu Panda 2.
Applying Color Theory to Digital Media and Visualization
This is another course on color, but focused more on theory and on non-entertainment applications, such as scientific visualization. The course is presented by Theresa-Marie Rhyne, a prominent visualization expert with three decades of experience as a researcher, educator, designer and artist. She has taught several courses on this topic, most recently at IEEE Visualization 2010 (a video of her slides from that talk is available online), and has a blog on the topic as well. Interestingly, she has already put up a video of the slides from the upcoming SIGGRAPH 2011 course.
Liquid Simulation With Mesh-Based Surface Tracking
While most fluid rendering and simulation work over the years has focused on level-set approaches, an important recent trend in this area consists of tracking a mesh over the surface of the fluid, thus enabling more detailed surfaces. This advanced course (prior knowledge of fluid simulation techniques is assumed) covers the current state of the art in this important area, and is presented by Chris Wojtan (Assistant Professor, Institute of Science and Technology Austria), Matthias Müller-Fischer (Research Lead, NVIDIA), and Tyson Brochu (PhD Candidate, University of British Columbia). Having performed much of the leading research in this area, the speakers are uniquely qualified to speak about the topic.
Although complex fluid simulations are used extensively in film VFX and animated features, they are currently too computationally expensive for games. As game platforms become more powerful, I believe this will change. There are already some impressive real-time demonstrations, for example the Raging Rapids Rides demo which will be shown at the SIGGRAPH 2011 Real-Time Live! program and the SIGGRAPH 2011 paper Real-Time Eulerian Water Simulation Using a Restricted Tall-Cell Grid, which has an impressive video here (check out the lighthouse part at the end). Note that one of the course speakers (Matthias) was involved with both of these examples.
Introduction to Modern OpenGL Programming
Dave Shreiner (co-author of the famous OpenGL Red Book, which has a new edition coming out this November) has taught an introductory course on OpenGL (almost) every year at SIGGRAPH since 1998. He was accompanied by various co-lecturers – most often Edward Angel – and evolved the course content to keep up with changes in the OpenGL API. The only two years Dave didn’t do this course were 2003 (when he did a “performance OpenGL” course instead of an introductory course – in some other years he did both), and 2010 (when there was no OpenGL course for the first time since 1992). Dave and Edward are back this year with an updated course, which should be of great interest to beginning graphics programmers, OpenGL programmers who have been using older versions of the API, or experienced graphics programmers with plans to start working with OpenGL.
An course on this topic couldn’t hope for better speakers. Besides his highly influential books and courses, Dave Shreiner also had an important role in the development of OpenGL (and its spinoff OpenGL ES) in the 15 years he worked at SGI (where OpenGL evolved from the proprietary IRIS GL library) and since, as Technical Advisory Panel Chair for The Khronos Group and Director of Graphics Technology at ARM. Edward Angel has taught at the University of New Mexico for over 30 years; he holds the positions of Professor Emeritus of Computer Science and Founding Director of the Art, Research, Technology and Science Laboratory (ARTS Lab). Edward has written several influential books on computer graphics, most notably the OpenGL Primer and Interactive Computer Graphics.
Modeling 3D Urban Spaces Using Procedural and Simulation-Based Techniques
As scene complexity increases, the amount of artist work (and thus the expense) required to create these scenes increases commensurately, a problem that afflicts both film and game production. Audience expectations are always increasing, and budgets cannot keep pace – more efficient ways to model large, complex scenes must be found. While most natural scenes are very complex, techniques for procedurally modeling them have been used in production for some time; see off-the-shelf products such as Vue and Speedtree, or in-house tools such as were used to model trees in Tangled. Urban scenes can be as complex, but tools for modeling them procedurally have been less widely used (the creation of 1930’s New York City in the 2005 remake of King Kong is a notable example – more details here). The last few years have seen a flourishing of research into procedural modeling of buildings and cities, and the fruits of this research are finding their way into production. This course will cover procedural as well as image-based and simulation-based modeling techniques, and is targeted at applications including computer games, movies, architecture, and urban planning.
This course will have five speakers, each extremely well-suited to teach a course of this type: Peter Wonka (Associate Professor, Arizona State University), Daniel Aliaga (Associate Professor, Purdue University), Carlos Vanegas (Research Assistant, Purdue University), Pascal Mueller (Founder & CEO, Procedural Inc.), and Michael Frederickson (Technical Director, Pixar).The first four speakers have, between them, performed or led most of the notable academic research in this area. Pascal Mueller has founded a company (Procedural Inc.) based on his research, which sells a commercial software package (CityEngine) for procedural urban modeling (Peter Wonka serves on Procedural’s advisory board). The last speaker, Michael Frederickson, was responsible for modeling the 40,000 buildings in the city of London as seen in the movie Cars 2, and it appears that this will be the topic of his presentation. Presumably (given his participation in this course, and also given the magnitude of the task) this was done procedurally. While watching Cars 2 (story issues aside) I was struck by the visuals in the film – the urban environments, especially London, in particular; I look forward to finding out how this was done.
3D Spatial Interaction: Applications for Art, Design, and Science
This course will be taught by Joseph LaViola (Assistant Professor, University of Central Florida) and Daniel Keefe (Assistant Professor, University of Minnesota). Last year at SIGGRAPH 2010, Prof. LaViola taught (with Richard Marks, the primary researcher behind Sony’s EyeToy and Move peripherals), a course about spatial interaction with videogame motion controllers. This year’s course, judging by its abstract, appears to be focused on applications other than videogames. These novel interfaces surely have interesting applications in many fields, and this course will be of interest to many. Both Prof. LaViola and Prof. Keefe have done important research in this field, and Prof. LaViola has authored a book on the subject.
Build Your Own Glasses-Free 3D Display
Last year, two of this course’s speakers, Douglas Lanman (Postdoctoral Associate, MIT Media Lab) and Matthew Hirsch (PhD Student, MIT Media Lab), taught a SIGGRAPH 2010 course called Build Your Own 3D Display. This year, they are joined by Gregg Favalora (Principal, Optics for Hire) and are focusing the course specifically on autostereoscopic displays, which do not require glasses. Douglas and Matthew have done important research into this area – most notably this SIGGRAPH Asia 2010 paper, and have taught versions of this course not only at SIGGRAPH 2010 (as mentioned), but also at SIGGRAPH Asia 2010. Gregg has 15 years experience as an entrepreneur, inventor and researcher and has authored multiple key publications and patents relating to autostereoscopic display design.
Advances in New Interfaces for Musical Expression
This course is presented by Michael Lyons (Professor, Ritsumeikan University) and Sidney Fels (Professor, University of British Columbia) who in 2001 organized the first workshop on New Interfaces for Musical Expression (NIME). This workshop, dedicated to scientific research on the development of new technologies for musical expression and artistic performance, has since blossomed into a full-fledged international conference. This course will summarize the content of the last several years of NIME, including both theory and practice, and presenting several case studies.