This is a continuation of the series of posts started here.

Character Rigging, Deformations, and Simulations in Film and Game Production

Character animation is one of those areas where film and game production have intriguing similarities as well as differences; especially in the ways that the character meshes deform in response to animation and simulation. This course includes three talks, each covering a different application domain: games, visual effects, and feature animation. These talks will be presented by:

• David Coleman (Senior CG Supervisor, Electronic Arts Canada). David (who has worked at Electronic Arts for 15 years and is currently responsible for the central team that provides rigging for many of EA’s sports titles) will present the games portion of the course. He will discuss character rigging, deformations and simulations in game production, emphasizing the technical restrictions imposed due to the real-time and interactive nature of games. This talk will also cover some strategies for setting up procedural secondary rigging systems in Maya, MotionBuilder and at run-time in games.
• Tim McLaughlin (Department Head and Associate Professor, Department of Visualization at Texas A&M University). Tim (who had 13 years of experience at ILM – most of it on digital creatures – before heading the Texas A&M Department of Visualization) will discuss rigging for visual effects. He will cover the unique requirements brought on by integration with live action but also the affordances offered by the limited range of scope of performance requirements relative to feature animation and games. Tim will discuss rigging modularity, provisions for animator control, non-linear deformations, areas of highest importance for deformations, and the efficient use of muscle systems.
• Larry Cutler (Supervising Character TD, DreamWorks Animation). Larry (who worked at Dreamworks Animation for 10 years, and at Pixar for four years previously) will be discussing rigging issues for feature animation. Larry’s talk will deal with the impact of character design, modeling, and scalability for thousands of shots on rigging, deformation, and simulation. He will discuss the issues arising from the unique needs of feature animation: accommodating for extreme range of motion, and increased emphasis on art directability and animator control. Larry will also cover hair, cloth, and facial animation systems.

Destruction and Dynamics for Film and Game Production

Another “X for film and games” course, this time focusing on rigid body dynamics and destruction / fracturing methods. The course will cover production aspects such as authoring tools and game engine integration, in addition to the computational and algorithmic aspects. Like the last course, this one will highlight interesting commonalities and differences between film and game practice. There are areas where each can learn from the other: the film techniques can point the way to future methods for games running on more powerful platforms; and the efficient game methods are useful for fast prototyping, previsualization and even speeding up final shots in film.

The course will start with a 30-minute presentation by the course organizer, Erwin Coumans (Principal Physics Engineer, AMD). Erwin has worked on physics in games for over a decade, and is also the main author of the open-source Bullet Physics Library. Although Bullet was originally designed for game use, it has been used on many films as well, including big-budget Hollywood blockbusters such as How to Train Your Dragon, Sherlock Holmes and 2012. Erwin will give an overview of the course, as well as a brief introduction to the basic theory of rigid body dynamics and destruction/fracturing methods. He will also cover collision detection and handling contacts, approximate methods for the modeling of stress and strain, and how to decide when and where to break rigid bodies into several parts. The course will continue with the following talks:

• Authoring Destruction With the Dynamica Bullet Maya Plugin (15 minutes), by Michael Baker (Faculty, Art Institute of Las Vegas): Michael has worked on Las Vegas casino games, visual effects for various short films and games, and the Bullet Physics Library (in particular the Dynamica Maya plugin which is the primary topic of his talk). Michael will discuss the development and use of Dynamica to support choreographed rigid body behavior such as progressive crumbling of pre-shattered objects, sequential structural failure and timed directional explosions.
• Destruction and Dynamics Artist Tools for Film (45 minutes), by Nafees Bin Zafar (Senior Production Engineer, Dreamworks Animation) and Mark T. Carlson (Lead Engineer, Dreamworks Animation): Nafees has worked on simulation and volumetrics tools (both at Dreamworks and in his previous job at Digital Domain) for over ten years, winning a Scientific and Engineering Academy Award in 2007. Mark has worked on cloth, fluid and crowd simulation for six years at DNA Productions, Walt Disney Animation and Dreamworks Animation. This talk will cover 3rd party software integration in the movie pipeline, building artist tools with Bullet, and authoring of destruction using Maya and Houdini. Examples from recent Dreamworks Animation movies will showcase the techniques described.
• Deformable Rigid Bodies and Fragment Clustering for Film (45 minutes), by Brice Criswell (Senior Software Engineer, Industrial Light & Magic): Brice has been developing production related software for 12 years with ILM, and specializes in rigid body and crowd dynamics. Brice’s talk is divided into three presentations. The first discusses a deformable rigid system which efficiently simulates on-impact bending and denting of normally rigid bodies. The second covers a fragment clustering system which allows artists to initialize sets of geometry as a single rigid body, then dynamically break the objects during the progression of the simulation. The third presentation covers the challenges involved in animating, simulating, and deforming the tentacle beard of the Davy Jones character in the Pirates of the Caribbean movies. Each of the talks will detail algorithms as well as production issues, and will include VFX production examples from prominent feature films.
• Procedurally Generating Fragmented Meshes for Games (15 minutes), by Phil Knight (Lead Programmer, Avalanche Software – a division of Disney Interactive Studios): Phil has 13 years of game development experience, working most recently on Cars 2, Toy Story 3, and Bolt, and previously on the Links and Amped series. His talk will cover a procedural technique for automatically generating fragmented meshes, especially useful for modeling large explosions with lots of fragmentation and debris. Besides detailing the technique itself, Phil will also describe the fragmentation tool (‘Frag’) which implements it, and its use in game production at Disney Interactive Studios.
• Accelerating Rigid Body Simulation and Fracture Using the GPU (30 minutes), by Takahiro Harada (Researcher, AMD): Takahiro Harada has performed research and development into physics simulation at The University of Tokyo and Havok as well as his current position at AMD (where he focuses on the use of GPU computing for physics simulation). He will present a GPU-based rigid body simulation which can be used to quickly simulate the large numbers of rigid bodies typically created by object destruction. The talk starts with an overview of the simulation and proceeds to the detailed GPU implementation of each stage of the simulation.

PhysBAM: Physically Based Simulation

Similarly to the previous course, this is targeted at physics simulation and has strong ties to film production. However, its structure is very different; instead of covering a variety of production examples, it focuses on one code library – PhysBAM, initially developed by Ronald Fedkiw and continued by him and many others at Stanford. PhysBAM is used by many VFX and feature animation houses including ILM, Disney Animation, and Pixar; large portions were recently released under an open-source license. This course is presented by Craig Schroeder (PhD Student, Stanford Computer Science Department); it will cover information on the PhysBAM library release: how to obtain the source code, set up the library, and use it to run example smoke and water simulations, as well as descriptions of visualization and rendering tools included in the release. In addition to the PhysBAM library, the course will explain the underlying techniques that make these simulations possible, in particular level set methods such as fast marching, fast sweeping, and the particle level set method. It will also address the important aspects of a fluid simulation, including advection, viscosity, and projection.

There are 12 courses left to cover; I’ll do so over my next few blog posts.

At 18 courses, the SIGGRAPH 2011 course program is smaller than it has been in previous years, but what it lacks in size it more than makes up in quality. I’ll go over the list with a focus on courses of interest to game developers and/or real-time rendering researchers. If you are going to be attending SIGGRAPH, this should help you decide which courses to attend – if not, you’ll at least know which course notes and Encore videos to hunt down after the conference. Since this post is turning out to be quite long, I’ll split it up into several parts, spread out over the next few days.

UPDATES:

• 6/20/2011: Added details to Beyond Programmable Shading regarding Peter-Pike Sloan’s talk and the Software Rasterization on GPUs talk, as well as correcting the titles of several of the speakers.
• 6/21/2011: Added links to the papers High-Performance Software Rasterization on GPUs and VoxelPipe: A Programmable Pipeline for 3D Voxelization (the second link is the paper webpage – no PDF yet).
• 6/24/2011: Removed an incorrect detail about the DEAA technique.
• 7/10/2011: Updated the description of the Battlefield 3 / Need for Speed: The Run talk in the Advances in Real-Time Rendering in Games course.

Advances in Real-Time Rendering in Games (Part I & Part II)

Since 2006, this course series (organized by Natalya Tatarchuk, formerly at AMD and now at Bungie) has been my favorite thing to see at SIGGRAPH. Each year it has showcased new content from the cutting edge of game and IHV graphics development. Since Natalya joined Bungie, the emphasis has been less on IHV demos and more on games, which in my opinion makes the course even better – this year looks like the best yet! Part I starts with a  brief introduction by Natalya and continues with four talks, each between 45 and 60 minutes in length:

• Bungie’s Graphics Secret Sauce, by Natalya Tatarchuk (Senior Graphics Researcher, Bungie) and Hao Chen (Engineering Lead, Bungie): Bungie’s talk will cover the graphics techniques developed for the award-winning game Halo: Reach,  along with some new research undertaken for Bungie’s next title.
• Rendering in Cars 2, by Christopher Hall, Robert Hall, and David Edwards (Programmers at Avalanche Software): this talk will describe rendering techniques used in Cars 2: The Video Game, including offloading of post-processing and stereoscopy computations onto the Playstation 3’s SPUs. Other topics covered will include new developments in color precision, post processing effects, shadows, and the use of light probes.
• Secrets of CryENGINE 3 Graphics Technology, by Tiago Sousa (Principal R&D Graphics Engineer, Crytek), Nickolay Kasyan (Senior Rendering Engineer, Crytek), and Nicolas Schulz (Graphics Engineer, Crytek): an overview of the novel deferred lighting approach used in CryENGINE 3, along with an in-depth description of optimization techniques (both general and platform-specific), as well as stereoscopic 3D rendering and shadowing techniques.
• Two Uses of Voxels in LittleBigPlanet2’s Graphics Engine Alex Evans (CTO & Co-Founder, Media Molecule) and Anton Kirczenow (Senior Programmer, Media Molecule): this talk will describe a PlayStation 3-centric implementation of real-time dynamic scene voxelization and demonstrate two ways this voxel representation was used for rendering and special effects in the game LittleBigPlanet 2.

Part II also starts with a short introduction by Natalya; this is followed by five 30-50 minute talks:

• More Performance! Five Rendering Ideas from Battlefield 3 and Need For Speed: The Run, by John White (Senior Rendering Engineer, NFS) and Colin Barré-Brisebois (Rendering Engineer, BF3): this talk will cover several techniques from Battlefield 3 and Need for Speed: The Run designed to increase performance without sacrificing visual quality. These will include chroma sub-sampling for faster full-screen effects, a novel DirectX 9+ scatter-gather approach to bokeh rendering, improved temporally-stable dynamic ambient occlusion, HiZ reverse-reload for faster shadow and tile-based deferred shading on Xbox 360 (the last topic is a good complement to Christina Coffin’s GDC 2011 presentation giving Playstation 3 implementation details).
• Physically-based Lighting in Call of Duty: Black Ops, by Dimitar Lazarov (Lead Graphics Engineer, Treyarch): Dimitar will give an overview of the lighting architecture used in the Call of Duty games to achieve competitive visual quality at 60 frames per second. He will then describe the process of introducing a physically-based lighting model to the series in Call of Duty: Black Ops, from the premise behind the model to the specific benefits and issues encountered when integrating it into the game.
• Real-time Image Quilting: Arbitrary Material Blends, Invisible Seams, and No Repeats, by Hugh Malan (Graphics Programmer, CCP Games): A pixel shader-based image quilting technique which handles situations where standard environment texturing has problems: transitions between arbitrary neighbor materials, localized texture features due to custom geometry, and geometry-dependent edge effects. Production details such as vertex sharing and compaction techniques, texture storage options, and implementation issues for PC and console will also be covered.
• Dynamic Lighting in God of War III, by Vassily Filippov (Lead Game Programmer, SCEA Santa Monica): this talk will cover a novel forward lighting approach used in God of War III to create rich dynamically lit environments with dozens of light sources applied to a single pixel. The description will include a complete mathematical explanation of the algorithm, as well as implementation details such as the combination of multiple lights into a single aggregate light per vertex on the Playstation 3’s SPUs, a new light interpolation approach which improved lighting accuracy, and the application of the aggregate lights per pixel on the GPU. Usability constraints, edge cases and ways to reduce artifacts will be covered in detail.
• Pre-Integrated Skin Shading, by Eric Penner (Rendering Engineer, Electronic Arts Vancouver): Eric will describe a technique for rendering realistic skin in games, where rather than gathering neighboring light to simulate subsurface scattering, the effects of scattered light are pre-integrated. This allows for achieving the non-local effects of subsurface scattering using only locally stored information and a custom shading model.

Filtering Approaches for Real-Time Anti-Aliasing

From a theoretical standpoint, performing anti-aliasing as a post-process is locking the barn door after the horses have bolted. However, such techniques have recently proven to be surprisingly effective in practice – a flurry of algorithms, implementations, and variants have created one of the most important real-time rendering trends. For this course, the organizers – Jorge Jimenez (Real-Time Graphics Researcher, University of Zaragoza) and Diego Gutierrez (Associate Professor, University of Zaragoza) – have tracked down the inventors of pretty much every important technique in this area and recruited them to present their work:

• Morphological Antialiasing (MLAA), presented by Alexander Reshetov (Senior Staff Researcher, Intel) – this technique was presented as a paper at the High Performance Graphics (HPG) conference in 2009; the impressive results shown sparked most of the current interest in this general approach.
• A Directionally Adaptive Edge Anti-Aliasing Filter, presented by Jason Yang (Principal Member of Technical Staff , AMD). This technique was also presented as a HPG 2009 paper, and was influential as well.
• A GPU-friendly variant of MLAA, presented by Jorge Jimenez (Real-Time Graphics Researcher, University of Zaragoza). This variant was published in the book GPU Pro 2; the talk will also cover recent developments not included in the book.
• A hybrid CPU/GPU MLAA variant implemented for Costume Quest on the XBox 360,  presented by Pete Demoreuille (Lead Programmer, Double Fine).
• The Playstation 3/SPU MLAA implementation first used in God of War III and subsequently made available to all Playstation 3 developers as part of the EDGE library. Tobias Berghoff (Senior Programmer, SCEE) will detail the implementation (including recent improvements), and Cedric Perthuis (Senior Staff Graphics Engineer, SCEA Santa Monica) will talk about how the technique was integrated into the God of War III engine.
• The SPU-based Anti-Aliasing technique (SPUAA) used on the Playstation 3 version of The Saboteur, presented by Henry Yu (Founder and CEO, Kalloc Studios). This technique has long been a topic of speculation among game developers, and will be discussed here for the first time.
• Subpixel Reconstruction Antialiasing (SRAA), presented by Morgan McGuire (Assistant Professor, Williams College and Visiting Scientist, NVIDIA). This technique was presented as a paper in the 2011 Symposium on Interactive 3D Graphics and Games (I3D).
• Fast approXimate Anti-Aliasing (FXAA), presented by Timothy Lottes (Developer Technology, NVIDIA). Fast and effective, this technique is currently being evaluated by many developers for inclusion in their games.
• Distance-to-Edge Anti-Aliasing (DEAA), presented by Hugh Malan (Graphics Programmer, CCP Games – formerly at Realtime Worlds).
• Geometry Buffer Anti-Aliasing (GBAA), presented by Emil Persson (also known as “Humus” – Graphics Programmer, Avalanche Studios).
• The Directionally Localized Anti-Aliasing (DLAA) technique used in Star Wars: The Force Unleashed 2, presented by Dmitry Andreev (Senior Rendering Engineer, Visceral Games – formerly at LucasArts).
• The temporal filtering anti-aliasing technique used in Crysis 2, presented by Tiago Sousa (Principal R&D Graphics Engineer, Crytek).

Beyond Programmable Shading (Part I & Part II)

Similarly to the “Advances in Real-Time Rendering” course, “Beyond Programmable Shading” is an “ensemble” course which has been presented annually at SIGGRAPH for several years running. As its name reflects, it deals with GPU-based graphics that go beyond the traditional graphics pipeline. This course has had uniformly high quality each year, and 2011 appears to be no exception. Part I starts with a 20-minute introduction by the course organizers – Aaron Lefohn (Lead Research Scientist, Intel) and Mike Houston (Fellow, AMD) – and continues with six 25-30 minute talks:

• Peter-Pike Sloan (Research & Development Lead, Disney Interactive Studios) will give a talk (title to be determined) about the applicability of current graphics research to games, discussing examples of research that works in games today, as well as research that does not work  – and why.
• GPU Architecture, by Mike Houston (Fellow, AMD): an overview talk covering GPU architecture – unlike similar talks in previous iterations of the course, the architecture talk is extended this year to include heterogeneous architectures.
• Scheduling the Graphics Pipeline, by Jonathan Ragan-Kelley (PhD Student, MIT): this is an extension of a talk given by Jonathan in last year’s course – it will include significant new material, including more specifics on how scheduling works in particular GPU architectures.
• Parallel Programming for Real-Time Graphics, by Aaron Lefohn (Lead Research Scientist, Intel): compared to the talk of the same name in last year’s course, this talk will be significantly re-written and updated, including an increase in the number of concrete examples.
• Software Rasterization on GPUs, by Samuli Laine (Senior Research Scientist, NVIDIA) and Jacopo Pantaleoni (Senior Architect, NVIDIA): software rasterization on GPUs can be an effective way to bypass the limitations of the GPU’s fixed-function rasterizer. Each of the speakers will be discussing papers they will publish at HPG 2011 – in Samuli’s case, High-Performance Software Rasterization on GPUs and in Jacopo’s case, VoxelPipe: A Programmable Pipeline for 3D Voxelization.
• The course organizers are still in the process of finalizing the topic and speaker of the last talk.

Part II starts with a brief welcome and re-introduction by Mike Houston. This is followed by four 30-40 minute talks, all new to this course series:

• Toward a Blurry Rasterizer, by Jacob Munkberg (Research Scientist, Intel): this talk will cover the current state of the art in rasterizing triangles with motion and defocus blur – this is a very active area of research, which I suspect will yield some important GPU advances in the near future. Jacob has co-authored several important papers in this area – most notable are the Graphics Hardware 2007 paper Stochastic Rasterization using Time-Continuous Triangles and the HPG 2011 paper Hierarchical Stochastic Motion Blur Rasterization.
• Order-Independent Transparency, by Marco Salvi (Research Scientist, Intel): similarly to the previous talk, this covers the current state of the art in an important topic on which the speaker has considerable expertise. Of Marco’s work on the topic, most notable is the HPG 2011 paper Adaptive Transparency (not yet available online but his GDC 2011 talk on the topic – including source code – is available).
• Interactive Global Illumination, by Chris Wyman (Associate Professor, University of Iowa): this is the third “state-of-the-art talk” covering a relatively broad topic. Chris’ publications page includes numerous papers on this topic, some including source code.
• User-Defined Pipelines for Ray Tracing, by Steven Parker (Director of High Performance Computing and Computational Graphics, NVIDIA): this is a more tightly focused talk than the previous three. It has the potential to be quite interesting, given the speaker’s central role in the development of the Optix ray tracing system (he was the first author on the SIGGRAPH 2010 paper) as well as his area of responsibility at NVIDIA.

The course closes with a 15-minute wrap-up talk by the organizers (on the topic “What’s Next for Interactive Rendering Research?”), followed by a 45-minute panel discussion between the various course speakers.

I’ll continue going over the remaining SIGGRAPH 2011 courses in my next few blog posts.