Author Archives: Naty

Direct3D 11 Details Part IV: Multithreaded Rendering

Direct3D 10 only allows graphics commands to be issued from a single thread (there is a multithreaded mode, but Microsoft explicitly warns against using it due to its poor performance). In an API such as Direct3D, issuing graphics commands involves a fair amount of CPU overhead. Given the trend towards increasing the number of cores on a processor rather than the performance of a single core, it is desirable to efficiently spread this work among multiple threads.

Direct3D 11 adds the ability to create display lists from multiple threads and execute them from the main rendering thread. In addition, the Device (which creates resources) has been separated from the Context (which issues graphics commands). This enables creating resources asynchronously. Deferred Contexts are used to create display lists and the Immediate Context issues graphics commands to the GPU, including the execution of display lists created on Deferred Contexts.

Unlike the other features in Direct3D 11, multithreaded rendering is not a hardware feature at all. With the appropriate drivers, D3D10 (perhaps even D3D9) hardware will be able to perform multithreaded rendering efficiently (some level of multithreaded performance will be available even without new drivers, but it was unclear what the limitations would be in this case).

Direct 3D 11 Details Part III: Compute Shaders & Unordered Memory

GPGPU (General-Purpose computation on GPU) approaches such as NVIDIA’s CUDA have become increasingly popular the last few years, recently coming full-circle with various non-traditional rendering algorithms (perhaps this should be called GPGPUG?). However, the existing solutions are vendor-specific, often requiring reprogramming even for different GPUs from the same vendor. They also tend not to “play well” with the traditional graphics pipeline. For example, on GeForce 8000-series GPUs using CUDA there is a large delay when switching between CUDA and traditional graphics rendering.

Direct3D 11 introduces a new kind of shader called a Compute Shader. A compute shader is invoked as a regular array of threads. The threads are divided into groups. Each group has 32KB of memory shared among the threads in the group. Thus the threads can use partial results computed by other threads in the same group, improving performance. Threads can also perform random-access reads and writes to graphics resources such as textures, vertex arrays or render targets. These memory accesses are unordered, although various synchronization instructions exist to impose ordering when needed.

Pixel shaders can also perform random-access (unordered) writes. This allows them to write data structures such as linked lists that can then be processed by a compute shader, or vice-versa (pixel shaders have always had the ability to perform random access reads via texture lookups).

Several examples of compute shaders were shown at Gamefest, performing post-process operations such as finding the average luminance of a render target, or computing a luminance histogram (both used in tone mapping). For these operations, a 2X speedup was quoted over the best performance possible using pixel shaders.

Compute shaders can also perform operations such as computing summed-area tables and fast-Fourier transforms significantly faster than traditional GPU methods. Microsoft is looking into providing library functions to perform such operations.

Microsoft speculated that algorithms such as A-buffer rendering and ray tracing could also be performed efficiently, but they don’t have any hard performance numbers for those.

Direct3D 11 Details Part II: Tessellation

Direct3D 11 adds three new pipeline stages, with the goal of enabling efficient tessellation of higher order surfaces. This is the Direct3D 10 pipeline, as shown in “Real-Time Rendering, 3rd Edition”:

Direct3D 10 Pipeline

The color of each stage indicates whether it is fully programmable (green), configurable (yellow) or fixed function (blue). The stages are described more fully in the “Graphics Processing Unit” chapter of the book. Note that the “Geometry Shader” stage is new to Direct3D 10, but the other stages have been in the pipeline for quite a while.

The Direct3D 11 pipeline adds three new stages between the vertex and geometry shader stages (framed in red). Two of the new stages are programmable (the hull and domain shader stages) and one is configurable (the tessellator stage):

Direct3D 11 Pipeline

This pipeline operates on meshes represented as a series of surface patches. Triangle and quad surface patches are primitives in Direct3D 11 (there is also a tessellated line primitive). The shape of each patch is defined by a number of control points. These control points are transformed, skinned and / or morphed one by one in the vertex shader.

The hull shader is called for each patch, using the patch control points from the vertex shader as inputs. The hull shader has two main responsibilities. The first is to (optionally) convert the control points from one representation (basis) to another. for example, it can implement the technique introduced in Loop and Schaefer‘s paper “Approximating Catmull-Clark Subdivision Surfaces with Bicubic Patches“. The control points are sent directly to the domain shader, bypassing the tessellator. The hull shader’s second responsibility is to compute appropriate tessellation factors, which are passed to the tessellation stage. This allows for adaptive tessellation, which can be used for continuous view-dependent LOD (level of detail). The tessellation factors are specified per patch edge, and range from 2 to 64. This means that each edge of the patch may be split into at least 2 (and as many as 64) triangle (or quad) edges.

The tessellator is a fixed-function (but highly configurable) stage, which uses the tessellation factors to tessellate (subdivide) the patch into multiple triangle or quad primitives. The tessellator does not have access to the control points – all tessellation decisions are made based on configuration and the tessellation factors passed on from the hull shader. Each vertex resulting from the tessellation is output to the domain shader. Only the patch parametrization coordinates are passed on for each vertex.

The domain shader operates on the patch parametrization coordinates of each vertex separately, although it can also access the transformed control points for the entire patch. The domain shader sends the complete data for the vertex (position, texture coordinates, etc.) to the geometry shader (or the clipping stage if no geometry shader is present). Effectively, it evaluates the surface representation at each vertex. Techniques such as displacement mapping can also be applied by this shader stage.

Although Microsoft gave an example using Catmull-Clark subdivision surfaces, the programmability of the pipeline enables other surface representations to be used. Alternatively, the tessellation stages can be turned off and traditional triangle or quad meshes can be used.

Direct3D 11 Details Part I: Intro

I attended Gamefest 2008 last week. Gamefest (formerly called Meltdown) is a Microsoft-run Windows and Xbox 360 game development conference. This year there were two notable announcements: XNA Community games (discussed in a previous blog post) and the first public disclosure of Direct3D 11.

Direct3D is, of course, the API used by most Windows games, but its importance extends beyond Windows. Direct3D features guide the development of graphics hardware in general, so these features are bound to show up in future consoles, as well as in OpenGL.

The announcement that Direct3D 11 would not be tied to the next version of Windows (as many had feared), and would be available on Windows Vista was very significant to Windows developers, many of whom complained about the tying of Direct3D 10 to Windows Vista. Direct3D 11 will also be available on Direct3D 9, 10, and 10.1 level graphics hardware (although the new features will not be available there, with the exception of some multithreading enhancements).

The fact that the Direct3D 11 API is a strict superset of the 10/10.1 API is also cause for relief among game developers. From Direct3D 9 to 10, the API went through extensive changes. These changes were mostly long-overdue cleanups and improvements, but they left developers supporting two very different APIs if they wanted to support the many customers using Windows XP and also expose the new Direct3D 10 hardware features.

This is the first part of a multi-part post which will summarize the essential facts about Direct3D 11, as known from the Gamefest slides. Eventually, the slides should show up on the XNA Presentations page.

Full disclosure of Direct3D 11 should occur later this year – the November 2008 DirectX SDK release will feature a preview version of the API, including full documentation and code samples.

Community-built games on Xbox 360

Many of our readers are not professional game developers, but do graphics or game programming as a hobby or as part of their academic research. Microsoft’s XNA Game Studio is interesting since it allows free development of Xbox 360 games. To be precise, although the software is free, Xbox 360 development does require a $99/year premium membership – still a bargain compared to the many thousands of dollars required for a professional console development kit. However, the resulting games could only be played by other people with premium memberships – not exactly a mass market.

This week, at Gamefest, Microsoft announced that these “homebrew” games could now be sold to Xbox 360 owners in general. Interestingly, the games will not be selected by Microsoft themselves (although I am sure they will do some gatekeeping) but by the community (similarly to the selection of posts at Digg or Slashdot).

If you are a game or graphics hobbyist who is intrigued by the idea of creating games to sell to almost 12 million Xbox 360 owners, then check out Microsoft’s XNA Creators Club website.

Graphics conference roundup

After taking another look at my recent post on 2008 conferences, I thought I would give some context on the various graphics conferences for people who are not familiar with them.

There are a handful of large, international conferences which cover the entire field of computer graphics:

  • The SIGGRAPH annual conference (technically the “International Conference on Computer Graphics and Interactive Techniques”) is the great-granddaddy of graphics conferences. It’s been around since 1974, and is by far the largest conference devoted to graphics. It has historically been attended mostly by academics and people working in film production, and most papers are not about real-time techniques. In recent years the conference has been making an effort to attract more attendees and speakers from the game industry. The quality of the papers tends to be quite high, and many of them are relevant, even the ones discussing offline techniques often have interesting stuff in them. Besides the papers, there are also courses (called “classes” this year) which are very good. In particular, the excellent (but somewhat verbosely-named) “Advances in Real-Time Rendering in 3D Graphics and Games” class has been presented for the last few years and has very good and relevant presentations from leading real-time graphics practitioners. Many of the classes on film production rendering techniques also have a surprisingly large amount of material relevant for real-time rendering. The Computer Animation Festival (which has this year been expanded to a full-scale film festival) showcases the best CG of the year and is always fun to watch. Although SIGGRAPH has so far always been held in the continental United States (often alternating between west coast and non-west coast locations), in 2011 it will be held in Vancouver, British Columbia.
  • SIGGRAPH Asia is a new arrival on the scene, being held for the first time this year. It is held in winter and is intended be one of three main “tripod” graphics conferences (with the North American SIGGRAPH conference in the summer, and the Eurographics conference newly moved to the spring). Spreading them evenly throughout the year in this way can enable researchers to submit work when it is ready and not wait for the SIGGRAPH submission deadline – or at least such is the theory. In the past, the existence of other conferences did not prevent most researchers from submitting their work to SIGGRAPH first, but perhaps this will change.
  • The annual Eurographics conference is the third “major graphics conference”. As its name would suggest, it is held in Europe every year, usually in beautiful locations such as Vienna, Prague, and Crete. The quality of the papers is usually quite high, though it also tends to have mostly non-real-time papers (perhaps even more so than SIGGRAPH).
  • Computer Graphics International is smaller than the preceding three. It is sponsored by the Computer Graphics Society (CGS).

There are also several regional graphics conferences:

  • Graphics Interface is the largest and oldest of these (it is roughly as old as the SIGGRAPH annual conference, and indeed claims to be “the oldest continuously-scheduled conference in the field”). It has always been held in Canada. It tends to have a strong HCI (Human-Computer Interaction) component, as well as some good real-time rendering papers.
  • Pacific Graphics (more properly “The Pacific Conference on Computer Graphics and Applications”) has been held in locations such as Tokyo, Taipei, Maui, and Macao. The papers are typically of high quality, and have included some important real-time rendering papers. It will be interesting to see how Pacific Graphics fares now that SIGGRAPH Asia has arrived on the scene.
  • WSCG is a Central European graphics conference. It has had some interesting real-time papers. Unlike the other conferences, the full proceedings of WSCG are freely available.
  • The Spring Conference on Computer Graphics is another Central European graphics conference.
  • SIBGRAPI has been held in Brazil for the past 20 years.
  • AFRIGRAPH is another relatively recent regional conference. It has been held in Cape Town, South Africa since 2001.

Besides the generic graphics conferences, there are many conferences focusing on specific subfields of graphics. The ones of most interest to real-time rendering practitioners and researchers are:

  • EGSR (“Eurographics Symposium on Rendering”) is a relatively large conference focused on all aspects of rendering, both offline and real-time. Some of the most important real-time rendering papers have been published through this conference.
  • I3D (“Symposium on Interactive 3D Graphics and Games”) has been around for about twenty years, although it has been an annual conference only for the past few years (and has added the “and Games” part of its title more recently still). I’ve attended it twice, it is a nice, small conference. The papers are a mix of HCI and real-time rendering papers, some of which have been quite important to the field.
  • Graphics Hardware (more properly, the SIGGRAPH/Eurographics Conference on Graphics Hardware) alternates between the USA (where it is co-located with SIGGRAPH) and Europe (where it used to be co-located with Eurographics – since Eurographics was moved to the spring it has been co-located with EGSR). In theory, the papers there should only be of interest to people designing graphics hardware but in practice many of the papers are of great interest to people writing software as well.
  • Like Graphics Hardware, the Symposium on Computer Animation is also held jointly by SIGGRAPH and Eurographics. It similarly alternates between the USA and Europe. Although technically not a rendering conference, the field of computer animation is in practice strongly linked to rendering (in particular real-time rendering) so it is of interest.
  • NPAR (or “Symposium on Non-Photorealistic Animation and Rendering”) is another joint SIGGRAPH/Eurographics conference. It is devoted to the rapidly growing field of non-photorealistic or stylized rendering. The first few conferences were held in Annecy, France but it was held in San Diego in 2007 and seems likely to alternate from now on.
  • Although the Symposium on Geometry Processing is also a joint effort between SIGGRAPH and Eurographics, it has only been held in Europe so far. Geometry processing is another topic which is strongly linked to rendering.
  • For the past three years the fast-growing field of interactive ray tracing has had its own conference, IRT (or “Symposium on Interactive Ray Tracing”). It is jointly held by IEEE and Eurographics, and alternates between the USA and Europe.

GDC (or “Game Developers Conference”) is another conference of interest to real-time rendering practitioners. Unlike the previously mentioned conferences, which are run by nonprofit professional organizations like ACM, IEEE, and Eurographics, GDC is run by Think Services, which is a for-profit corporation. It is really more like a trade show than an academic conference, and the presentations do not undergo a strict peer-review process. Much of the material relates to non-graphics topics like gameplay and audio design. Nevertheless, there is much interesting material on real-time rendering presented there by game and graphics hardware developers. Although GDC is held in San Francisco, the GDC brand has recently expanded to cover conferences in Texas, China, and France.

Anyone interested in the field of real-time rendering would be well-advised to attend one of these conferences if possible. If not, the printed proceedings can be purchased for reasonable prices and most are available through various digital libraries. Better still, many of the papers (as well as class notes and other materials) can be found on the web for free – Ke-Sen Huang’s excellent homepage makes for a good starting point.

Papers from 2008 conferences

One extremely valuable resource mentioned in our portal page is Ke-Sen Huang’s lists of paper preprints from recent conferences. In the last two months, a whole bunch of these have popped up for this year’s conferences. The most relevant of these are:

Some of the other conferences listed on Ke-Sen’s central page deal with topics, such as animation or geometry processing, which may also be of interest. In general, conferences such as these are fertile ground for finding cool and useful new ideas (along with some wildly impractical ones!).