60 Hz, 120 Hz, 240 Hz…

Update: first, take this 60 vs. 30 FPS test (sadly, now gone! Too much traffic, is my guess). I’ll assume it’s legit (I’ll be pretty entertained if it isn’t). If you get 11/11 consistently, what are you looking for?

A topic that came up in the Udacity forum for my graphics MOOC is 240 Hz displays. Yes, there are 240 Hz displays, such as the Eizo Foris FG2421 monitor. My understanding is that 60 Hz is truly the limit of human perception. To quote Principles of Digital Image Synthesis (which you can now download for free):

The effect of temporal smoothing leads to the way we perceive light
that blinks, or flickers. When the blinking is slow, we perceive the
individual flashes of light. Above a certain rate, called the critical
flicker frequency (or CFF), the flashes fuse together into a single
continuous image. Far below that rate we see simply a series of still
images, without an objectionable sense of near-continuity.

Under the best conditions, the CFF for a human is around 60 Hz [389].

Reference 389 is:

Robert Sekuler and Randolph Blake. Perception. Alfred A. Knopf, New York, 1985.

This book has been updated since 1985, the latest edition is from 2005. Wikipedia confirms this number of 60 Hz, with the special-case exception of the “phantom array effect”.

The monitor review’s “Response Time and Gaming” section notes:

Eizo can drive the LCD panel at 240 Hz by either showing each frame twice or by inserting black frames between the pictures, which is known to significantly reduce blurring on LCD panels.

This is interesting: the 240 Hz is not that high because the eye can actually perceive 240 Hz. Rather, it is used to compensate for response problems with LCD panels. The very fact that an entirely black frame can be inserted every other frame means that our CFF is clearly way below 240 Hz.

So, my naive conclusions are that (a) 240 Hz could indeed be meaningful to the monitor, in that it can use a few frames that, combined by the visual system itself, give a better image. This Hz value of the monitor should not be confused with the Hz value of what the eye can perceive. You won’t have a faster reaction time with a 120 Hz monitor.

The thing you evidently can get out of a high-Hertz monitor is better overall image quality. I can imagine that, on some perfect monitor (assume no LCD response problem), if you have a game generating frames at 240 FPS you’re getting rendered 4 frames blended per “frame” your eye received. Essentially it’s a very expensive form of motion blur; cheaper would be to generate 60 FPS with good motion blurring. Christer Ericsson long ago informally noted how a motion-blurred 30 FPS looks better to more people than 60 FPS unblurred (and recall that most films are 24 FPS, though of course we don’t care about reaction time for films). What was interesting about the Eizo Foris review is that the reviewer wants all motion blur removed:

You probably already own a 120 Hz monitor if you are a gamer, but your monitor most likely does not have the black frame insertion technology, which means that motion blurring can still occur (even though there is not [sic] stuttering because of 120 Hz). These two factors are certainly not independent, but 120 Hz does not ensure zero motion blurring either, as some would have you believe.

The type of motion blur they describe here is an artifact, blending a bit of the previous frame with the current frame. This sort of blur I can imagine is objectionable, objects leaving (very short lived) trails behind them. True (or computed) motion blurring happens within the frame itself, simulating the camera’s frame exposure length, not with some leftover from the previous frame. I’d like to know if gamers would prefer 60 FPS unblurred vs. 60 FPS “truly” blurred. If “unblurred” is in fact the answer, we can cross off a whole area of active research for interactive rendering. Kidding, researchers, kidding! There would still be other reasons to use motion blur, such as the desire to give a scene a cinematic feel.

For 30 vs. 60 FPS there is a “reaction time” argument, that with 60 FPS you get the information faster and can react more quickly. 60 vs. 120 vs. 240, no – you won’t react faster with 240 Hz, or even 120 Hz, as 60 Hz is essentially our perceptual maximum. My main concern as this monitor refresh speed metric increases is that it will be a marketing tool, the equivalent of Monster cables to audiophiles. Yes, there’s possibly a benefit to image quality. But statements such as “there is not [sic] stuttering because of 120 Hz” make it sound as if our perceptual system’s CFF is well above 60 Hz – it isn’t. The image quality may be higher at 120 or 240 Hz, and may even indirectly cause some sort of stuttering effect, but let’s talk about it in those terms, rather than the “this faster monitor will give you that split-second advantage to let you get off the shot faster than your opponent” discussion I sometimes run across.

That said, I’m no perception expert (but can read research by those who are), nor a hard-core gamer. If you have hard data to add to the discussion, please do! I’m happy to add edits to this post with any rigorous or even semi-rigorous results you cite. “I like my expensive monitor” doesn’t count.

p.s. I got 4/11 on the test, mainly because I couldn’t tell a darn bit of difference.

SIGGRAPH 2014 Book Crop

I’ve updated the graphics books listing hosted at our site. This is excruciatingly dull HTML editing; I hope it helps you out. Many of the additions are from CRC, since I was able to view their books at SIGGRAPH – the number of book vendors seemed way down this year, maybe two total? If you find (or wrote!) a relevant book that’s not listed, let me know.

The secret takeaway on our webpage: check the additional links I give at the end of most listings. Many books have some sort of free preview and a related website with code, lecture notes, etc. For example, Multithreading for Visual Effects has a website that includes the SIGGRAPH 2013 course notes that the book is based on.

I like that the new book Introduction to Computer Graphics: A Practical Learning Approach has an associated website named http://www.envymycarbook.com/, chosen because the book’s overarching project is developing a race driving game. Calling their book Envy My Car would have been wonderfully foolish. I guess this is a reason why we still have publishers.

There are also other interesting resources you can find tucked away in these websites, such as this list of on-line articles related to Game Engine Architecture. A bunch of the URLs listed there are easily-discovered wikipedia links, but quite a few are solid blog entries or other web pages you might not find in a quick search. This sort of editorial grooming of web resources is valuable. The 2nd edition’s list of URLs is not up yet, and I can understand why. Please don’t remind me how dated a fair bit of our own main page has become – managing links is a giant time suck, so I appreciate it whenever anyone else makes this sort of effort.

Dig deep enough on some of these book websites and you might find oddities such as this list of ten reasons to write a computer graphics textbook. I guess we’re in the bastard category?

I did do some back-filling, adding older books that could (someday) be relevant to interactive rendering, e.g., Production Volume Rendering. I didn’t add all possible vaguely-related books. From the cover and title, The Magic of Computer Graphics looks like a coffee-table book, pretty pictures and minimal content. Looking inside, it turns out to be a heavy-duty text on materials and illumination theory. For example, by page 11 you’re exposed to an integral for the BRDF, and that’s the ninth equation introduced by then. I left it out mostly because it’s an odd duck. The book Visual Perception from a Computer Graphics Perspective looks like a good volume if you’re really really into perception, but not all that related to interactive 3D graphics. I was also tempted by Digital Geometry in Image Processing, mostly because of the cover – I’m in solidarity with anyone who voxelizes teapots. This book sounds like computer graphics, but instead turns out to give a glimpse at how huge the world is. There’s a whole area of study of the theory of measurement for pixel and voxel centered coordinates? Wow. But it doesn’t look all that relevant. Feel free to read it and prove me wrong, that would be great.

No book reviews for now, as I haven’t seriously examined the newer books yet. I’ve asked for a (very) few review copies, and hope to cover these in the upcoming months. There is one book I know I won’t review (and won’t list), this alternate-universe version of Real-Time Rendering, accidentally issued by CRC Press without the realization that they already had a book with this title. An embarrassment for them, so I feel a little rude to mention it, but honestly… It was on display at the CRC booth, but not next to their “other” Real-Time Rendering, which would have made a good photo.

Luckily CRC can’t sue itself for passing off and unfair competition. It’s an interesting area of the law – titles are not copyright; trademark applies to only a series of books (e.g. “… for Dummies”). Searching on Introduction to Computer Graphics will turn up about four books, including the new one from CRC. This is fair, since the title is pretty generic and none of the books has established itself as the well-known one. I look forward to someone testing the waters in the future and publishing Physically Based Rendering: From Hog to Lard.

 

Free New Computer Vision Book

The book “Computer Vision Metrics: Survey, Taxonomy, and Analysis” is available for free download as a PDF or other formats. Go to the “Source Code/Downloads” tab in the middle of the page and work your way through the labyrinth. Also, you can get the Kindle edition for free. From my pretty limited knowledge of image processing, this looks like a useful survey book, running through common techniques and pointing to relevant references. Me, I was interested in segmentation algorithms for non-photorealistic rendering, and it has a reasonable section all about this topic.

Also, don’t forget that the (also good) book “Computer Vision: Algorithms and Applications” is free for download as a PDF (and without the maze; here’s the direct link).

Big World, Secrets of the Teapot

If you’re a member of SIGGRAPH, one perk is that you have access to the ACM Digital Library’s graphics related content. The SIGGRAPH benefits document notes:

  • Access to all ACM SIGGRAPH related content in the ACM Digital Library (This includes SIGGRAPH, SIGGRAPH Asia, and about 20 or more small conferences)

I learned where to find the list of 20 small conferences, it’s here. And it’s not 20, it’s over 100. Admittedly, some of these symposia were run just once or twice, but I appreciate the access nonetheless. It’s a big world! Wandering through this list is fascinating, and a little nostalgic – “Ahh, remember when that topic was a hot new trend? Whatever happened to it?” Honestly, it’s exciting to see so many areas where graphics has an effect. If I had students looking for research topics in graphics and no strong preference about what area they wanted to explore, I’d point them at this page as a source for inspiration, dry as it looks.

I asked about this list because I had a problem accessing some NPAR papers through the DL. As usual, I drove around the damage by using Google Scholar and finding the papers I wanted elsewhere, for free. To the ACM’s credit, they responded to my query about whether I was supposed to have access to NPAR, since I had access in the past. I was indeed, and they fixed the DL the next day. So, the takeaway is that if you find you don’t have access and think you should, let the ACM know at acmhelp@hq.acm.org.

Finally, I found this just peculiar, on this page:

teapots

Seriously? The secrets of the teapot cannot be fully revealed? Who (the heck) would not give reprint permission? Or was it just a matter of someone being unreachable, and the default being the text couldn’t be reprinted? There’s a story there…

Books page updated

I spent an inordinate amount of time just updating the books page at this site. It hadn’t been done for about two years – I can finally check this task off the list. It took awhile tracking down related websites for each book, especially Google Books samples, which can be quite large and worthwhile for some books. I also cleaned out older volumes from the listing and updated the recommendations list.

From what I can tell – and please do tell me if I’ve missed anything – beyond API books (OpenGL and DirectX) and the GPU Pro series, there have been very few new graphics books since 2013. The major release has been a new edition of Computer Graphics: Principles and Practice. There was also The CUDA Handbook, which is somewhat graphics related but not strictly so. I also included The HDRI Handbook; even though it’s more a user’s guide, it does have some good bits about the theory of tone mapping and much else, in an area that can use more coverage. I don’t bother listing the many books about Unity 3D, the Unreal Engine, etc., since those truly fall in the area of guides.

Anyway, that’s all until SIGGRAPH, when I can take a look at what else is out there.

GPU Pro 5 is out

Really, the title says it all, the book GPU Pro 5 is shipping. Sadly, there’s no “Look Inside” for the book on Amazon; I’ll hope they at least put the Table of Contents there. You can find a rough Table of Contents on the CRC site; rough in that you can’t see the number of pages for each article. A few articles are quite lengthy: Physically Base Area Lights is 34 pages long, Hi-Z Screen-Space Cone-Traced Reflections is an incredible 44. The rest are in the 10-20 page range.

You can get a taste of the book at the GPU Pro blog, it has previews of a large number of the articles. At $70 this is not a casual purchase, but if you’re a practitioner and just one article saves you 2 hours, the book’s more than paid for itself.

Me, I was amused to see the following, a model from Morgan McGuire’s high-quality model repository – hey, that’s from our world! (And you thought I was done with Minecraft references here.)

VoxeliaMC

Free Golden Paint Spectra Spreadsheet

The short version is go see this page here.

I’m collaborating on a little hobby project with Andrew Glassner. It involves paint spectra, so we were hunting down a spectrometer. We’d heard good things about ColorMunki, though it’s pricey for a hobby project. There’s now even a $40 spectrometer from Public Labs; they’re located a few miles from where I live.

Andrew decided to ask Golden Artist Colors, Inc., if they had spectra available for their paints. Happily, they did and agreed to release these for free, noting that “spectra are spectra”. This saves us a lot of grunt work and expense, which is great. I felt so good about them that I decided to make a page about their dataset so that researchers and developers could also benefit from it.

Golden Artists sample data

Chartreuse and Puce

Yes, I should weight in about Apple’s Metal announcement, or Google’s Tango smartphone, or talk about Oculus Rift, or Word Lens (which really is cool and free for a short time, so get it now), or something. But, others have said enough on them, so let’s talk about two colors, chartreuse and puce.

I’ve blogged about chartreuse before (and answers here). The gist: it’s a color that a lot of people think they know, but don’t (including myself, once upon a time). Get it fixed in your mind before reading further. I decided to actually make it a tiny part of the Udacity MOOC, asking students about it for fun. 10,450 students responded, and here are the results:

  • 41% said yellow-green
  • 22% said red-orange
  • 19% didn’t know
  • 18% said deep purple

Please use this knowledge for good, or evil. So what about puce? Well, last night I was trying to think about other colors I wasn’t sure about. I know what drab is – that’s the color of commerce, accountancy, and business (hey, I don’t make this stuff up). One I though of was puce (aka, peuce, puse, peuse). I again got it wrong. Also, it turns out this has to be about the most disgusting color name ever – you’ve been warned. It sounds nicer in French, though.

This is a mostly content-free post (you really should see Word Lens, though – it’s like magic), but I should have an interesting color-related announcement next week, if all goes well.

Free/Cheap Processing Course by Andrew Glassner

Andrew Glassner has made an 8 week course about the graphics language Processing. The first half of the course is free; if you find you like it, the second half is just $25. Even if you don’t want to take the course, you should watch the 2.5 minute video at the site – beware, it may change your mind. The video gives a sense of the power of Processing and some of the wonderful things you can do with it. My small bit of experience with the language showed me it’s a nice way to quickly display and fiddle around with all sorts of graphical ideas. While I dabbled for a week, Andrew used it for half a decade and has made some fascinating programs. Any language that can have such a terrible name and still thrive in so many ways definitely has something going for it.

glassner1

Why use WebGL for Graphics Research?

guest post by Patrick Cozzi, @pjcozzi.

This isn’t as crazy as it sounds: WebGL has a chance to become the graphics API of choice for real-time graphics research. Here’s why I think so.

An interactive demo is better than a video.

WebGL allows us to embed demos in a website, like the demo for The Compact YCoCg Frame Buffer by Pavlos Mavridis and Georgios Papaioannou. A demo gives readers a better understanding than a video alone, allows them to reproduce performance results on their hardware, and enables them to experiment with debug views like the demo for WebGL Deferred Shading by Sijie Tian, Yuqin Shao, and me. This is, of course, true for a demo written with any graphics API, but WebGL makes the barrier-to-entry very low; it runs almost everywhere (iOS is still holding back the floodgates) and only requires clicking on a link. Readers and reviewers are much more likely to check it out.

WebGL runs on desktop and mobile.

Android devices now have pretty good support for WebGL. This allows us to write the majority of our demo once and get performance numbers for both desktop and mobile. This is especially useful for algorithms that will have different performance implications due to differences in GPU architectures, e.g., early-z vs. tile-based, or network bandwidth, e.g., streaming massive models.

WebGL is starting to expose modern GPU features.

WebGL is based on OpenGL ES 2.0 so it doesn’t expose features like query timers, compute shaders, uniform buffers, etc. However, with some WebGL 2 (based on ES 3.0) features being exposed as extensions, we are getting access to more GPU features like instancing and multiple render targets. Given that OpenGL ES 3.1 will be released this year with compute shaders, atomics, and image load/store, we can expect WebGL to follow. This will allow compute-shader-based research in WebGL, an area where I expect we’ll continue to see innovation. In addition, with NVIDIA Tegra K1, we see OpenGL 4.4 support on mobile, which could ultimately mean more features exposed by WebGL to keep pace with mobile.

Some graphics research areas, such as animation, don’t always need access to the latest GPU features and instead just need a way to visualization their results. Even many of the latest JCGT papers on rendering can be implemented with WebGL and the extensions it exposes today (e.g., “Weighted Blended Order-Independent Transparency“). On the other hand, some research will explore the latest GPU features or use features only available to languages with pointers, for example, using persistent-mapped buffers in Approaching Zero Driver Overhead by Cass Everitt, Graham Sellers, John McDonald, and Tim Foley.

WebGL is faster to develop with.

Coming from C++, JavaScript takes some getting use to (see An Introduction to JavaScript for Sophisticated Programmers by Morgan McGuire), but it has its benefits: lighting-fast iteration times, lots of open-source third-party libraries, some nice language features such as functions as first-class objects and JSON serialization, and some decent tools. Most people will be more productive in JavaScript than in C++ once up to speed.

JavaScript is not as fast as C++, which is a concern when we are comparing a CPU-bound algorithm to previous work in C++. However, for GPU-bound work, JavaScript and C++ are very similar.

Try it.

Check out the WebGL Report to see what extensions your browser supports. If it meets the needs for your next research project, give it a try!