Category Archives: Miscellaneous

Humblebrag on the third anniversary of the MOOC

I just realized today that it’s been three years since the 3D Interactive Graphics MOOC came out. It’s still chugging along, surprisingly enough, getting about 35 signups a day. Of course, completion rates are a small fraction of that – I’d like to know myself what it is. Me, I’m still answering questions on the discussion board.

It’s been a good week for positive comments from people taking the course. Who wouldn’t like reading posts such as this, “It is incredibly easy to learn and the examples are vivid and awesome. I wish the professors at my university were like you!” Which I take as more a reflection of the level of teaching at that (unnamed) university – I know there are more engaging and dynamic teachers than me. The takeaway is that videos and demonstrations that are just a link away can offer a fair bit, just as films have some advantages compared to live performances. Integrating these newer technologies into the classroom is the exciting challenge.

Another person gave praise to my short dot product explanation videos, even adding links to them on Wikipedia’s dot product page (which I just edited, removing my name). Looking at those videos now, hey, they’re pretty good! Here’s one showing how the dot product and cosine are related. Find the others here.

Remember how three years ago MOOCs were going to destroy the university system, and that everyone would get a cheap college education? The reality is that MOOCs are inexpensive (usually free) distance-learning systems for relatively well-off, educated people out of school who want to study a specific topic. You also have to be quite self-motivated to plow through a course, since the usual external motivators of a college education – getting a degree, keeping the parents happy, getting your money’s worth, and staying in school for the parties – are all missing.

I’d like to see are more graphics MOOCs beyond Ed Angel’s and mine. But the reality is that MOOCs are expensive and whether there’s a viable business plan blah blah blah.

Whatever the case, knowing that I’ve been able to help a number of people get some understanding of this great field of ours has been an unalloyed joy. Honestly, working on this course has been a lovely and lucky opportunity for me, and one of the best things I’ve done with my life.

Don’t be mean

[Some on Twitter noted that I should be using milliseconds instead of FPS. This kind of misses the point, but let’s avoid distractions, here’s the article with that change. The sad part is that you then miss my hilarious joke about how I use FPS in the article, because if I used SPF you’d think I was talking about tanning. Which makes me think of another joke about rendering cows and the time it then takes to tan their hides. I’m full of great dad jokes.]

I think I’m reading “The Economist” too much, as I keep trying to come up with punny article titles. Sorry.

So, how do you measure a representative value for milliseconds per frame?

I don’t care about the mechanics, which timer call you use, etc. Just assume you successfully start timer/end timer and get some length of time in milliseconds for the frame. What do you do with these timings?

I usually see things such as an average, or a running average (average of last 20 or 50 or 100 or whatever frame times). I think this is mostly bad. As someone pointed out, almost everyone has more than the average number of legs. I find the same: in a given run there can sometimes be some frames where things noticeably slow down for whatever reason, some load on the computer. What you’re often trying to measure (as a graphics developer) is the performance of the rendering system itself, not the computer’s overall performance.

So, I currently use one of these two, or both: shortest time, or median time, over whatever set of frame times I have. Both have their uses. Shortest time is justifiable (to me, at least) because, assuming you have a very fine-grained timer, your best time is in some sense the “purest” measurement of the time a frame takes. Whatever other processes in your system are slowing down the other frames isn’t your concern. The timer doesn’t lie, you really did go that fast for one frame.

The other measure I’m OK with is the median. If your benchmarking system is going through a series of different frames (an animation or simulation is running, or the camera is orbiting, etc.), then grabbing the median frame is good. Choosing it instead of the average then doesn’t give so much weight to outliers. Better yet, graph the results and see whether the outliers are consistent.

Update: A number of game and VR developers pointed out that their major interest is maximum frame time. Makes sense: for a good experience (especially with VR) you don’t want to drop below your target of 30 FPS, 60 FPS, or 90 FPS.

My point is that the average, the mean, is not so good: often external slowdowns throw off the average enough and at random enough intervals that the average is very noisy and so, pretty useless. Taking the median, the central time of the sorted set, cuts out much of this variance, making each sample have an equal effect on the result.

Anyway, that’s where I’m at with benchmarking. What do you do? Comment here, tweet-reply, or email me at erich@acm.org and I’ll summarize.

p.s. pro tip: walk through your rendering pipeline every once in awhile, watching each step. It’s hard to really know where the time goes without doing so. I did this last week while looking at another bug and found a little logic error was causing a certain path to always do an additional post-process when it usually wasn’t needed. Free performance boost with a two-line fix! But, not something discoverable by benchmarking, because the variance is too much to notice “just” a few frames of difference.

This happens every few years. My favorite lucky find was around 15 years ago, walking through code in an established project and seeing that it was rendering twice for each time it displayed. A one-line change gave us 2x performance.

Seven Things for April 6, 2016

Let’s get visual. Last in the series, for now.

Seven Things for April 5, 2016

All linked out yet? Here’s more worthwhile stuff I’ve run across since last SIGGRAPH.

Seven Things for April 4, 2016

Next in the continuing series. In this episode Jaimie finds that the world is an illusion and she’s a butterfly’s dream, while Wilson works out his plumbing problems.

Seven Things for April 3, 2016

The things:

Seven Things for April 2, 2015

I haven’t made one of these link posts for awhile. This one’s recent news, the ones to come will have more fun stuff.

sadly, human computers mostly got “calculate this boring number” assignments and very rarely got “if i was james counterstrike and i fired this rpg at this nightorc tell me how many gibs would come out”: one of history’s true missed opportunities

One more variable…

Michael Cohen was looking at John Hable’s useful test image:

He noticed an odd thing. Looking at the image on his monitor (“an oldish Dell”) from across the room, it looked fine, the 187 area matched the side areas:

ok

(yes, ignore the moires and all the rest – honestly, the 187 matches the side bars.)

However, sitting at his computer, the 128 square then matched the side bars:

bad

Pretty surprising! We finally figured it out, that it’s the view angle: going off-axis resulted in a considerably different gray appearance. Moving your head left and right didn’t have much effect on the grays, but up and down had a dramatic effect.

Even knowing this, I can’t say I fully understand it. I get the idea that off-axis viewing can affect the brightness, but I would have thought this change would be consistent: grays would be dimmed down the same factor as whites. That last image shows this isn’t the case: the grays may indeed be dimmed down, but the alternating white lines are clearly getting dimmed down more than twice as much, such that they then match the 128 gray level. It’s like there’s a different gamma level when moving off-axis. Anyone know the answer?

Addendum: and if you view the first image on the iPhone, you get this sort of thing, depending on your zoom level. Here’s a typical screen shot – I’ve trimmed off the right so that the blog will show one pixel for pixel (on desktop computers – for mobile devices you’re on your own). This darkening is from bad filtering, see the end of this article.

iphone

Follow-up: one person commented that it’s probably a TN panel. Indeed, there’s a video showing the tremendous shift that occurs. The blue turning to brown is particularly impressive. I haven’t yet found a good explanation of what mechanism causes this shift to occur (Wikipedia notes other monitors having a gamma shift, so maybe it is something about gamma varying for some reason). There’s a nice color shift test here, along with other interesting tests.

Even better: check out this amazing video from Microsoft Research. (thanks to Marcel Lancelle for pointing it out)

Sci-Hub

Here’s a fascinating article on Sci-Hub, the “Pirate Bay” of scientific research papers. Really, go read it.
 
My sympathy lies with Alexandra Elbakyan. The key points to me are that researchers already informally download or ask other researchers for preprints. Sci-Hub wastes less time for this process. In physical terms, the very minor value-added of the final copy vs. the author’s draft is the main thing being “stolen”. Given that researchers make no royalties off the papers, there’s no loss to them there. The main thing journals sell is prestige.
 
That said, I don’t want journals to die on the vine, they deserve some money (though certain publishers seem way too profitable). I don’t see a good solution to these constraints:
  • Research papers should be free to anyone to access, especially since the authors do not earn royalties and want their papers to be read.
  • Publishers deserve to eat. Update: by which I mean, whoever is hosting and maintaining the journal deserves some reasonable amount of money. I don’t subscribe to the “people making buggy whips should have their jobs maintained and the automobile should be outlawed” school of thought.
In a sense, we already have a solution: author pre-prints are sometimes available on their websites. Google Scholar does a fairly good job finding these. But gathering these pre-prints on a single site is considered illegal; pre-prints themselves are probably illegal since the publisher usually owns the final article, but publishers rarely attack their unpaid writers the researchers to take their own work off their own sites. So the pre-print solution is not very good. It’s spotty coverage at best and the number of authors’ sites decrease over time as they move or die. A more permanent repository is needed.
One solution to the problem is the one-time fee to the journal to coordinate peer review (which is usually done by an unpaid editor researcher anyway) and publish the article (layout is done by the researcher, which is the norm in my field). If these fees were, say, $200 for a 12 page paper, great (well, not great, but at least understandable). For the publishers that allow this form of payment, it’s more like $2000 on up.
 
Another solution is to no longer use a paid publisher. Online journals such as the Journal of Computer Graphics Techniques are where there is no paid publisher involved, and a university provides permanent storage and distribution of the contents. In this model there are literally no costs to the researchers or readers, just the university or other institution pursuing its mission of the dissemination of information. There’s plenty of other things for publishers to publish and market and distribute, so they’ll still eat.
 
Back when journals and article reprints were on paper, and when layout and distribution was done by the publisher, $25/paper costs made sense. The internet and websites aren’t free, but nearly so. So why the high fees? Because they can.