Papers are the “main event” at CIC. Unlike the papers at computer science conferences (which are indistinguishable from journal papers), CIC papers appear to be focused more towards “work in progress” and “inspiring ideas”. This stands in contrast to the work published in color and imaging journals such as the Journal of Imaging Science and Technology or the Journal of Electronic Imaging. This distinction is actually the norm in most fields – computer science is atypical in that respect.
Note that since CIC is single-track, I was able to see (and describe in this post) all the papers, including some that aren’t as relevant to readers of this blog.
Root-Polynomial Colour Correction
Images from digital cameras need to be color-corrected, since they typically have sensors which cannot be easily mapped to device independent color-matching functions.
The simplest mapping is a linear transform (matrix), which can be obtained by taking photos of known color targets. However this assumes that the camera spectral sensitivities are linear combinations of the device-independent ones, which is not the case.
Polynomial color correction is another option which can reduce the error of the linear mapping by extending it with additional polynomials of increasing degree. However, polynomial color correction is not scale-independent – there is a chromaticity shift when intensity changes (e.g. based on lighting). This shift can be quite dramatic in some cases.
This paper proposes a new method: root-polynomial color correction. It is very straightforward: simply take the nth root of each nth order term in the extended polynomial vector. Besides restoring scale-independence, the vector also becomes smaller since some of the terms now become the same (e.g. sqrt(r*r) = r).
Experiments showed that with fixed illumination, root-polynomial color correction performed similarly to higher-order polynomial correction. It performed much better if the illumination level changes, even slightly. A large improvement is achieved by adding only three terms to the linear model, so this technique provides very good bang for buck.
Tone Reproduction and Color Appearance Modeling: Two Sides of the Same Coin?
This invited paper was written and presented by Erik Reinhard (University of Bristol), who has done some very influential work on tone mapping for computer graphics and has also co-authored some good books on HDR and color imaging.
Tone mapping or tone reproduction typically refers to luminance compression (often sigmoidal), intended to map high-dynamic range images onto low-dynamic range displays. This can be spatially varying or global over the image. However, tone mapping typically does not take account of color issues – most tone mapping operators work on the luminance channel and the final color is reconstructed via various ad-hoc methods – the two most popular ones are by Schlick (“Quantization Techniques for Visualization of High Dynamic Range Pictures”, 1994) and Mantiuk (“Color correction for Tone Mapping”, 2009). They do not take account of the various luminance-induced appearance phenomena that have been identified over the years: the Hunt effect (perceived colorfulness increases with luminance), the Stevens effect (perceived contrast increases with luminance), the Helmholt-Kohlrausch effect (perceived brightness increases with saturation for certain hues), and the Bezold-Brücke effect (perceived hue shifts based on luminance).
Color appearance models attempt to predict the perception of color under different illumination conditions. They include chromatic adaptation, non-linear range compression (often sigmoidal), and other features used to compute appearance correlates. They are designed to take account of effects such as the ones mentioned in the previous paragraph, but most of them do not handle high dynamic range images (there are some exceptions, such as iCAM and the model presented in the 2009 SIGGRAPH paper “Modeling Human Color Perception Under Extended Luminance Levels”).
Tone mapping and color appearance models appear to have important functional similarities, and their aims partially overlap. The paper was written to show opportunities to construct a combined tone reproduction and color appearance model that can serve as a basis for predictive color management under a wide range of illumination conditions.
Tone mapping operators tend to range-compress luminance and ignore color. Color appearance models tend to identically range-compress individual color channels (typically in a sharpened cone space) and do separate chromatic adaptation. A recent color appearance model by Erik and others (“A Neurophysiology-Inspired Steady-State color Appearance Model”, 2009) combines chromatic adaptation and range compression into the same step (basically doing different range compression on each channel), which Erik sees as a step towards unifying the two approaches.
Another recent step towards unifying the two can be seen in HDR extensions to color spaces (“hdr-CIELAB and hdr-IPT: Simple Methods for Describing the Color of High-Dynamic Range and Wide-Color-Gamut Images”, 2010) which replace the compressive power function with sigmoid curves. A similar approach was taken for HDR color appearance modeling (“Modeling Human Color Perception Under Extended Luminance Levels”, 2009). Image appearance models such as iCAM and iCAM06 incorporate HDR in a different way, taking account of spatial adaptation.
Some of the most successful tone mapping operators are based on neurophysiology, but put the resulting “perceived” values into a frame buffer. This is theoretically wrong, but looks good in practice. Color appearance models instead run the model in reverse from the perception correlates to display intensities (with the display properties and viewing conditions). This is theoretically more correct, but in practice tends to yield poor tone mapping since the two sigmoid curves (one run forward, one in reverse) tend to cancel out, undoing a lot of the range compression. An ad-hoc way to combine the strengths of both approaches (the color management of color appearance models and the range compression of tone mapping operators) is to run a color appearance model on an HDR image, then resetting the luminance to retain only chromatic adjustments and compressing luminance via a tone mapping operator. However, it is hoped that the recent work mentioned above (combining chromatic adaptation & range compression, sigmoidally compressed HDR color spaces, and HDR color appearance models such as iCAM) can be built upon to form a more principled unification of tone mapping and color appearance modeling.
Real-Time Multispectral Rendering with Complex Illumination
Somewhat unusually for this conference, this paper was about a computer graphics real-time rendering system. The relevance comes from the fact that it was a multispectral real-time rendering pipeline.
RGB rendering is used almost exclusively in industry applications, however it is an approximation. Although three numbers are enough to describe the final rendered color, they are not enough in principle to compute light-material interactions, which can be affected by metameric errors.
The authors wanted their pipeline to support complex real world illumination (image-based lighting – IBL), while still allowing for interactive (real-time) rendering. They used Filtered Importance Sampling (see “Real-time Shading with Filtered Importance Sampling”, EGSR 2008) to produce realistic (Ward) BRDF interactions with IBL.
The implementation was in OpenGL, using 6 spectral channels so they could use pairs of RGB textures for reflectance and illumination, two RGB render targets, etc. After rendering each frame, the 6-channel data was transformed first to XYZ and then to the display space, optionally using a chromatic adaptation transform.
The reflectance data was taken from spectral reflectance databases and the spectral IBL was captured by removing the IR filter from a Canon 60D camera and taking bracketed-exposure images of a stainless steel sphere with two different spectral filters.
The underlying mathematical approach was to use a set of six spectral basis functions and multiply their coefficients for light-material interactions, as in the work of Drew and Finlayson (“Multispectral Processing Without Spectra”, 2003). However, the authors found a new set of optimized basis functions (primaries), optimized to minimize error for a set of illuminants and reflectances.
The authors compared the analysis of their results with best-of-class three-channel methods such as the one described in the EGSR 2002 paper “Picture Perfect RGB rendering using Spectral Prefiltering and Sharp Color Primaries”. The results of the six-channel method were visibly closer to the ground truth (the RGB rendering had quite noticeable color errors in certain cases).
Choosing Optimal Wavelengths for Colour Laser Scanners
Monochrome laser scanners are widely used to capture geometry but are incapable of capturing color information. Color laser scanners are a popular choice since they capture geometry and color at the same time, avoiding the need for a separate color capturing system as well as the registration issues involved in combining disparate sources of data. These scanners scan three lasers (red, green, blue) to simultaneously obtain XYZ coordinates as well as RGB reflectance.
However, laser scanners are effectively point-sampling the spectral reflectance at three wavelengths, which is known to be a highly inaccurate method, prone to metamerism. Also, the three wavelengths typically used (635nm, 532nm, and 473nm for the Arius scanner – similar wavelengths for other scanners) are chosen for reasons unrelated to colorimetric accuracy.
The authors of this paper did a brute-force optimization process to find the three best wavelengths for minimizing colorimetric error in color laser scanners. They found that the same three wavelengths (460nm, 535nm, and 600nm) kept popping up, regardless of the reflectance dataset, the difference metric, or any other variation in the optimization process. The errors using these wavelengths were much lower than with the wavelengths currently used by the laser scanners – the color rendering index (CRI) improved from 48 to 75 (out of a 0-100 scale). Interestingly, adding a fourth and fifth wavelength gave no improvement at all.
Since these wavelengths are independent of the color space, difference metric and sample set, they must be associated with a fundamental property of human vision. These wavelengths are very close to the ‘prime colors’ (approximately 450nm, 530nm, and 610nm) identified in 1971 by Thornton (“Luminosity and Color-Rendering Capability of White Light”) as the wavelengths of peak visual sensitivity. These wavelengths were later shown (also by Thornton) to have the largest possible tristimulus gamut (assuming constant power), and are therefore optimal as the dominant wavelengths of display primaries. The significance of these wavelengths can be understood by applying Gram-Schmidt orthonormalization to the human color-matching functions (with the luminance function as the first axis) – the maxima and minima of the two chromatic orthonormal color matching functions line up along these three wavelengths. In other words, these wavelengths produce the maximal excitation of the opponent color channels in the retina.
These results are applicable not just to laser scanners but also to regular (broadband-filter) cameras and scanners, in guiding the dominant wavelengths of the spectral sensitivity functions.
(How) Do Observer Categories Based on Color Matching Functions Affect the Perception of Small Color Differences?
The CIE 2° and 10° standard observers that underlie a lot of color science are well-understood to be averages; people with normal color vision are expected to deviate from these to some extent. There is even a CIE standard as to the expected variation (the somewhat amusingly-named CIE Standard Deviate Observer). However, this does not say how human observers are distributed – are variations essentially random, or are people grouped into clusters defined by their color vision? In last year’s conference, a paper was presented which demonstrated that humans can be classified into one of seven well–defined color vision groups. This paper is a follow-on to that work, which attempts to discover if observers ability to detect small color differences depends on the group they belong to.
It turns out that it does, which opens up some interesting questions. Does it make sense to customize color difference equations and uniform color spaces to each category? Modern displays with their narrow-band primaries tend to exaggerate observer differences, so it might be a good time to explore more precise modeling of observer variation.
A Study on Spectral Response for Dichromatic Vision
Dichromats are people who suffer from a particular kind of color blindness; they only have two types of functional cones. Previous work has dealt with projections from 3D to 2D space but didn’t deal with spectral analysis; this work aims to remedy that. The study looked at three types of dichromats (each missing a different cone type), classified visible and lost spectra for each, and validated certain previous work.
Saliency as Compact Regions for Local Image Enhancement
The goal of this paper is to improve the subjective quality of photographs (taken by untrained photographers) by finding and enhancing their most salient (visually important) features.
It was previously found that people highly prefer images with high salience (prominence), where a region is highly distinct from its background. However untrained photographers often capture images without salient regions. It would be desirable to find an automated way to increase salience, but salience is very difficult to predict for general images.
This paper sidesteps the problem by finding an easier-to-measure correlate – spatial compactness (a certain property is spatially compact if it is concentrated in a relatively small area). The idea is to look at the distribution of pixels with certain low-level attributes such as opponent hues, luminance, sharpness, etc. If the distribution is highly compact (peaked), then there is probably high saliency there and enhancing that attribute will make the photograph look better. There are a few additional tweaks (small objects are filtered out, and regions closer to the center of the screen are considered more important) but that is the gist. The enhancements they did are relatively modest (5-10% increase in the most salient attribute). The results were surprisingly strong: 91% of people preferred the modified image (which is quite an achievement in the field of automatic image enhancement).
The Perception of Chromatic Noise on Different Colors
Pixel size on CMOS sensors is steadily decreasing as pixel count increases, and appears set to continue doing so based on camera manufacturer roadmaps. This increases the likelihood of noisy images; noise reduction filters (e.g. bilateral filters) are becoming more important. Tuning these filters correctly depends on a good model for noise perception. Previous work has shown that the perception of chromatic noise (noise which does not vary luminance) depends on patch color; this study was done to further explore this and to attempt an explanation.
It was found that the perception of chromatic noise was weakest when the noise was added to a grey patch, and strongest when the noise was added to a purple, blue, or cyan patch. Orange, yellow and green patches were in the middle.
Further experiments implied that these differences could be due to the Helmholtz-Kohlrausch (H-K) effect, which causes chromatic stimuli of certain hues to appear brighter than white stimuli of same brightness. Due to this effect, the chromatic noise on certain patches was partially perceived as brightness noise, which has higher spatial visual resolution.
Predicting Image Differences Based on Image-Difference Features
Image-difference measures are important for estimating (as a guide to reducing) distortions caused by various image processing algorithms. Many commonly used measures only take into account the lightness component, which makes them useless for applications such as gamut mapping where color distortions are critical. This paper takes a new approach, by combining many simple image-difference features (IDFs) in parallel (similar to how the human visual cortex works). The authors took a large starting set of IDFs, and (using a database of training images) isolated a combination of IDFs that best matched subjective assessments of image difference.
Comparing a Pair of Paired Comparison Experiments – Examining the Validity of Web-Based Psychophysics
Paired comparison experiments are fairly common in color science, but it is difficult to get enough observers. Some attempts have been made to do experiments over the web; this could greatly increase observer count, but has several issues (uncalibrated conditions, varying screen resolutions, applications like f.lux that vary color temperature as a function of time, etc.).
This paper describes a “meta-experiment” meant to determine the accuracy of web experiments vs. those conducted in a lab.
The correlation between web and lab experiments appears to be poor. That’s not to say that the data gained is not useful; when working on consumer applications, results are typically viewed in uncontrolled conditions. The web experiment performed for this paper ended up not having many participants and had a few other issues (bad presentation design, etc.)
The authors are now doing a second web experiment which has had a lot more participants and better correlation to the lab experiment. They hope to come back next year with a paper on why this second experiment was more successful.
Recent Development in Color Rendering Indices and Their Impacts in Viewing Graphic Printed Materials
Background information on color rendering indices can be found in the “Lighting: Characterization & Visual Quality” course description in my previous blog post. The current CIE standard (CIE-Ra) has several acknowledged faults (use of obsolete metrics such as the von Kries chromatic adaptation transform and the U*V*W* color space, low saturation of the test samples). A CIE technical committee (TC 1-69) was started in late 2006 to investigate methods that would work with new light sources including solid state/LED; this paper reports on the current status of their work.
There have been several proposals for color rendering indices. The current front-runner is based on the CAM02-UCS uniform color space (itself based on the CIECAM02 color appearance model). Various test sample sets were evaluated. The committee currently have a set of 273 samples primarily selected from the University of Leeds dataset (which contains over 100,000 measured reflectance spectra), and are working on reducing it to around 200. The color difference weighting method and scaling factors were also adjusted. Finally, the new index was compared with several others in a typical graphic art setting (common CMY ink set and 58 different D50-simulating lighting sources), and was found to perform well.
Memory Color Based Assessment of the Color Quality of White Light Sources
Although color rendering indices such as the one discussed in the previous paper are needed for professional applications where color fidelity is important, for home and retail lighting color fidelity is not necessarily the most desirable lighting property, instead lights that make colors appear more “vibrant” or “natural” may be preferred. Recognizing this, very recently (July 2011) a new CIE technical committee (TC1-87) was formed to investigate an assessment index more suitable for home and retail applications.
Many such metrics have been proposed over the years, most of which use a Planckian or daylight illuminant as an optimal reference. However, some light sources produce more preferred color renditions than these reference illuminants. This paper focuses on an attempt to define color quality without the need for a reference illuminant.
The approach is based on “memory colors” – the colors that people remember for certain familiar objects. The theory is that if a light source renders familiar objects close to their memory colors, people will prefer it. Experiments were performed where the apparent color of 10 familiar objects was varied and observers selected the preferred color as well as the effect of varying the color (e.g., whether changing saturation relative to the preferred color is perceived as worse than changing the brightness, etc.). This data was fit to bivariate Gaussians in IPT color space to produce individual metrics for each object. The geometric mean of these was rescaled to a 0-100 range, with the F4 illuminant at 50 (which is also its score in the CIE-Ra metric) and D65 at 90 (D65 is a reference illuminant in CIE-Ra, but was found to be non-optimal for memory color rendition).
The authors did a large study to validate the new metric and found that it matched observer’s judgments of visual appreciation better than the other metrics. For future work, they are planning to study how cultural differences affect memory colors.
Appearance Degradation and Chromatic Shift in Energy-Efficient Lighting Devices
During the next few years, many countries will mandate replacement of the incandescent lamp technology which has served humanity’s lighting needs faithfully since 1879. Incandescent lamps, being blackbody radiators, appear “natural” to consumers – they have very good color rendering and remain on the Planckian locus over their lifetime (albeit shifted in color temperature). The general CIE color rendering index (CIE-Ra) is not a sufficient metric – the speaker showed three lights, all with CIE-Ra of 85 and correlated color temperature (CCT) of 3000K; they didn’t look alike at all.
Most consumers inherently recognize the difference between incandescent and energy-efficient lamps. The lighting from the latter just doesn’t “look natural” to them. When asking focus groups about important lighting considerations, they first mention appearance issues: color quality (color rendering), color temperature (warm, normal, or cool white), form factor (shaped like a bulb, a tube, or other), dimmability (will a household triac dimmer work with it?), and glare. Appearance issues are followed by efficiency, brightness, lifetime, environmental friendliness and instant on/off.
The two types of energy efficient lights in common use today are compact fluorescent lamps (CFL) and light emitting diode (LED). In CFLs, a mercury vapor UV light excites phosphors, which emit light in the visible spectrum. White light LEDs have a blue LED which excites phosphors. Both of these light types are characterized by two-stage energy conversion. There are other energy-efficient lighting devices (HIR, HID, OLED, hybrids), but these are not practical for residential lighting.
Since both LEDs and CFL phosphors are operated at high energy densities, heat causes them to degrade over time. Since white is obtained via multiple phosphors, the differential degradation (between phosphors or between phosphors and LED) causes a chromatic shift during usage.
The authors measured aging for all three types of lamp over 5000 hours. The incandescent barely changed. The CFL had some phosphor types degrade a lot & others somewhat, causing a shift toward green. The LED lamp had huge degradation in the phosphors and almost none in the blue LED – light comes from both so the color shifted quite a bit towards blue. Both energy-efficient lamps started with bad color rendition (CRI) and it got a lot worse; luminous efficacy (lumens/Watt) also decreased.
In theory, UV sources combined with trichromatic phosphors that age uniformly could solve the problem, but that challenge has not yet been solved. Emerging energy-efficient lamp types (ESL and others) are supposed to help but aren’t ready yet, which is worrying since the transition has already started.
During Q&A, the speaker stated that he doesn’t think color rendering indices are useful at all; instead he uses color rendering maps that show the color rendering for various points on the color gamut simultaneously. These color rendering maps can show which colors are most affected. Since computers are now fast enough to compute such a map in less than a second, why use a single number? Also, the CIE CRI in common use is overly permissive – it will give high scores to some pretty bad-looking illuminants. Of course, for this very reason the light manufacturers will fight against changing it.
Meta-Standards for Color Rendering Metrics and Implications for Sample Spectral Sets
Like the previously presented paper “Recent Development in Color Rendering Indices and Their Impacts in Viewing Graphic Printed Materials”, this is also a report on the work done by the TC-69 CIE technical committee working on proposals for a new standard color rendering index, but by a different subcommittee. Neither paper appears to represent a consensus; presumably one of these approaches (or a different one) will eventually be selected.
In a recent meeting, the technical committee recommended selecting a reflectance sample set for the new CRI that is simple and as “real” as possible. This paper will talk about potential “meta-standards” by which to select the new CRI standard, and what this means for the reflectance sample set.
Their approach is based on the idea that the CRI should be equally sensitive to perturbations in light spectra regardless of where in the visible spectrum the perturbation occurs. This implies that the average curvature of the reflectance sample set should be uniform, since an area with higher curvature will be more sensitive to perturbations in light spectra. The average curvature of the 8 current CRI samples is very non-uniform, unsurprising due to the low sample count.
At first they tried to select 1000 samples from the University of Leeds sample set (which includes over 100,000 reflectance spectra). The samples were picked to be roughly equally spaced throughout the color set. The average curvature was still highly non-uniform, since many of the materials share the same small set of basic dyes and pigments. Generating completely random synthetic spectra would solve this problem, but then there would be no guarantee that they would be “natural” in the sense of having similar spectral features and frequency distributions. The authors decided to go for a “hybrid” solution where segments of reflectance spectra from the Leeds database were “stitched” together and shifted up or down in wavelength. This resulted in a set of 1000 samples with a much smoother curvature distribution while keeping the “natural” nature of the individual spectra.
1000 samples may be too high for some applications, so the authors attempted to generate a much smaller set of 17 mathematically regular spectra which yield similar results to the set of 1000 hybrid samples. The subcommittee is proposing this set (named “HL17”) to the full technical committee for consideration.
Image Fusion for Optimizing Gamut Mapping
There are various methods for mapping colors from one gamut to another (typically reduced) gamut. Each method works well in some circumstances and less well in others. Previous work applied different gamut mapping algorithms to an image and automatically selected the one that generated the best image based on some quality measure. The authors of this paper tried to see if this can be done locally – if different parts of the same image can be productively processed with different gamut mapping algorithms, and if this produces better results than using the same algorithm for the whole image.
Their approach involved mapping the original with every gamut mapping algorithm in the set, and generating structural similarity maps for each algorithm. This was followed by generation of an index map for the highest similarity at each pixel. Each pixel was mapped with the best algorithm, and the results were fused into one image.
Simple pixel-based fusion results in artifacts, so the authors tried segmentation and bilateral filtering. Bilateral fusion ended up producing the best results, then segmented fusion, followed by picking the best overall algorithm for each image, and finally the individual algorithms. So the fusion approach was promising in terms of visual quality, but computation costs are high. They plan to improve this work as well as applying it to other imaging problems like locally optimized image enhancement and tone mapping operators.
Image-Adaptive Color Super-Resolution
The problem is to take multiple low-resolution images and estimate a high-resolution image. There has been work in this area, but challenges remain, especially correct handling of color images. The authors treated this as an optimization problem with simple constraints (individual pixel values must lie in the 0 to 1 range, warping and blurring must preserve energy over the image, as well as some assumptions on the possible properties of blurring and warping). They add a novel chrominance regularization term to hand color edges properly. The results shown appear to be better than those achieved by previous work.
Two-Field Color Sequential Display
Color-sequential displays mix primaries in time rather than in space as most displays do. Since the color filters are removed (replaced by a flashing red-green-blue backlight), the power efficiency is increased by a factor of three. However, very high frame rates are needed (problematic with LCD displays) and the technique is prone to color “breakup” artifacts.
This paper proposes a display composed of two temporal fields instead of three, to reduce flicker. Optimal pairs of backlight colors are found for each screen block to reduce color “breakup”. This is implemented via an LCD system with local RGB LED backlights. The authors built a demonstration system and experimented with various images. Most natural images are OK, but some man-made objects look bad. The number of segments can be increased, reducing the errors but not eliminating them. They were able to achieve reasonable results with 576 blocks.
Efficient Computation of Display Gamut Volumes in Perceptual Spaces
This paper discussed fast methods to compute gamut volume – the motivation is for use in optimizing display primaries (three or more). I’m not sure how important it is to do this fast, but that is the problem they chose to solve.
Computing gamut volume of three-primary displays in additive spaces is very easy (just the magnitude of the determinant of the primary matrix). The authors want to compute the gamut volume in CIELAB space, which is more perceptually uniform but has non-linearities which complicate volume computation. They found a way to refactor the math into a relatively simple form based on certain assumptions on the properties of the perceptual space. For three-primary displays in CIELAB this reduces to a simple closed-form expression.
Computing gamut volume for multi-primary displays is more complex. The authors represent the gamut as a tessellation of parallelepipeds. To determine the total volume in CIELAB space they solve a numerical problem in a way similar to Taylor series.
Appearance-Based Primary Design for Displays
LED-backlit LCD displays have recently entered the market. They have many advantages over traditional LCD displays: higher dynamic range, high frame rate, wider color gamut, thinner, more environmentally friendly, etc. There are two main types of such displays. RGB-LED-based LCD displays can potentially deliver more saturated primaries (and thus wider color gamuts) due to the narrow spectral width of the LEDs used, while white-LED-based LCD displays might provide high brightness and contrast but smaller gamuts by using high efficiency LEDs in combination with the LCD-panel RGB filters.
The choice between the two is primarily a tradeoff between saturation and brightness. However, the two are linked due to the Hunt effect, which causes perceived colorfulness to increase with luminance. The Stevens effect (perceived contrast increases with brightness) is also relevant. Could these effects lead to a win-win (increased perceived saturation and contrast, as well as actual brightness) even if actual saturation is sacrificed?
The authors investigated two possible designs. One adds a white LED to an RGB LED backlight (RGBW LED backlight). The other keeps the RGB LED backlight, and adds a white subpixel to the LCD (RGBW LCD). The RGBW LED backlight design proved to work best, with an increased white up to 40% providing increased colorfulness as well as brightness. The RGBW LCD white-subpixel design always decreased perceived colorfulness regardless of the amount.
This was determined via a paired comparison experiment. It is interesting to note that neither CIELAB nor CIECAM02 models predicted the result for the RGBW LED backlight – CIELAB predicted that colorfulness would decrease, while CIECAM02 predicted it would increase but not the right amount. In the case of the RGBW LCD subpixel design, both CIELAB and CIECAM02 predicted the results.
HDR Video – Capturing and Displaying Dynamic Real World Lighting
This paper (by Alan Chalmers, WMG, University of Warwick) described the HDR video pipeline under development at the University of Warwick. It includes a Spheron HDRv camera (capable of capturing 20 f-stops of exposure at full HD resolution and 30 fps), NukeX and custom dynamic IBL (image-based lighting) software for post-production, various HDR displays (including a 2×2 “wall” of Brightside DR37-P HDR displays), and a specialized HDR video compression algorithm (for which they have spun off a company, goHDR).
Prof. Chalmers made the case that the 16 f-stops which traditional film can acquire is not sufficient, and showed various examples where capturing 20 f-stops produced better results. He also discussed the recently begun European Union COST (Cooperation in Science and Technology) Action IC1005-7251 “HDRi” which focuses on coordinating European HDR activity and proposing new standards for the HDR pipeline.
High Dynamic Range Displays and Low Vision
This paper was presented by Prof. James Ferwerda from the Munsell Color Science Lab at the Rochester Institute of Technology. Low vision is the preferred term for visual impairment. It is defined as the uncorrectable loss of visual function (such as acuity and visual fields). Low vision (caused by trauma, aging, and disease) affects 10 million people in the USA, and 135 million people worldwide.
HDR imaging offers new opportunities for understanding low vision. This paper describes two projects: simulating low vision in HDR scenes, and using HDR displays to test low vision.
The framework behind tone reproduction operators (which simulate on an LDR display what an observer would have seen in the HDR scene) can be adapted to simulate an impaired scene observer instead of a normal-vision one. Aging effects (such as increased bloom and slower adaptation) can also be simulated.
The importance of using HDR displays to test vision comes from the fact that people with low vision have problems in extreme (light, dark) lighting situations, such as excessive glare or adaptation issues. In addition, there are theories that changes in adaptation time can be good early predictors of retinal disease. However, standard vision tests use moderate light levels so they are not capable of identifying adaptation or other extreme-lighting-induced issues.
Before experiments could be started on the use of HDR displays for vision testing, the NIH (very reasonably) wanted to ensure that these displays could not cause any damage to the test subjects’ vision. Damage caused by light is called “phototoxicity” and can be related to either extremely high light levels in general, or more moderate levels of UV or even blue light. Blue light has recently been identified as a hazard, especially to people with retinal disease. The International Committee on Non-Ionizing Radiation Protection (ICNIRP) has established guidelines for safe light exposure levels, including blue light.
The authors estimated the phototoxicity potential of HDR displays, using the Brightside/Dolby DR37-P as a test case. At maximum brightness, they computed the amount of light which would reach the retina, with the ICNIRP “blue light hazard” spectral filter applied. The result was 4 micro-Watts; since the ICNIRP limit for unrestricted viewing is 200 micro-Watts of blue light, there appears to be no phototoxicity issue with HDR displays. Another way of looking at this: to reach the ICNIRP limit, the display would have to produce the same luminance as a white paper in bright sunlight: 165,000 cd/m2 (for comparison, the DR37-P peak white is about 3000 cd/m2 and the current Dolby HDR monitor – the PRM-4200 – peaks at 600 cd/m2).
Appearance at the Low-Radiance End of HDR Vision: Achromatic & Chromatic
This paper (by John J. McCann, McCann Imaging) studies how human vision works at the low end, close to the absolute threshold of visibility. In particular, does spatial processing change? There are a lot of physiological differences between rods and cones – spatial distribution, wiring, etc., so it might be expected that spatial processing would differ between scotopic and photopic vision. A series of achromatic tests designed to demonstrate various features of spatial vision processing were tested in extreme low-light conditions. The result was exactly the same as in normal light conditions – it appears that spatial processing does not change.
The authors also did experiments with low-light color vision. Although rods by themselves cannot see color (which requires at least two different detector types with distinct spectral sensitivity curves), they can be used for color vision when combined with at least one cone type. In particular, light which has enough red to activate the L cones (but not S or M) and enough light in the right wavelengths to activate the rods will enable dichromatic color vision using the rods and L cones (firelight, a 2000° K blackbody radiator, has the best balance of spectral light for this). This enabled comparing the spatial component of color vision in low-light and normal-light conditions. As before, the observers saw all the same effects, showing that spatial processing was the same in both cases.
Hiding Patterns with Daylight Fluorescent Inks
This paper describes the use of daylight fluorescent inks (which absorb blue & UV light and emit green light, in addition to reflecting light as normal inks do) to create patterns inside arbitrary images which are invisible under normal daylight but appear with other illuminants.
The authors looked at different combinations of regular and fluorescent inks and calculated the gamut for each one. The areas of the gamut that are metameric (under D65) with regular inks can be used to hide patterns. They also calculated proper ink coverage amounts needed to match the fluorescent and regular inks under D65.
Optimizing HANS Color Separation: Meet the CMY Metamers
The Halftone Area Neugebauer Separation (HANS) approach presented at last year’s CIC offered opportunities for optimizing various aspects of the printing process. This year’s paper further explores some of those possibilities.
Regular color halftoning works by controlling the coverage of each colorant, e.g. cyan, yellow, and magenta in a CMY system. HANS extends this to controlling the coverage of each possible combination of colorants (these are called the Neugebauer primaries). For example, a CMY system has 8 Neugebauer primaries: white (bare paper), cyan, magenta, yellow, blue (combination of cyan & magenta), green (combination of cyan & yellow), red (combination of magenta & yellow), and black (combination of all three colorants). Trichromatic color printing (e.g. CMY) has only one halftone pattern for each color in the available gamut. Extending this to more primaries (as HANS does) allows for metamers – different halftone patterns that can obtain the same color, and thus optimization opportunities.
With CMY inks, the authors found that the ink use varied quite a bit depending on the metamers used, indicating that a significant amount of ink could be saved even with such a limited ink set. HANS can save even more ink when used with more typical ink sets which include four or more inks.
Local Gray Component Replacement Using Image Analysis
Gray Component Replacement (GCR) refers to the practice of saving ink in a CMYK printing system by replacing amounts of CMY by similar amounts of K. GCR advantages include deeper blacks, ink savings, and increased sharpness of small details. But it does have one large drawback – it can cause excessive graininess (visible noise) in certain cases. This causes most printer manufacturers to use it very lightly, if at all.
This paper seeks to exploit the fact that noise perception depends on content activity. Noise is quite visible in smooth areas, not so visible in “active” areas. A GCR scheme that adapts to the content of the image has the potential to realize significant GCR benefits without causing noticeable noise.
One problem with this approach is that existing methods to find the “active” areas of the image either do not take account of the properties of the human visual system, require too much computation, or both. The authors’ insight was that cosine-based compression schemes have been heavily designed to exploit the properties of the human visual system, and can be adapted to this application. They do a DCT (discrete cosine transform) of the image and run it through a weighting matrix originally designed for JPEG quantization. The authors put the result through a mapping table (values based on experimentation) to find the desired black ink amount.
A Study on Perceptually Coherent Distance Measures for Color Schemes
This is related to the NPAR 2011 paper “Towards Automatic Concept Transfer”, which allowed for transferring color palettes associated with a concept (such as “earthy”) to images. This paper attempts to find an automated way to assess similarity between color palettes so that different palettes associated with the same concept can be explored.
Most of the existing color metrics either require the compared palettes to have the same number of colors, are dependent on color ordering, or both. The authors came up with a metric called “Color-Based Earth-Mover’s Distance”, which performed well.
Effects of Skin Tone and Facial Characteristics on Perceived Attractiveness: A Cross-Cultural Study
The aim is to study the impact of observer’s cultural background on the perception of faces. Earlier studies showed that slightly more colorful skin tones are more attractive than measured ones, and that larger eyes are rated more attractive. Some cultural background effects were found as well.
The authors took a set of face images, and manipulated skin color, eye size, distance between eyes, and nose length. The resulting images were shown to sets of both British and African observers.
Conclusions: Observers were more sensitive to changes in facial characteristics for faces of their own ethnic group than to those of other ethnic groups. British observers preferred skin colors of higher chroma, with a hue angle of about 41 degrees. African observers had preferences for more reddish, higher chroma faces.
Color Transfer with Naturalness Constraints
The motivation of this work (done at Hewlett-Packard) was to make it easier for untrained users to make pleasing collages by assembling photographs on top of a themed background. In many cases the colors of the selected photographs do not match each other or the background. The photographs may come from different cameras, and some might be downloaded from the web. Some of the images might even be drawings or paintings.
The idea is to use color transfer to make the various images match more closely each other as well as the background. There has been previous work in color transfer, but none of it fit the constraints of this specific application. The users often know what the original photo looked like and will not accept drastic changes. They are also not willing to do extensive manual tinkering to get good results.
Naturalness is important – colors in each image should be modified as a whole, familiar object colors (esp. skin) should remain “plausible”, and the white point should not change too drastically. The authors used a color adaptation model, constraining the color changes to those consistent with adapted appearance under natural illuminants. For each image, they find an estimated illuminant (defined by its CCT and luminance). This is not white point balancing or illuminant estimation; most consumer photos are white-balanced already. The idea is to describe the collective characteristics of the image’s color with parameters that are amenable to transfer to another image through an illuminant adaptation model. A simple Bayesian formulation was used to find the CCT and luminance, based on based on a set of 81 simulated illuminants (9 CCTs and 9 log-spaced luminance levels), applied to 170 object surfaces from Vrhel’s natural surface reflectance database (ftp://ftp.eos.ncsu.edu/pub/eos/pub/spectra/). Once a CCT & luminance is found for each image, color transfer was handled as a chromatic adaptation process. Any model would work; the authors used RLAB since they found it best suited to their needs.
If the photos differ greatly in tone as well as color, they can also do a “tone transfer” process using similar methods.
The results were very effective in improving the user-created collage while still keeping the individual photographs natural-looking and recognizable.
The Influence of Speed and Amplitude on Visibility and Perceived Subtlety of Dynamic Light
Modern light sources (such as LEDs) enable continuously varying color illumination. Users have expressed a desire for this, but prefer slow/small changes over fast/large ones. For residential lighting, it’s important that the dynamic lighting not distract or be obtrusive. In other words, the changes need to be subtle.
This work explores what “subtle” means in the context of dynamic lighting. Do people understand subtlety in the same way? Can you measure a subtlety threshold that is distinct from visibility?
The authors tried dynamic lighting with different amplitude and speed of changes, and asked people whether they considered the lighting subtle, and whether they could see it happening at all.
The dynamics were considered to be subtle if they were slow, smooth, and over a narrow hue range. People seem to agree on what “subtle” means; a subtlety threshold is distinct from the visibility threshold and can be measured.
Blackness: Preference and Perception
In practice, printed black patches vary in hue. This study attempted to determine among a range of blacks of varying hues, which blacks are most preferred and which are considered to be “pure black”.
Used various hues from Munsell color system – three neutrals (with Value 0, 1 and 2 – N0, N1, N2) and the “blackest” version (Value 1 and Chroma 2) of the midpoint of each Munsell hue (5R, 5YR, 5Y, 5GY, 5G, 5BG, 5B, 5PB, 5P, 5RP), for a total of 13 patches. The patches were presented to the observers in pairs against a neutral grey background.
Blackness: N0 (true black) was perceived to be closest to pure black. N1 and 5PB were in second and third places. 5R, 5RP and 5Y were considered to be the least pure black. There was little difference found between UK and China populations, or between genders.
Preference: On average 5B, 5PB and 5RP were most preferred. 5GY, 5Y and 5YP were the least preferred. Here there were some differences between Chinese and UK observers, and quite large differences based on gender.
Summary: blackness perception is not strongly linked to nationality and gender, but preference among black colors is. Observers appeared to have a strong preference for bluish blacks and purplish blacks over achromatic and even pure blacks.
Evaluating the Perceived Quality of Soft-Copy Reproductions of Fine Art Images With and Without the Original
This study was done as part of a project to evaluate the perceived image quality of fine art reproductions. The reproductions were printed on the same printer but based on digital scans done at different institutions using different methods. The goal was to see how the presence or absence of the original for comparison effects how people rank the different reproductions.
Two experiments were performed. One under controlled conditions in a laboratory, ranking the different reproductions of each artwork both with and without the original for comparison. The second study was done via the web, in uncontrolled conditions and without the originals.
For the controlled experiment with the original, the subjects were shown the images in pairs and asked to click on one more closely matching the original. The experiments without the original (both in the lab and on the web) were also based on pairs, but the subjects were asked to click on the image they preferred.
In the case of the experiment with the original, the subject’s rankings corresponded closely to measured color differences between the original and reproductions. In the two experiments done without the original, there was no such correlation.
There were low correlations between the results of the controlled experiments done with and without the originals, implying that preference is not strongly linked to fidelity vs. the original. However, there was a strong correlation between the controlled and web experiments done without the originals, implying that testing conditions do not significantly impact the preference judgment of images.
In the web-based experiment, the subjects were also asked to click on the parts of the picture that most influenced their choice. Users tended to click on specific objects, especially faces.
Scanner-Based Spectrum Estimation of Inkjet Printed Colors
Fast and accurate color measurement devices are useful for the printing process, but such devices are expensive. Scanners are cheap and some printing presses even have them integrated into the output paper path, but they are not accurate measuring devices.
This paper describes a method for using knowledge about both the printing process and the scanner characteristics to estimate spectral reflectances of printed material based on scanner output. The scanner response is estimated by scanning various patches with known spectra. Then the spectra of the printed materials are inferred from their scanned pixel values, the scanner response, and a physical model of the printing technology. The method yielded fairly accurate results.
Evaluating Color Reproduction Accuracy of Sterero One-shot Six-band Camera System
Multi-band imaging can be a good solution for accurate color reproduction. Most such systems are time-sequential and cannot capture moving objects or handle camera motion. Several multi-band video systems have been developed, but they all require expensive optical equipment.
The proposed system uses a stereo six-band camera system to acquire depth information and multi-band color information (for accurate color reproduction) simultaneously for moving scenes. The system is comprised of two consumer-model digital cameras, one of which is fitted with an interference filter which chops off half of each of the sensor R, G and B spectral sensitivity curves. Care needs to be taken during processing since the parallax between the two camera images may affect color reproduction. There are also issues relating to gloss, shade, lighting direction, etc. that need to be resolved.
The authors use a thin-plate spline model to deform the images to each other for registration purposes. When a corresponding point cannot be found, they use only the unfiltered image for color reproduction of that pixel. The authors evaluated color accuracy with a Macbeth ColorChecker chart. The colorimetry was accurate and they even got some information on the reflectance spectra.
Efficient Spectral Imaging Based on Imaging Systems with Sensor Adaptation Using Tunable Color Pixels
Current multispectral cameras come in two main types.
- Time multiplexing multispectral camera – for static scenes only (high quality capture can take up to 30 seconds).
- Color filter array (CFA) with 6 or more filter types, many cameras do this today (Canon Expo 50 megapixel camera). But this causes a large loss of resolution.
This paper discusses the use of tunable imaging sensors with spectral sensitivity curves that can be dynamically varied on a pixel-by-pixel bases. One such sensor is the transverse field detector (TFD), which exploits the fact that the penetration depth of photons into Silicon depends on their wavelength. The TFD uses a transverse electrical field to collect photons at various penetration depths.
The authors simulated a TFD-based system with several selectable per-pixel sensor capture states. The idea is to analyze where the primary spectrum transition is happening and ensure the sensor has a sensitivity peak there. The system has an initial stage where the derivatives of a preview image are fed to a support vector machine (SVM) that has been trained on a set of images for which the ground truth reflectance spectra were known.
For evaluation, the authors simulated spectral capture systems of all three types – time multiplexing, CFA, and tunable sensor. They found that there is a big improvement on the tunable sensor when going from only one possible state per pixel to two, but no improvement when further increasing the number of states. For scientific applications, the tunable system did slightly outperform the other two in terms of accuracy, and gave similar results for consumer images (the authors suspect that with a better choice of sensor states an improvement would be possible here too). More importantly, the tunable sensor technique doesn’t suffer from the primary drawbacks of the other two (reduced resolution in the case of CFA, multiple-shot requirement in the case of time multiplexing).
A New Approach for the Assessment of Allergic Dermatitis Using Long Wavelength Near-Infrared Spectral Imaging
Hyperspectral imaging can help with diagnosis of allergic dermatitis, which is one of the most common skin diseases. Near infra-red (IR) can penetrate under the skin and show what is happening there. The author’s system showed early stages of both disease and treatment success before visible changes were apparent. It could also clearly discriminate irritation from allergic reaction (which is very difficult from visual inspection), even distinguishing between different types of allergic reactions.
Saliency-Based Band Selection for Spectral Image Visualization
Visualizing multispectral data on an RGB display always involves some data loss, but the goal is to show the most important data while keeping a somewhat natural-looking image. The authors of this paper used saliency (visual importance) maps (previously used to predict where people would spend most time looking in an image), to find the most important three channels.
Spectral Estimation of Fluorescent Objects Using Visible Lights and an Imaging Device
Many everyday materials (for reasons of safety, fashion or others) contain fluorescent substances. The principle of fluorescence is that the material absorbs a photon, goes from ground state to a highly excited state, slowly goes to a less excited stage and finally jumps back to ground state, releasing a less-energetic (lower frequency) photon.
Standard fluorescent measurements involve either two monochromators (expensive and only usable in laboratory setups) or UV light and a spectro-radiometer (hard to estimate accuracy; also, the use of UV lights poses a safety problem).
The authors of this paper propose a method for estimating the spectral radiance factor of fluorescent objects by using visible lights and a multispectral camera. Measurements assume that the fluorescence is equally emitted along all wavelengths lower than the excitation wavelength, which is a pretty good assumption for most fluorescent materials. Analysis of the results showed them to be of high accuracy.