Revisiting Color Efficient Coding through Material Perception

An essential objective of early visual processing is to handle the redundant inputs from natural environments efficiently. For instance, cone signals from natural environments are highly correlated across cone types, indicating channel redundancy. Early visual processing transforms the signals into color and luminance information, such as principle component analysis, and thus achieves efficient and decorrelated representations of natural scenes [1–3]. Building on these findings, previous research has investigated the effect of color on visual tasks such as object recognition using grayscale conversion, which separates luminance from color [4, 5]. However, recent work suggests that when focusing on object materials, color and luminance remain highly redundant due to complex optical properties [6, 7]. Although this finding indicates that there may be a more efficient decomposition of signals, the specific algorithms remain unknown. This study derives that a classic computer graphics algorithm, the median cut [8], offers a novel approach to enhance visual processing efficiency while capturing diagnostic features to separate material from object geometry information (Fig. 1a and 1b). Human behavioral experiments show that color reduction based on the algorithm disturbs material classification while preserving object recognition (Fig. 1c). These findings suggest that object geometric structures are available only from low-bit information. Finally, considering material information can be estimated from summary statistics of image sub-spaces, this study suggests an efficient decomposition of input color images (Fig. 1d).