If you are on a Govt. or Corporate network this site and the content may not display or playback correctly. Reload the page if you have any issues.
AC/R - Technical Study
Albedo COlor and Roughness
Breaking down what drives premium quality in our textured models begins with distinguishing two values:
- Ensuring captured image data is at all usable
- Providing the type data that pushes extreme realism
The more stuff in any environment, the more that lighting becomes a problem. It’s not only impractical to manage multiple lights used to capture every surface of all that stuff, each light projecting new shadows and specular reflections spells trouble. The ideal solution simplifies the array of challenges. All-in-one AC/R delivers:
- Mobility – small form factor implies speed, productivity, and scalability
- Power – a blinding 286,000 lumens means low noise
- Relighting – imagery is natively delit
- Realism – optimized empirical data complimented by AI inferred
The following breakdown defines some terms and clarifies basic concepts.
Polarized Lighting
Under the Hood
Co-polarization: Polarized light (strobe plus filter) reflecting off surfaces passes through a second polarizer covering the lens, the polarization angle between both filters runs in parallel. The resulting exposure contains both color and specular information that is substantially shadow-free.
Cross-polarization: Polarized light reflecting off surfaces passes through a second polarizer covering the lens, the polarization angle between both filters is 90°. Nearly all specular reflections are filtered out, leaving pure diffuse color.
Diffuse/Specular Separation
The rapidly evolving reality capture industry suffers from confusing terminology. Albedo is defined as the difference between incident and reflected light (of planets), but the reflected light from a surface can be either highly diffuse/matte or specular/shiny or anything in between. Whereby albedo color describes the pure diffuse color hiding beneath any shine, albedo roughness describes the smoothness (or lack thereof) of that material.
The cross-polarized image in the above slider delivers albedo color, so here’s how we arrive at albedo roughness.
Albedo roughness requires us to isolate specular information from color+specular in the co-polarized image. Since the cross- and co-polarized image pairs share identical views, we simply subtract one from the other to tease out albedo roughness information, which here again can range from highly matte to super glossy, depending on the smoothness of microfacets.
Geometry
Scene reconstruction uses the cross-polarized imagery. The original high-poly model (left) began with a whopping 747 million tris (overly dense wireframe omitted). It was then decimated to 1 million tris (right). With Nanite in Unreal 5 handling billions of tris, conventional workflow for optimization is changing, potentially obviating normal maps for most users. As will soon be clear why, normal maps serve AC/R in a special way. For now, just remember that during decimation the SfM stores the detail from the high-poly mesh in normal maps, which we’ll return to further down.
Albedo Color
The same color information used to produce geometry is repurposed for base color texture maps. The above slider illustrates the key advantage of shadow-free, empirical albedo color (right). The exaggerated low roughness and long (virtual) shadows to the left illustrate how a material’s color remains largely hidden in the shadows and beneath specular reflections.
Look once more at the slider above comparing co- and cross-polarized source photography. Note in the co-polarized view how washed out the color is. The asphalt looks gray, whereby the cross-polarization filters out the specular reflection to reveal how infused it is with brown sediment.
Take Away – No AI-inferred base color will ever compete with empirical albedo color.
Albedo Roughness
The roughness shader (in Unreal Engine) sees black as low roughness (gloss), sees white as high roughness (matte). It’s said, “The universe is shiny”, the only thing purely matte being a Black Hole. The shininess, or roughness, of everything else sits somewhere along a continuum and is mapped to the roughness parameter in shades of gray. Note the tiny black spots on the right, these attributed to shiny flecks of felspar in the asphalt.
Normal Map
Normal maps use shadow sculpting and specular reflections to create the illusion of shape in micro surfaces. The silhouette of the macro geometry remains unchanged, which isn’t a concern when the focus is on texture.
Obviously, the detail contained in the original 747 million tris mesh packs a ton into the normal map. While that sounds like more than ample resolution in the (perceived) geometry, it’s the high-frequency detail conveyed by albedo roughness that takes texture next level. The isolated specular information is further processed, a) to augment the normal map with higher frequency detail, and b) to drive the roughness map with the highest frequency detail.
Pushing the Limit
Rewiring your brain to grasp what’s won with AC/R is only possible when first you learn about texture maps, which isn’t realistic if you don’t work with them directly. Never mind, use your eyes. As you play with the slider below, spend some time comparing the ambient lit, common photogrammetry on the left to what all is happening on the right via AC/R to inform your brain about the seemingly limitless detail in the asphalt.
Beyond the usefulness of relighting to aesthetics, consider what value AC/R portends for industrial applications. Virtual replicas of infrastructure, AI used for inspections and anomaly detection, the future looks bright for a solution that:
- Ensures usable data captured at scale in highly inaccessible environments
- Doesn’t depend on arbitrary, ambient lighting
- Supports relighting (including fully light-denied environments)
- Optimizes for empirical-based albedo color and albedo roughness
- Leverages AI to push the limits of accurate material reflectance
Only one solution checks each those boxes, and that’s AC/R
Testimonials
See what Professionals think of CAR
