ImageNet in the material world, a large-scale 6-dimensional material real-shot database OpenSVBRDF released｜SIGGRAPH Asia

In the field of computational graphics, material appearance depicts the complex physical interaction between real objects and light, which can usually be expressed as a bidirectional reflection distribution function (Spatially-Varying Bidirectional Reflectance Distribution Function, abbreviated as SVBRDF) that changes with spatial position. . It is an integral component of visual computing and has extensive applications in fields such as cultural heritage, e-commerce, video games, and visual effects.

In the past two decades, especially after the popularity of deep learning, the demand for high-precision and diverse digital material appearances has been increasing in academia and industry. However, due to technical challenges, it is still very difficult to collect large databases, and the number of publicly available real-shot databases of material appearance is currently very limited.

To this end, the research team of the National Key Laboratory of Computer-Aided Design and Graphics Systems of Zhejiang University and Hangzhou Xiangxin Technology Co., Ltd. jointly proposed a new integrated system using Robust, high-quality and efficient capture of planar anisotropic material appearance. Using this system, the research team built the OpenSVBRDF public material database.

## 图 1: Some materials in the OpenSVBRDF database display. Each row belongs to the same material category.

This is the first large-scale measured database of 6-dimensional SVBRDF, with a total of 1,000 high-quality planar samples and a spatial resolution of 1,024×1,024 ,

is equivalent to more than 1 billion measured BRDF, covering 9 categories including wood, fabric and metal.

Database homepage: https://opensvbrdf.github.io/

Currently, the database is open to non-commercial applications Totally free. You only need to submit basic information to apply for an account on the website. After passing the review, you can directly download relevant data and codes including GGX texture maps. The related research paper "OpenSVBRDF: A Database of Measured Spatially-Varying Reflectance" has been accepted as a long article by ACM SIGGRAPH ASIA 2023 (Journal Track), the top international conference on computer graphics.

Paper homepage: https://svbrdf.github.io/

Technical Challenge

Direct sampling methods make dense measurements of physical materials under different combinations of lighting and viewing angles [Lawrence et al. 2006]. Although this can obtain high-quality and robust acquisition results, it is inefficient and requires high time and storage costs. Another option is a prior knowledge-based reconstruction method that can reconstruct materials from sparse sampled data. Although this improves efficiency, its quality is unsatisfactory when the a priori conditions are not met [Nam et al. 2018]. In addition, although the current SOTA optical path multiplexing technology has achieved high acquisition efficiency and reconstruction quality, the algorithm is not robust enough when dealing with highly complex materials such as brushed metal and polished veneer [Kang et al. 2018].

Figure 2: Representative work of existing material collection research. From left to right they are [Lawrence et al. 2006], [Nam et al. 2018] and [Kang et al. 2018]. Among them [Kang et al. 2018] is the team's early work published in ACM SIGGRAPH in 2018.

Hardware

In order to efficiently scan material appearance, research The team built a nearly half-cube near-field illumination multiplexing device with dimensions of approximately 70cm × 70cm × 40cm. The sample is placed on a clear acrylic plate and can be quickly slid in/out via drawer slides to achieve high throughput rates. The device consists of 2 machine vision cameras and 16,384 high-brightness LEDs. The two cameras capture samples from angles of approximately 90 degrees (primary viewing angle) and 45 degrees (secondary viewing angle). The LEDs are distributed on 6 sides of the device. The self-developed high-performance control circuit is responsible for independent brightness control of each LED and achieves high-precision synchronization of light source projection and camera exposure at the hardware level.

^{# 3: Collect the appearance of the device and the photos of the two perspectives.}

Collection and reconstruction

This system innovatively combines It combines the advantages of the current two popular methods based on network prediction and fine-tuning. It can not only increase the physical collection efficiency through differentiable illumination pattern optimization, but also further improve the quality of the final result through fine-tuning, thus achieving high performance for planar SVBRDF for the first time. Robust, high-quality, and efficient acquisition and reconstruction.

Specifically, to reconstruct the physical sample, the researchers first established a high-precision correspondence between the two camera views by matching dense SIFT features under uniform illumination. For physical acquisition, the lighting pattern is first optimized as part of the autoencoder to achieve efficient acquisition. The autoencoder automatically learns how to reconstruct complex appearances based on measurements from two views and represents the results as intermediate neural representations. Subsequently, the neural expression was fine-tuned by plotting image errors based on photos taken by the main-view camera under 63 equivalent linear light sources to improve the quality and robustness of the final results. Figure 3 shows the processing flow of the entire system. Please see the original paper for more details.

# 图 4: The collection and reconstruction process of the entire system.

Results

The researchers collected and reconstructed a total of 9 Categories, a total of 1,000 samples of appearance, in order to facilitate direct use of the standard physics-based rendering pipeline (PBR), this study also fitted the neural expression to the industry standard anisotropic GGX BRDF model parameters. Figure 5 shows the sub-parameters/properties of the material reconstruction results. Each sample stores 193 raw HDR photos (total size 15GB), intermediate neural representations (290MB), and 6 maps, including texture and transparency maps representing GGX parameters (total size 55MB). The spatial resolution of both neural expressions and texture maps is 1,024×1,024.

##                                                                                                                                                                                                                                                                                     .

In order to prove the correctness of the reconstruction results, the researchers combined the photos from the main perspective (the first row of the figure below) and the neural expression drawing results ( The second row of the figure below) is compared. Quantitative error (expressed as SSIM/PSNR) is noted at the bottom of the plot. As can be seen from the results in the figure below, this system achieves high-quality material reconstruction (SSIM>=0.97, PSNR>=34db).

## 图 6: Comparison of actual photos and nerve expression drawing results in the main perspective.

#In order to further prove the generalization of the reconstruction results in the viewing angle domain, the researchers used photos taken from two viewing angles illuminated by point light sources and used The results of GGX fitting parameter plotting were compared, verifying the correctness of the reconstruction results across views.

^{##                                                                                                                                                                                                 .}

The researchers also demonstrated the application of the database in three aspects: material generation, material classification and material reconstruction. Please refer to the original paper for specific details.

^{: 8: Use OpenSVBRDF to train Materialgan to achieve material generation and interpolation.}

#                                                                                                                                                                                                          .

# BRDF reconstruction quality.

Outlook

Researchers will work to expand existing Database, adding material samples showing diverse appearances. In the future, they also plan to build a large-scale, high-precision measured object database that includes both material appearance and geometric shape. In addition, researchers will design a public benchmark in the direction of material estimation, classification and generation based on OpenSVBRDF, and provide solid data guarantee to promote the future development of related research through objective and quantitative standard testing.

The above is the detailed content of ImageNet in the material world, a large-scale 6-dimensional material real-shot database OpenSVBRDF released｜SIGGRAPH Asia. For more information, please follow other related articles on the PHP Chinese website!