Super realistic rendering! Unreal Engine technology expert explains the global illumination system Lumen

Real-time global illumination (Real-time GI) has always been the Holy Grail of computer graphics.

Over the years, the industry has also proposed various methods to solve this problem.

Commonly used method packages constrain the problem domain by utilizing certain assumptions, such as static geometry, rough scene representation or tracking rough probes, and interpolating lighting between the two.

#In Unreal Engine, the global illumination and reflection system Lumen technology was founded by Krzysztof Narkowicz and Daniel Wright.

The goal was to build a solution that was different from its predecessors, able to achieve unified lighting and a baked-like lighting quality.

Recently, at SIGGRAPH 2022, Krzysztof Narkowicz and the team talked about their journey of building Lumen technology.

Software Ray Tracing - Height Field

The current hardware ray tracing lacks powerful GPU computing power support. We don't know how fast hardware ray tracing will be, or even if new generation consoles will support it.

Therefore, the software ray tracing method is used. It turns out to be a really useful tool for scaling or supporting scenes with a lot of overlapping instances.

Software ray tracing provides the possibility to use a variety of tracking structures, such as triangles, distance fields, and surfels. , or heightfields.

Here, Krzysztof Narkowicz abandons studying triangles and briefly studies surfels, but for those geometries that require considerable density to represent, updating or tracking surfels is Quite expensive.

After initial exploration, height fields were the most suitable as they map well into the hardware and provide surface representation and simple continuous LODs.

Because we can use all POM algorithms, such as min-max quadtree, its tracking speed is very fast.

Additionally, multiple height fields can represent complex geometries, similar to rasterizing bounding volume hierarchies.

It is also very interesting to think of it as an acceleration structure of surfels. A single texel is a surfel restricted by a regular grid. .

In addition to the height field, Lumen has other properties, such as albedo or lighting, so that it can calculate the lighting every time.

In Lumen, the developer named this complete decal projection with surface data Cards, which is the capture position.

Rasterized triangle

Raymarched cards (height field)

Raymarched cards are too slow for each card in the scene. Therefore, a card acceleration structure is needed, and the developers chose a 4-node BVH. It is built for a complete scene, with every frame on the CPU and uploaded to the GPU.

Then in the trace shader we will do a stack based traversal and dynamically order the nodes so that the closest nodes are traversed first.

BVH Debug View

Capture Position

The trickiest part here is how to place the height field so that the entire net is captured grid. Krzysztof Narkowicz said, "One of the ideas is based on the global distance field of the GPU. Every frame we will trace a small set of primary rays to find rays that are not covered by the card.

Next, For each undiscovered ray, we will traverse the global distance field using surface gradients to determine an optimal card orientation and range, resulting in a new card.

Capture position of the global distance field

On the one hand, it is confirmed that it can generate cards for the entire merged scene without having to grid to generate cards. On the other hand, it turns out to be quite finicky in practice, as it will produce different results every time the camera moves.

Another idea is to put each Mesh cards as a mesh import step. Do this by building a geometric BVH where each node will be converted to N cards.

As follows:

Rasterized triangle

Ray Step Card (High Field)

##Card Position View

This method encountered problems in finding a good location because the BVH node was not a good proxy for placing the card.

So, the researchers Another idea was proposed: follow the UV unwrapping technique and try to cluster surface elements.

Since they were dealing with millions of polygons provided by Nanite, they replaced the triangles with Surface elements.

At the same time, they also switched to a less constrained free guide card to try to match the surface better.

Freely Guided Card Position

By trying this method, this method works very well for simple shapes, but before converging to more Problems arose with complex shapes.

Finally, Narkowicz switched back to axis-aligned cards, but this time generated by bin clusters and per-mesh.

The unique properties of tracking height fields can also achieve cone tracking.

Cone tracking is useful for reducing noise Very efficient because a pre-filtered single cone trace represents thousands of individual rays.

Ray Tracing

Cone Tracking

For each card, the developers also store a complete pre-filtered mip-map chain with surface height, lighting and material properties.

When tracing, select the appropriate stepping ray based on the cone footprint and perform ray tracing on it.

Tracking of edges without jams and edges with jams

Merge scene representation

Tracing large numbers of incoherent rays in software is very slow. Ideally, a single global structure would be used instead of multiple height fields.

When the cone footprint gets larger and larger, an exact representation of the scene is not actually needed and can be replaced by a more approximate representation for faster speeds.

A more complex scene with dozens of cards to trace each ray

The first successful method was to implement pure voxel cone tracing, where the entire scene is voxelized at runtime, as in the classic "Interactive Indirect Illumination Using Voxel Cone Tracing" article accomplish.

Rasterized triangle

Ray Step Card (Height Field)

##Voxel Cone Tracking

The ray stepping card continues with voxel cone tracking

And this The main drawback of the method is leakage due to excessive fusion of scene geometry, which is especially noticeable when tracking coarse low-mapping.

The first technique to reduce image leakage is to track the global distance field and only sample voxels close to the surface. During the sampling process, opacity accumulates as the sampling range expands, and when tracking is stopped, the opacity will reach 1. This always performs accurate sampling near the geometry to reduce graphics leakage.

The second technique is to voxelize the interior of the mesh. This greatly reduces leakage at thicker walls, although this can also cause some excessive occlusion.

Other experiments include tracking sparse voxel blobs and voxels per face transparency channel. The purpose of both experiments is to solve the ray direction voxel interpolation problem, i.e., an axis-aligned solid wall will become transparent for rays that are not normal to the wall.

Voxel Bit Brick stores one bit per voxel in an 8x8x8 brick to indicate whether a given voxel is empty. A two-stage DDA algorithm is then used for ray stepping. Voxels with transparent surfaces are similar but have the same DDA and increasing transparency along the ray direction. It turns out that both methods are not as good at representing geometry as the distance domain and are quite slow.

Voxels with transparency

The earliest way to track the merged representation was to use a global distance field and a global per-scene card The shading hits perform cone tracking. That is, iterate through a BVH, find out which cards in the scene affect the sampling points, and then sample the moderate sliding level of each card based on the tapered footprint.

This article abandoned this method because it was not considered to only use it to represent the far-field trajectory, but was regarded as A direct replacement for high field light stepping. Somewhat ironically, this abandoned approach came closest to the solution we ultimately arrived at two years later.

First demo

At this point, we can already produce some pretty good results:

Despite this, we encountered a lot of graphics leakage issues, and in this simple scene, the performance was less than ideal even on a high-end PC GPU.

In order to solve the leakage problem to process more instances, processing is completed in under 8 milliseconds on PS5. This demo was a real catalyst.

Compared with previous solutions, the first and biggest change is the replacement of height field tracking with distance field tracking.

To mask the health point, insert the ray of the health point from the card, because the distance field has no vertex properties, this way, the uncovered area will only cause energy loss, not leakage.

For the same reason, change voxel cone tracing to global distance field ray tracing.

At the same time, we have also made many different optimizations and time-divided different parts of Lumen through caching solutions. It's worth noting that without cone tracking we have to denoise and cache the tracking more aggressively, but again that's a long and complicated story.

This is the final result after we sent the first demo, which was consistently under 8ms on PS5, including updates to all shared data structures such as the global distance field. The current performance is even better. For example, the completion time of the latest demo is close to 4 milliseconds, and the quality has also been significantly improved.

End

In short, this article has completely rewritten the entire Lumen, and there are many different ideas that have not been implemented. On the other hand, some things are repurposed. Just like originally we used cards as a tracking representation, but now they are used as a way to cache various calculations on the surface of the mesh. Similar to software tracking, it started as our main tracking method, mainly cone tracking, but ended up being a way to scale down and support complex heavy scenes with a lot of overlapping instances.

The above is the detailed content of Super realistic rendering! Unreal Engine technology expert explains the global illumination system Lumen. For more information, please follow other related articles on the PHP Chinese website!