Ambient Occlusion: An Extensive Guide on Its Algorithms and Use in VR
Every year, developers try to raise the bar by making all game components of the highest quality because they directly affect detailing, image realism, and the level of immersion.
Most of what is considered a technical component in a game is connected with light. The various game engines use different methods for transmission of light.
One of these methods is called “ambient occlusion” (AO).
The first method that allowed simulating ambient occlusion in real time was developed by the research and development department of the German company Crytek in the development of graphical components of the CryEngine 2 game engine. “Crysis,” released in the fall of 2007, is the first game on the CryEngine 2 game engine that used the ambient occlusion method called SSAO (Screen Space Ambient Occlusion). The screen space ambient occlusion method works as a post-effect, and it is calculated in the screen space.
Today, this technology is used in all modern game engines. Many people underestimate the ambient occlusion technology, but in its importance, it stands only behind the effect of global illumination (GI). Global illumination and ambient occlusion are complementary effects; their task is to approximate the effect of real radiosity that occurs in real life in the multiple reflections of light and shading.
GI models how light is bounced off of the surface onto another surface, whereas ambient occlusion approximates the shading effect from GI.
This article highlights the meaning of ambient occlusion, the main AO algorithms, and their use in virtual reality (VR).
What Is Ambient Occlusion?
Ambient occlusion is a shading and rendering method used for global background or indirect shading of objects.
This method is used to add more realism to the render because it creates soft global shadows that contribute to the visual separation of objects.
This is a global method, i.e., the brightness value of every point of the object depends on other objects in the scene.
Shadows do not remain black, they dissipate, creating halftones and obscuring those places in space that get less light.
In most cases, ambient occlusion is calculated by constructing a hemisphere of rays emanating from a point on the surface in all directions and then checking them for the intersection with other objects. Rays that reach the background or sky increase the brightness of the surface, while the rays that cross other objects do not add brightness.
As a result, points surrounded by a large amount of geometry are displayed as darker ones, and points with a small amount of geometry in the visible hemisphere are displayed as lighter ones.
This means that the shading effect will be more noticeable between any objects that are located close to each other.
Here is an example of ambient occlusion. As we see, its absence makes the scene flatter and deprives a significant part of the depth that our brain subconsciously perceives.
Thus, the main idea of ambient occlusion is to imitate and calculate shading only in that part of the scene that we directly see.
The Main Ambient Occlusion Algorithms
There are various shading algorithms that provide different degrees of realistic shadow-rendering from global illumination. They differ in cost, the speed of operation, and type of input data.
What is Screen Space Ambient Occlusion (SSAO)?
SSAO (Screen Space Ambient Occlusion) is the least demanding computer graphics technique for approximating the ambient occlusion effect in real time, although the actual quality of the effect can vary greatly from game to game based on how well it was implemented by the developers.
This effect is visible on bends and indentations, but it is not visible on flat surfaces and on convex surfaces because they do not block the light access to each other. This effect is very important because its absence makes the scene look very flat.
Principle of Operation
The screen space ambient occlusion method is pretty simple. When the screen space ambient occlusion effect is built, each point on the screen is examined, and a hemisphere for each point is created. The task of this hemisphere is to determine how much light this point will collect from the outside space or how many shadows from surrounding objects fall over it.
In order to do this, a series of samples are made in the sphere in which it is checked whether the point penetrates some kind of surface or not. If it doesn’t penetrate, then the sky is seen, and this point receives additional light.
In order to check whether this point is shaded, a hemisphere inside the Z-buffer is built, and it is checked if this hemisphere is occluded by the Z-buffer.
For this method, the normal map is used to determine the normal of the surface relative to which the hemisphere is built, but the shading is determined by the ray-matching method. Depth buffer ray-sampling occurs inside the texture in order to find out if this point is shaded in the indicated direction.
When the SSAO effect is used for forward rendering, then the whole scene should be rendered again in order to get normals and depth, full-screen texture in addition to the depth buffer that is obtained during forward rendering.
Therefore, in deferred shading, this process is simpler because there is one of the buffers in which the normals for each point pixel-by-pixel are stored.
Screen space ambient occlusion is very expensive since for a single point, on average, dozens of ray-tracings in different directions are required in order to determine which directions are available for light penetration into this point and which are not.
Therefore, in its pure form, it is almost never used because it is costly.
In this regard, different ways to optimize the decrease in the number of samples are used.
- Interlaced sampling results in artifacts due to the fact that in some pixels, ray-tracing occurs, but in other pixels, it doesn’t occur.
- Downsampling is used, not for the original image, but for a reduced image by half, and then the texture is stretched, but it can seem blurred.
- Temporal accumulation to re-use calculations from previous frames.
Other Methods that Can Save Resources
1. Baking ambient occlusion onto the texture:
In this case, this texture is used as part of the material that is applied to the model. In some models, it is baked as a Green/Blue channel of a Metallic+Roughness texture. If the effect of the screen method is totalized, ambient occlusion from the texture is added to it, then more detailed shading is obtained. This method compensates for the shortcomings of another method – baking AO onto lightmaps. A drawback of this method is that in the baked AO texture, there will be no shading from adjacent objects. In fact, in this texture, only a self-shadowing map can be obtained.
2. Baking ambient occlusion onto lightmaps provided that lightmaps are available
This method is “free” because lightmaps are available, but it decreases the baking speed and adds extra time to the calculation of the ambient occlusion effect. This method is more accurate because at the baking stage, not only nearby surfaces but also larger space can be analyzed in order to understand how the rays move. The main disadvantage is that lightmaps aren’t very detailed, and the size of the texel is 1/10 or 1/20 of a meter. This means that AO baked onto lightmaps will contain only the largest parts, and all minor details will not be baked. Therefore, this approach is suitable only for the transmission of very large details.
SSAO Settings
There are certain parameters that should be taken into account.
- The radius of the sphere or hemisphere within which ray-sampling is performed. The larger the radius is, the less defined the AO effect is.
- The density of the rays directly affects the high cost of the effect.
- The intensity of shading – a multiplier we multiply by.
In some games, this parameter isn’t too noticeable; but in other games, it is made very noticeable in order to highlight inner angles and cavities.
Problems Connected with SSAO
Though SSAO is an effective method, some problems can occur.
Problem 1: Insufficient shading
Here is an example of insufficient shading. The tank bottom should be shaded because it should receive little light, but since the depth buffer has no information about the back face, the screen space effect does not obscure these areas. VXAO gives higher quality, and more accurate shading is obtained since there is information about back faces.
Problem 2: Shading from back faces or any occluded for eye geometry
SSAO does not have information of what is beyond the edge, and during camera motion, shading appears at the edges. Wherever the SSAO touches the edges of the screen, flickering of this post-effect is observed.
Problem 3: SSAO in the screen plane
If this effect is made in the plane of the screen, then it should be taken into account that the radius should change on near and far objects.
SSAO: Pros and Cons
Pros
−It doesn’t depend on scene complexity. It works fully only with the pixels of the final image.
−It works with dynamic scenes.
- It can be easily integrated into any modern graphics pipeline.
Cons
−It is expensive, and it depends directly on the image resolution
– the higher the image resolution is, the more expensive this effect will be, and the more it will be necessary to simplify the algorithm by reducing the number of rays and reducing the radius.
- It is difficult to blur the noise correctly without interfering with depth discontinuities, for instance, object edges.
What Is Horizon Based Ambient Occlusion (HBAO)?
HBAO (Horizon Based Ambient Occlusion) is an algorithm for approximating the ambient occlusion effect introduced in 2008 by NVIDIA as an upgraded SSAO variant. This method gives a better visual result, and it is more expensive than SSAO. Principle of Operation HBAO uses a physically-based algorithm that approximates an integral with depth buffer sampling. There is a certain surface in which the angles are measured, and how much light will be occluded is checked taking into account not only the depth but the direction of the normals of the surface in relation to the world (horizon). Thus, the direction of the surfaces is additionally taken into account in order to obtain a more accurate result.
Pros
−Image quality is higher than the SSAO algorithm.
−It has additional hardware acceleration on NVIDIA video cards.
Cons
−It has an increased performance hit compared to SSAO.
−Rendering HBAO at reduced resolutions causes flickering of the image.
What Is HBAO+?
In comparison to HBAO, HBAO+ doubles the detail level of the AO effect, uses the latest DirectX 11 technologies, and runs much faster. Principle of Operation In order to calculate HBAO+, checkerboard rendering is used. For processing the shading, a part of the previous frame and half of a new one are used. As a result, fewer GPU costs are required, and flickering is removed because the shading is rendered in the original resolution.
Pros
−It requires fewer GPU costs, so it allows rendering the shading at the original resolution without decreasing it.
−It offers a speedy implementation with a far richer, more detailed image.
Cons
−Due to the temporal accumulation from the previous frame, in dynamically changing scenes and sudden camera movements, it can be noticed that the AO effect doesn’t adapt immediately.
What Is High Definition Ambient Occlusion (HDAO)?
HDAO (High Definition Ambient Occlusion) has been developed by the AMD company. This algorithm is based on the Gather4 technology, which collects four texels in one register. Principle of Operation HDAO and HBAO have been generalized to be the same. They have a similar principle of operation – the method increases the number of samples used when calculating the areas that should be darkened and renders at full resolution for a more accurate representation of AO.
Pros
−It has additional hardware acceleration on AMD video cards.
Cons
−This algorithm is designed for AMD video cards that, according to the Steam Hardware & Software Survey, are much less popular than NVIDIA video cards.
What Is Voxel Accelerated Ambient Occlusion (VXAO)?
VXAO (Voxel Accelerated Ambient Occlusion) is a method that works with voxels – the equivalent of a pixel in 3D. DirectX 12 allows doing retracing and voxelizing a scene in search structures.
By doing that, a slightly more accurate method that takes into account more complex scene geometry can be obtained. However, this method is even more expensive than other methods. VXAO is part of the VXGI (Voxel global illumination) surround lighting technology, which more correctly takes into account direct and reflected light. Principle of Operation The VXAO algorithm encompasses three passes: voxelization, voxel post-processing, and cone tracing. Voxelization means that triangle-based meshes are rendered into a 3D texture.
Its performance depends on the total number of triangles, their size, and the number of draw-calls needed to render them. Post-processing involves the processes of clearing, filtering, and downsampling voxels. Its performance depends on the total number of voxels produced during voxelization. Then cone tracing is conducted in screen space. Its performance depends on the shading rate, screen resolution, and the cone-tracing pass in 1080p resolution.
Pros
−Provides the most accurate shading.
−VXAO handles dynamic scenes quite well.
Cons
−Only high-end GPUs will be able to take full advantage of it.
−Depending on the implementation, VXAO can be 3-4x slower than HBAO+.
−It requires the latest GTX video card.
The Use of AO Algorithms in Virtual Reality
In virtual reality, SSAO is very important, but it is less important than for ordinary games. This is due to the effect of stereo vision.
With stereo vision in virtual reality, our brain sees the depth-buffer, reconstructing it from two stereo pictures. The more objects are shifted in the pictures, the closer they are to the viewer, and thus the brain actually receives depth information and recognizes it.
Thus, unlike PC games, where there is no depth information on the monitor and such effects like AO are needed to emphasize depth, in VR, even without the AO effect, the viewer understands the depth of the scene and much needs this effect much less.
On top of that, due to parallax, the eyes perceive slightly different images, and SSAO, due to its screen space nature, generates slightly different results for them. They make AO seem “illusive” as each eye sees mismatched AO results. Therefore, in larger AO radiuses, it can even make a bad, general image impression and break immersion.
Another aspect is that all screen space effects depend on the resolution – the higher the resolution is, the more they affect performance, especially such a “heavy” effect like AO.
A characteristic feature of virtual reality is that huge-resolution images are rendered, and the main constraining problem is the low-fill rate of the video card. Therefore, the inclusion of the SSAO effect can decrease FPS by almost two times.
Unity Ambient Occlusion
ARVI has tested many AO algorithms available on a Unity Asset Store and has come to the following conclusion. The test was conducted on the Dragon scene (FPS for GTX 970), all effects with default parameters:
- No Effects: 146 FPS;
- TSSAO (Temporal screen space ambient occlusion): 99 FPS;
- SSAO Pro (Screen space ambient occlusion): 144 FPS;
- SESSAO (Sonic Ether’s screen space ambient occlusion): 126 FPS;
- Cinematic Image Effect AO: 126 FPS;
- Amplify Occlusion: 144 FPS.
So, standard cinematic effects from Unity reduce FPS from 146 to 126, amplify occlusion reduces FPS insignificantly (from 146 to 144), SESSAO reduces FPS to 126, SSAO Pro (it is one of the fastest algorithms) reduces FPS to 144, and TSSAO reduces FPS from 146 to 99. These effects don’t work with Single and Multi-passes in VR.
For VR, Amplify Occlusion 2.0 can be a good solution. As it is stated, “Amplify Occlusion 2.0 completely discontinues HBAO-based techniques in favor of the much superior Ground Truth Ambient Occlusion (GTAO), bringing quality and accuracy closer to traditional ray tracing.”
This method supports VR Single and Multi-Passes, and it offers high quality, fast operation time, and a performance boost.
Also, take a look at Unity’s new implementation of AO in PostProcessing Stack v2. It has many interesting features such as scale-wise AO.
Bottom Line
Thus, it is better to use even the simplest AO algorithm than not to use it at all.
We are very excited by new ray-tracing features of DX12 and NVIDIA RTX video cards. It seems that in the future, AO methods will go in the direction of fully dynamic ray-tracing instead of Depth-Buffer ray marching.
Our ultimate recommendation is as follows:
If your computer copes with the game well without AO, then you can choose either SSAO or HBAO with the lowest quality to save more performance. Usually, these methods decrease FPS by no more than 10%.
If performance allows, then HBAO+ or HDAO are possible options. If you have a top-end graphics card, then go for the VXAO algorithm.
Interactive Comparisons
- Ambient Occlusion On or Off
After
- Ambient Occlusion Ultra vs Ambient Occlusion Low
After
- SSAO vs HBAO+
After
- HBAO+ vs Ambient Occlusion Low
After
- HBAO+ vs Ambient Occlusion Ultra
After
- VXAO vs HBAO+
After
