Sidenote: Discussing Ray-Tracing Implementations and Why RT Faces an Uphill Battle

Last time I discussed ray-tracing, it was largely to discuss if RTX, as implemented by Nvidia, had been a bust. That post and its conclusions all focused largely on Nvidia’s efforts around marketing, branding, and supporting developer efforts with the technology, and what I felt is ultimately that Nvidia hadn’t really made ray-tracing an attractive proposition for enough gamers, certainly not enough to force upgrades off of older GPUs and onto the first-gen RTX GPUs launched with the RTX 2000 series.

However, there is another side worth discussing, and so I want to split this into two parts – a discussion of the various implementations of ray-tracing in games, and then a look at why developers may be hesitant to move to it even as the hardware performance with RT improves.

First, I want to share my own personal experiences over the last few weeks with my RTX 3080 and the few games I’ve been able to try.

As a longtime and current main WoW player, naturally, the first RT-enabled title I played was…Control. Control has a full-stack RT implementation and looks stunning with ray-tracing on, and with an RTX 3080, it is even playable with full RT and detail settings maxed at 3440×1440 even without DLSS!

But, joking aside, I did try the Shadowlands implementation of RT, and, well…

• 

• 

• 

• 

Can you tell the difference?

Let’s try one more set of comparisons…

• 

• 

• 

• 

How about now?

I put this here not to drag the WoW team, because getting something like ray-tracing working at all in a game engine that is 21 years old (work on WoW started in 1999!) is incredible stuff, but because to a lot of people, myself included at one point, ray-tracing is this near mythical panacea that makes every rendered 3D scene look incredible, when in truth, ray-tracing in games currently comprises 4 different techniques.

First, A Brief Explainer of Ray-Tracing

Ray-tracing as a technology determines the visual detail of a scene by casting the rays of light that would interact with and illuminate the scene, calculating how the lights bounce and affect the scene, and computing a final value per pixel to display. Current game ray-tracing, with the exception of Quake II RTX and Minecraft RTX, are a technique called hybrid rendering. Normal games are rasterized, with a series of triangles being drawn on screen and shaded through the execution of shader programs and code. Lighting is typically pre-baked, but systems like probe-based lighting can determine the lighting values of open space in the scene to change surface colors on moving objects like player characters to approximate what lighting would realistically look like.

Current hybrid-rendering techniques still do the rasterization steps to render the scene, and then use ray-tracing to sample the lighting in the scene and modify the rasterized output to match the impact that ray-traced lights have in the scene. This makes rendering this way a two-step process at a high level – a rasterization pass to draw the scene assets and prepare for ray-tracing, and then an RT pass to calculate the path of light through the scene and output the final image. Both of these steps are actually comprised of multiple steps of their own, but for the sake of simplicity, we’ll focus largely on the RT step.

Ray-Traced Shadows

WoW’s implementation, along with other AMD partner titles (DiRT 5 and Godfall) and some older RT titles (Shadow of the Tomb Raider, ironically) use ray-traced shadows. Ray-traced shadows have a few benefits – they are the least computationally expensive form of ray-tracing technology and can add some realism to a scene without tanking the framerate. When doing RT shadows, the game simply casts the rays from designated light sources, determines how they intersect objects in the scene, and then uses those calculations to draw shadows on screen.

The advantage of RT shadows is that they present a more realistic shadow with accurate lighting directionality and correct levels of edge softness for the distance from the light source. Where they can present problems is in what objects in a scene have their lights traced for this math. You can trace from multiple scene lights, or instead from a single global source. The issue I have with WoW’s ray-tracing implementation and what makes it underwhelming is simple – it casts from a single global source hanging over the scene (the sun/moon/room light) and other ambient lights in the scene do not affect this calculation in any way. We’ll revisit this point later, but because WoW uses a lot of pre-baked ambient lights, it sometimes creates sharp illusion breaks, where an RT shadow (and the game’s normal shadows for that matter) look very out of place. For example, the room where you fight the Stone Legion Generals in Castle Nathria has a large red stained glass window, which has a pre-baked and applied visual effect of rear illumination, casting a large red and black shadow onto the balcony. It looks really cool, but this large red light has no effect on the scene at all, and even with RT shadows set to High, the characters in the scene will still have a single shadow casting in a different direction from the scene lighting because it is being drawn based on the overhead global scene light.

Ray-Traced Ambient Occlusion

Ambient occlusion is a 3D rendering technique for showing the impact of items blocking the ambient light from other parts of the scene. An example can be stairs with side walls – ambient occlusion creates the soft shadows that the walls would leave on the sides of the stairs while allowing the remaining light to reach the rest of the stairs. This image from Intel shows how it works by exposing the geometry underneath and how light is occluded around the objects.

Current dynamic ambient occlusion techniques get pretty close to maintaining a sense of realistic lighting in a scene, but often are based on static lighting that is pre-baked into the scene or screen-space ambient occlusion, which can only make use of the lighting data in the current viewport frustum (and not the entire scene), which can create some visual irregularities. Ray-traced ambient occlusion does something very similar to ray-traced shadows, but specifically for the AO on objects and not the scene as a whole. It can be similarly hardware-intense when compared to RT shadows, or slightly more so, depending on how many scene lights it is handling. This is one of those “blink and you’ll miss it” type of settings, because I would challenge most people, myself included, to seriously observe the impact of AO in a scene and be able to point it out, but RT technology adds a layer of realism to how it is handled.

Ray-Traced Reflections

Now we’re getting to the really good stuff. RT reflections use ray-tracing to create reflections on any surface identified as reflective. Most games with reflections today use screen-space reflections, which determines what can be shown in a reflection based on what is on-screen, or cube maps that are dynamically updated to show changes in the scene per frame, mapping a version of the scene to a texture that is projected onto a reflective surface. These are fine for the most part and have their roles to play (screen-space reflections generally do better with ground-based reflective surfaces like glass or metal flooring or reflective bodies of water, as the amount of screen-space data that can be shown and appear accurately reflected is high) while cube maps still generally see use for upright or other such objects where screen-space may not always include things that would be reflected and thus would look odd.

RT reflections, well, use ray-tracing to determine the values of light that reach a reflective surface, thus making them sharper and more accurate than either other technique, as well as allowing them to be better applied on odd surfaces (car windows and bodies or other such curved surfaces). This is slightly more involved than shadows, as accurate color data for each portion of the reflection is needed to ensure an accurate picture. There can also be fun anomalies like in first person games, where the player character is typically not rendered, which leads to either that character being MIA in the reflection, having to be specifically accounted for in rendering, or looking weird in games where they render a portion of the player character in first-person view to allow for certain view angles to work.

RT reflections are something the current consoles can do reasonably well, as both Spider Man: Miles Morales and Watch Dogs: Legion feature them and they still play well and the reflections add a lot to the games in question for realism – for gameplay, neither game is twitch-action-y enough for reflections to add a gameplay advantage, but it is cool! To this point, the technologies we’ve discussed largely don’t get lumped into most people’s definition of ray-tracing, because most of us think of this last one when we think of RT…

Ray-Traced Global Illumination

RT Global Illumination (gonna say RTGI from here out) is what most of us think of as RT. A light is bounced around the scene, adding color and highlights to objects in the scene, with the final color value per pixel being determined by a real-time raytracing pass instead of a probe-based lighting system or pre-baked lighting maps. This is the most computationally expensive as the whole scene has to be traced, with a larger number of lights and intersections to compute and run through on the RT pass. The results of this are much increased realism, as light now interacts with everything in the scene correctly, modifying surface colors based on the results of all scene lighting (in theory) and removing most of the anomalies that intersections of pre-baked lights and probe-lighting can create.

But that now leads us to the big issues with RT as a technology…

Why Isn’t Ray-Tracing More Popular With Developers?

There are a few obvious things we can address right away with this question. Most easily, adoption rates of ray-tracing hardware remain low. Last generation’s Nvidia cards were not a high mark for an upgrade, prompting a lot of people to sit out the generation, and the current gen Ampere GPUs are worthwhile upgrades but nearly impossible to actually buy, and the AMD Radeon RX 6000 cards remain in even shorter supply. In the most recent Steam Hardware Survey, the total usage of ray-tracing capable GPUs was 15.71% of the market. Steam is not the full PC gaming market, especially not the even-smaller number of people who would opt-in to the Steam Hardware Survey, but it is a representative sample and allows us to observe some basic trends at a high level.

This lack of adoption goes into the console market too, as only a few million Xbox Series and PS5 consoles are in the wild, and as such, optimizing titles currently in development for this tech is just not necessarily worthwhile.

The bigger problem, though, is that there is an observable lack of developer enthusiasm for RT technologies. At a core level, I think most developers are like us – it seems like a cool technology and is really neat, and you can see and hear tons of different presentations and discussions from various game developers nerding out over it as a technology. What you’ll see, though, is that professionally, it just isn’t getting highly implemented.

One reason seems to be that developers so far have only largely implemented it as a promotional tool. AMD sponsored Godfall, DiRT 5, and WoW and promotes the games in exchange for being able to discuss how their technology enables awesome raytracing performance for them. Likewise, Nvidia has similar deals with titles like Battlefield V, Cyberpunk 2077, Watch Dogs Legion, and more. Nvidia gets to try and push an RTX GPU sale, and the developer gets their game shown off outside of their own marketing. So far, the only title I know of that has any form of ray-tracing without some kind of sponsorship or promotional deal that is readily apparent is Crysis Remastered, which also, coincidentally, uses a custom RT component in its engine that can run on non-RT GPUs, albeit slowly (but not as slowly as DXR implementations!).

But critically, something that caught my eye was last August’s Hot Chips conference, when Microsoft discussed that while they did have RT hardware in the Xbox Series X, it was something that developers have indicated a certain amount of hesitation towards. On the surface, this is actually a bit weird – RT does, in theory make development of levels and assets easier. Currently with pre-baked lighting, it takes a lot of effort and time to make a scene look a certain way. A light emitter has to be placed, precisely tuned for the look the artist wants, and then submitted for rendering, which ray-traces the light through the scene to determine the visual impact of the light and create a final lightmap that is then stored as a part of the game’s assets. This rendering can sometimes take hours of time to build the final lightmap, and it also means that if the level changes in any way, the lightmap has to be redrawn, taking that time again for the new pass. Ray-tracing can be done easier, in that you generally just have to place and tune the light emitters in a scene, or define a surface as reflective, and/or define what objects will cast shadows or occlude light. No pre-rendering of the light is required and you can tune as you go, being able to make tons of smaller adjustments or iterating on the look of a scene without any need to bake the lights.

But, something I realize looking at WoW, is that a lot of developers might have a reason for preferring older lighting techniques outside of the adoption rates of RT hardware – artistic style. The advantage to pre-baked lights, besides the reduced computational requirements for the player, is that they allow the artists to define their own look for a scene. While a lot of us love and play games with photorealistic graphics, many of us play games with their own unique art style.

WoW uses a lot of baked lighting to add ambience to a scene. Could you create the stained-glass window of Castle Nathria with ray-tracing? Sure, probably. But then the visual impact of it requires a precise light emission that translates across settings and different user configurations, whereas pre-baked, you can account for it in development. WoW is fully of lights that don’t affect anything – they don’t seem to contribute to a probe-based system for moving objects and tend to, at most, only light a single spot or provide an animated ambient occlusion effect, but these lights add to a scene without changing its character. If you had RT lighting in the Howling Fjord, then you’d have this odd contrast of cool blues and greens with large roaring orange fires, which the baked lighting allows the developers to constrain or expand on with an artistic flair. Ray-tracing is a stylistic gamble, that the natural laws of how light works and intersects objects in the scene will illuminate things in a way that is visually pleasing and works in the context of what the artists want. Today, game artists may not be fully ready to tune a ray-traced scene to look great – hell, even full-stack RT implementations like Cyberpunk 2077 still use some baked lights at max RT settings to get the look the team wanted!

Ray-tracing will, in 10 years, likely be the standard mode of game rendering. Everyone playing modern at the time titles on PC and consoles will likely have GPUs that will be able to handle it well (or they’ll finally be able to buy an RTX 3080 there in 2031), and a new generation of game artists will be trained on how to best use RT light emitters to decorate a scene in the way they want it to look, both for hyper-real titles but also for games with their own unique art styles. For right now, and for the immediate future, while RT is a cool future and a fun curiosity you can enable and gawk at, it isn’t really something that is worth worrying about or pushing to always turn on, although some implementations are worth it if you can get the performance to play well (Control is, for sure!).

Oh, and the screenshots above? All max settings, and both in the same order for RT Shadows setting – Disabled, then Fair, then Good, and then Max.

Advertisement