At the start of the month I wrote that a 3DGS capture stores its sun in its colours, and that a search over the captures here found a usable sun in exactly two of them: Olsztyn, up 48 degrees and about three and a half times brighter than the sky, and La Sarraz, weak and hazy. The other places were shot under cloud. This week that measurement started doing something at runtime. As I write this it is behind a flag, off by default, and not on the live site.
The shadow receiver on the splats has existed for a while, but only behind a flag, and for a simple reason: the runtime sun was frozen at noon, so the shadow it cast agreed with no capture. A car would throw its shadow straight down while every shadow baked into the street around it leaned to one side. Now the directional light in those two places is aimed along the direction the relight tool measured off the capture’s own shadow edges, so the added shadow and the baked ones lean the same way.
How dark the added shadow gets is not a slider. If rho is the capture’s measured sun-to-sky brightness, lit ground is sky plus sun and shadowed ground is sky alone, so the shadow removes a fraction 1 - 1/(1 + rho) of the light. That comes out at 0.43 for La Sarraz and 0.76 for Olsztyn. It is applied per splat, with no extra depth pass. On the overcast captures there is no direction to match, and an invented low sun there would be a second sun the photograph never had. So I did not invent one. An overcast sky is brightest at the zenith, and the light for those places already points straight down, so the shadow it casts is the footprint directly under a car or a walker, which is where a cloudy day puts it. Its strength is 0.4, taken from the contact-shadow model: at the edge of a car’s body, about 60 per cent of the open road’s light is left. I started with only the two sunny places, then turned it on everywhere a day later after seeing it on them. It did not last the afternoon. A straight-down shadow does not match a capture whose walls visibly throw shadows sideways, whatever the sun estimator concluded, and the benchmark said the shadow roughly halved the frame rate on this laptop’s Intel UHD 620 (Ludlow about 6,200 against 11,500), with a single shadow lookup instead of four changing nothing. That last part was the clue: the lookup was running for every pixel of every overlapping splat, dozens deep, to get the same answer each time, because in this mode the shadow only depends on the splat’s centre. Moved to once per splat in the vertex shader, the same run reads about 7,200 against 9,800, a quarter instead of a half. Later the same day it went back on by default, at that cost. The cloudy places now cast along the direction their own shading leans, the sun estimator’s best fit even where it was too weak to call a sun, and those directions still need checking by eye.
What this is not, yet, is relighting in the full sense. The baked sun is still in the colours. The added shadow agrees with it instead of fighting it, which is the part that makes a car look like it belongs in the street. Taking the original light out, so a place can move to another hour of the day, is the harder half and is still open.
Two smaller things landed beside it. For a day, weaker devices rendered the place at 0.7x and upsampled over time. That only works if every frame is nudged by a sub-pixel offset the upsampler can undo, and the streamed town was reading the camera before the engine applied that nudge, so the meshes were jittered and the splats were not. The upsampler was un-jittering splats that had never moved, which reads as shimmer. The town now applies the same offset the engine does. Even with that fixed, the upsampled frame looked soft next to a native one, so a day later the 0.7x default came off again: it stays behind a flag until it stops costing sharpness.
The other one I have to report as a step back. The streamed town has a baked table of which parts of a place are hidden from where, and on 30 October I switched it from drawing hidden parts at lower priority to not drawing them at all. At the La Sarraz walk spawn on an integrated GPU that halved the splats drawn and took the frame rate from 11.75 to 21.5. It was measured at one spot, standing still. Moving around, the town had blank patches that filled in as you walked towards them. The table is computed from one eye per 4 metre cell at head height, through a coarse 1 metre voxel version of the collision mesh, and the lookup ignores how high the camera is, so a chase camera or a drone sees over things the table says block the view. It went back to lower priority the next day, and the speedup goes with it until the table knows about height. A win measured at one standing spot was not a win yet.