I then spent some time fixing bugs and debugging scenes. Note to everyone: Dabrovic Sponza has pretty awful tesselation, don't use it at all possible. But anyway, I changed one of the doors in Dabrovic Sponza to emit lots of light and let my radiosity solver run on a course tesselation for a few dozen seconds (it did direct + some of the first bounce indirect in that time frame), giving me:
I'm going to try on Crytek Sponza; then profile and do the easiest optimizations possible before 3. But I ran radiosity on Sponza and had a soft lighting image in <1 minute; so I win!