Hollow Earth is one of the possible types of underworlds I proposed in the OP. The other one, which I haven't seen as many people talking about, is the flat Earth with an underside. In this scenario, there is a normal day/night cycle, where it's day on the underside when it's night on the surface above, and vice versa (because the sun is literally passing under the planet).

There is the matter of what happens to things that fall off the underside. If the planet is a flat disk, and gravity still points "down" into the void below the underside, then gravity must work differently than for a normal planet. Gravity might be more like a magnetic field, with "poles" that cause it to be directed. The "up pole" is on the surface, while the "down pole" is on the underside. This means that as you move away from the surface/underside, especially if you're far from the center of the disk, then gravity will start pushing sideways. If something falls off the underside, the gravity lines will eventually curve all the way around the disk, causing whatever fell off the underside to take a very long path back to the surface. This could also explain floating islands; it's essentially the same idea as magnetic levitation. This might mean that floating islands are only stable in certain configurations, and that outside those configurations they will drift, and may flip over and crash into the surface or underside.

I imagine this change in gravity would also affect things on the surface. Mountains or tall buildings might need to tilt to one side in order to stay upright, for example. In fact, it should be possible (if you have strong enough materials) to build a tower that goes from the surface to the underside, curving around the disk by following the gravity lines. Also, if things that fall off the underside end up back on the surface, then we could use the idea of having a layer of magma between the surface and the underside that's responsible for tectonic activity. Perhaps some parts of the surface are "sinking" into the magma ocean, and in turn being used as construction material for the underside, which then gradually erodes away and is deposited again on the surface.

I wonder if this "magnetic" gravity would have an effect similar to the Coriolis effect? Given that gravity will change direction as you move away from the surface or underside, it might not be that different from how it was presented in that video about gravity on Halo. Assuming that gravity lines emanate from the entire disk, and not just from the poles, then gravity would be mostly normal close to the surface/underside, so projectiles wouldn't experience much of a problem as long as you're aiming for someone on the ground. If you shoot into the sky (either above or below), then the bullets/arrows/whatever might take some weird paths.