Evolution of bottom boundary layers on three dimensional topography -- Buoyancy adjustment and instabilities