Maybe someone smarter than me can chime in, but I think this is just the way the Solo feature is meant to work.
But maybe this will help: Ableton settings > Record,Warp&Launch > Exclusive > Solo (make it grey)
…then you can have multiple tracks Solo-ed at the same time. You could set your ‘backup’ video set to Solo, and add other tracks as you like by clicking their Solo button.
Or, just de-Activate all the tracks, then Activate your ‘backup’ video track, and then Activate/de-Activate the other audio tracks as needed.