Shared Compendiums

Is there a reason to not simple move all your scenes over into a shared compendium and delete them from your world and only import as you need them? Deleting the scenes and tokens from your art library seems to save you space and shared compendium scenes bring everything along.

Is there a downside to this?

No downsides to it!
We do actually recommend this as most as possible if you can ^^