Hi there 
Compendiums are not loaded up at world start like the world data is, but they are still stored inside of the world’s folder in user data and will take up storage space.
The warning is based on space taken up, with the understanding that larger storage footprints are more likely to run into slow loading issues, but this isn’t always a 1:1.
The warning is simply letting you know that there is a lot of data in that world, but if you are not running into slow world loading issues with this, then there is no cause for concern.
One thing that might help you save storage space if the world compendiums carry data that you would like to have accessible to multiple words, is to move the data in them to a shared compendium module which you can share between multiple worlds. That will lower the world’s storage footprint, and also let you get access to those compendiums from other worlds.
Hope that helps!