Heya 
In-game performance is largely dependent on the complexity of the currently displayed scene, and the impact of any active modules. If you have a low framerate in Foundry itself, you may also want to enable low-performance mode in Foundry’s settings. I’d strongly recommend checking if the poor performance you describe holds true in Safe Configuration, with modules and the active scene disabled.
As a module, the module to show latency is known to be inaccurate and is heavily influenced by Javascript loads in the browser. Checking the developer tools console (F12) for any red errors can provide insights. Resolving module conflicts often leads to a noticeable performance boost.
Since you mention server size, I believe you’ve already checked your world size, and you do mention that hardware acceleration is active. It should be noted that hardware acceleration isn’t always in use even if enabled by the browser, and I’ve encountered a few cases where the OS was preventing the browser from making use of it. You can double check that it’s being used properly by running a WebGL Report and checking that your GPU is showing as the Unmasked Renderer.
If you are willing, opening a ticket in the #ticket-support channel of the Forge Discord server and posting an invite to your game there will allow one of us to hop in with you to attempt to help with troubleshooting, as in the cases where we’ve encountered this there is usually an answer for the poor performance you’re seeing once we delve into it.