Updated Pathfinder 2E and migrated to new version, suddenly my world folder size exploded

Hi there, I updated to PF2E 1.13.0. Got a ton of “Unknown System error, -122; Unknown System error, -122, write” spammed in. Uhoh.

The migration apparently worked. I then tried to turn off modules to find if they caused the error.

After that, I now can’t start the server again because I have my world quota exceeded (120%). The folder for my extinction curse campaign is now 520MB; locally, it’s only about 360MB. In the assets library, it’s 320MB.

Please help!

Try to run through this: World Size Check see if you can get it cleaned out. Otherwise @kakaroto might have to be of assistance here.

Thanks, that -almost- helped. I found the culprit - a juicy 522MB scenes.db.

However, the suggested workaround of deleting the junk out of it with notepad++ doesn’t work as notepad++ (and notepad, and editor) crash when trying to access the huge file.

EDIT: I tried deleting the scenes.db and reuploading the world, but got an error:
Failed to import 1 Worlds
Found 2 missing Assets in your Worlds

Can @kakaroto fix this server-side?

You could try notepad2, which I generally have more success with. But if you’re low on RAM, it would help to close down some heavy programs.
All that 522MB will need to be loaded into RAM to be usable, as neither notepad2 nor notepad++ have any optimizations for that.

If that’s not possible still, you could look into using Free Large File Editor

If you don’t mind losing data, deleting the scenes.db would work, but I believe you need to add an empty file back. Make sure you have file extensions turned on, which is seems like you do.

I do mind losing data :frowning: could open the file with Free Large File Editor, but couldn’t find the problem. I have zero experience with code.

The file is 522MB. On my local folder, it’s 8MB.

Oh you have a recent local backup? Well why not just use that!

When opening the file, there’s probably seemingly normal looking data, but a single line somewhere is excessively long and contains duplicate data tens of thousands of times. If you find it, it might reveal what caused it, but might not.

It would be good to figure out what caused the issue though, I’m guessing it was a module, but it’s strange that this started happening after gamesystem update, those are usually not a problem.

No, my backup isn’t that recent, i just mentioned that 8MB to 522 MB is quite the difference :wink:

Opening the scenes.db looks like this. Just walls and walls of text. I can’t make heads or tails of it.

Try zipping up the single file, if it’s much smaller than 500MB, upload it somewhere and send me a link here privately.
If it’s still around 100MB or so, it’s probably unsalvageable and too much work to fix.

Got it down to 4MB.

The issue had something to do with the mob of commoners with sickles. There were a lot of duplicates on each scene, but I couldn’t determine why. Every time it attempted to save the scene, it added new data to multiple actors.

The fixed file was sent back to the OP.

1 Like

The file unfortunately did not help, I got the same error message when migrating to the new PF2E system.

On my local Foundry copy, updating to the new version went flawlessly, so I imported that world into Foundry, which worked without a problem.

So, my problem is solved (only lost a bit of progress), but the issue itself seems to be Forge exclusive.

Was your local PF2E system the same version? Perhaps there’s a problem with the newest version.

It was updated to the newest version locally without a hitch. I updated the Forge installation and import worked.

Well, it’s unfortunate that we don’t fully understand the reason for it happening, but I’m glad it worked out regardless.

The problem persists - I updated from Pathfinder 2e 1.13.1 to 1.13.2. Again, write error that leads to the campaign being >500mb in size.

Luckily I now make backups of my world after every game session, prep and before updating. I downloaded the backup and migrated it locally to 1.13.2 without a problem. Re-uploaded to Forge and it works.

While this workaround means I can update the game system and campaign world, it’s not ideal in the least. I hope you guys get this fixed, as it clearly has something to do with Forge itself, as there is no error at all while updating locally.

At least it’s only this one world making trouble. other worlds, under the same PF2E system, update without a hitch.

The theory KaKaRoTo had was that why it works locally is that the same thing happens there, but foundry manages to clean up eventually because there’s no storage limit. On the forge, you hit the 500mb limit and then Foundry can’t actually solve it anymore.

You can try this by going through it locally and checking if the world size changes while the update occurs.

There shouldn’t be anything The Forge itself does to cause this, more like it becomes apparent, as you run out of space.
We’ve had other people having issues with migrating the latest pathfinder updates, where it’s simply doing too much, but not like your issue.

It’s likely we’ll gain a better picture of this over the next weeks, it seems like something might be up with the gamesystem itself.
Though, to be fair, you do have 200 commoners, all with their own items etc. I’m not sure what PF2 is doing while migrating, but it has to be related to it.

I did see that they forced an update to the bestiary on 1.13.2 for some reason, so that’s likely why the error occurred again for you. Though it might be likely that it keeps happening when larger migrations are done.

Anyway, try to check what happens to your world size locally, it might be too quick to fully keep track of, but you should see a huge spike in size, followed by a reduction soon after - and that spike probably goes over 500mb, thus the issues on The Forge.