The assets library needs a mass select/delete option

I uploaded a bunch of icons for my game, then realised that I should put them in a folder to keep things neat. Having to delete them one at a time with a confirmation window and a reload each time is really painful for 100+ files

3 Likes

It definitely does. I want to do a revamp of the UX for the assets library, especially when it comes to adding multi file selection, moving/renaming of files and folders, and optimizations with the upload (i.e: uploading multiple files at once instead of one at a time for example)

1 Like

Is there any chance we can get SFTP access as a workaround for the time being?

SFTP will definitely not be happening, sorry.

Here is a response I gave someone else when it comes to why SFTP access will never be an option :

Just so we’re clear, if you think you need FTP access, you’re wrong, and if you really need it, then my service is probably not meant for you. Giving access to that to users is a recipe for disaster, just like google or dropbox (or roll20) isn’t giving you FTP access to your remote files, there is no reason for you to have it on the forge. Anything you want or need to do will be possible and 1000 times easier and safer to achieve with the provided tools than with an FTP client. The Forge provides convenience, and FTP or other similar tools are everything but convenient, and simply having that as an option will cause a lot of confusion to users who will think they need to figure out how to install and configure a separate app in order to access a service whose purpose is to make your life easier and not require you to install or configure anything

That being said, this wouldn’t apply to the assets library because those aren’t actual files, they are S3 objects in an object store. Modifying a file also needs to propagate the change to 40 servers that are geodistributed so you don’t have your player that is on the other side of the planet receiving the old file, so there are many considerations here that need to be taken into account.
There are also two reasons for not yet having the ability to do renames :

  • There is no concept of ‘renaming’ in object stores, only copies, and there are ridiculous rate limits on those unfortunately (imposed by the service provider, of 2 file copies per 5 minutes :man_facepalming: ).
  • I don’t want people to start renaming their files and breaking their worlds so I want to have a system in place so that if you rename a file, it would also change the URL that points to that file in all the worlds that are using it. I also don’t want that auto-rename in world data to cause data corruption or breaks, so it needs to be done carefully.

Also, of course, things have been crazy getting ready for release, and so I had other higher priority things to handle first.
I hope that helps.

1 Like

Note: forgot to mention. I do know that asset management is an important aspect of the site and it is a priority to get that improved, and it is one of the next things I want to work on.

1 Like

This completely makes sense, and if I used my developer brain instead of my user brain for half a second I would have already realized this. I will just be patient instead and wait for your management UI.

1 Like

The asset uploads already got way faster recently, so yay!

Currently, uploading assets seems to just silently ignore files if files with the same name already exists. An “[ ] Overwrite files” checkbox would be handy. Better ways to delete too, as said above. Those are my main things, otherwise the Asset Lib is lovely.

The below are more minor things.

I just tried to rename a Asset Lib folder containing multiple thousands of files (an icon library) and ran into errors. I assume this is because of S3 shenanigans and there not really being “folders”. Maybe a warning when trying to rename large folders would be cool? Can see this being confusing for users.

I don’t need FTP access, but is there a way to access the normal data folder? (outside of uploading a game everytime I change the data, I presume)
Some modules require uploading files into their folder, support for this would be cool.
(And I know we’re supposed to use the asset lib, but if some module requires me to upload translation text files or CSV files into its directory, I can’t really do anything, besides maybe petition the module author for more options.)

Also, the filepicker gets slow with larger number of files in a folder very quickly, not sure yet whether that is on the Foundry or Forge side yet, though.

The asset lib integration also sometimes seems to act up - when opening the filepicker, sometimes I get a “there is no folder data/icons” error and get back up to the root dir, because it seems it tries to access data instead of the asset lib. More of a minor annoyance, as you just can navigate back.

One welcome thing to improve would be the search bar, currently it seems very basic - if you’ve got a file “goblin warrior”, searching for “goblin warrior” willd show it, but “warrior goblin” will not.

Edit:

Would it be useful to allow users to override this, optionally? E.g. all my players are from the same country, even the same region and I don’t see that changing for that game, ever, so I would be extremely fine with not taking up that much complexity and global storage by being able to select a country per game, or an AWS region or whatever - or maybe even just “US”, “Europe”, etc.

Hi @fallayn, welcome and thanks for the detailed analysis! I’ll be releasing the new Assets UI within the next few days so I’d love to get some feedback from you on that! :+1:

No, it will always overwrite, but what you may be hitting is a combination of a few layers of caching since the URL remains the same.
First, check that your own browser isn’t caching the image instead of querying the server to see if it changed, second, the CDN will be caching it, and while I do a purge of the CDN cache when you overwrite the file, it isn’t instant. It’s supposed to be, but it can take anywhere from 5 seconds to 5 minutes, depending on which server is closest to you, and how congested the network is (and maybe there’s some rate limiting as well, I’m unsure and wasn’t able to pinpoint what makes it slow or fast to do the purge from all caches worldwide).
I’ve also had some people saying that it worked after they restarted the Foundry server… I’m not sure why that would be (since caching is done on the client, not on the Foundry server), but that’s something I’ve heard more than once… maybe just the fact that they restarted the server delayed their attempt long enough for all the caches to be refershed.

What errors did you run into? I’ve done mass renames with thousands of files as well, and haven’t had any problem, it just takes longer (I think it took about 15 seconds to do 30000 file renames).
I suppose if there’s more than a thousand files, I can have it show a warning about the operation may take some time, and a loading spinner.

You can download the module, do your edits, then edit the version number of the module in the module.json file (so The Forge doesn’t install the Bazaar verison), then use the import wizard. But let’s be clear here, any module that requires you to do something like that is a module that is badly designed or written, because as soon as an update happens, whatever changes you did will be overwritten, and a module should always ask the user for the file, either through the Foundry file browser, or through a simple “open file” dialog for local files.

Yeah, that’s a Foundry issue, it doesn’t have any kind of progressive loading of content, so it tries to render a huge list of files which will be slow. The new Assets Library UI does avoid that by progressively adding files as you scroll down.

I think there is currently a bug in the 0.6.x Foundry where trying to access any folder “icons” will cause it to think you’re trying to access the core foundry icons folder. Maybe that’s your issue? Other than that, there is also an issue from the Forge’s side where if you browse for a file where the url is not from within your own assets library (i.e: you browse for a token where the image was set by a user from within their own library, or you browse for a file that was linked to a bazaar file), then it will try to browse the directory within your own library, which will often result in an empty directory. It shouldn’t throw an error about “there is no folder” though.

Humm… I’ll make a note of that, it won’t go into this release of the revamp, but it’s an interesting point. At the moment, it’s a search for a specific filename basically, so it will try to match exactly, what you’re asking for is a tokenized search which would be a fair bit more complex to implement and would also be slower to do as you would need to tokenize every filename in order to be able to find them. I also got another request for it to be a “filter” for the current directory only, instead of a “search everywhere” kind of thing, so there’s definitely some improvement that needs to happen to that search bar.

No, that wouldn’t be possible as I don’t have that much granular control over it, and because I purge the cache from the entire CDN for a specific URL. That cache purge is what needs to be propagated, but the file doesn’t get uploaded to the other servers until someone from that region tries to access it. So in your case, you upload the file, the first time you access it, it will be slightly slower as it caches it locally, then every other time, it will be fast as the file is available on the server closest to you.

Hopefully that helps clarify some things.
Thanks for the feedback!

Thanks for the detailed reply, and yes, that did qualify all of those things, I think :slight_smile:

It was a generic red error saying… nothing specific, something like “there was an error renaming…”.
If I run into it again I’ll make a note of it :slight_smile: Maybe it was just a hiccup.

The asset library improvements & filepicker handling large file lists sounds awesome!

The new UI that handles large file lists isn’t for the file picker within Foundry, it’s for the assets lib browser within the Forge site itself.

Alright… just because I’m maybe a couple of days away from releasing this to the production server, here’s a sneak peak of the new UI (please tell me it’s beautiful! :wink: ) :

Ah, now I got what you meant - still cool, though!

Damn, that looks nice. :slight_smile: Looking forward to the update!

And thanks for Forge, its really a great way to use Foundry. Really happy so far.

1 Like

It’s out! Enjoy! :smiley:

Closing this issue now! Thanks everyone for your patience!

I know this issue is considered closed, but I finally got around to doing some tidying up today. I really love the new UI, but I feel like it’s still missing a mass select/delete option. I can go through and tick boxes one at a time, but the unfortunate oversight which led me to upload a whole bunch of icons without putting them in a folder means there are literally hundreds of boxes I’d ahve to tick. I would love to see a “select all” button, so I could do that, then manually deselect the ~30 folders I want to keep.

Yep, this was suggested to me already and it was an oversight from my part, which I will rectify the next time I work on that area of the site.

Awesome, thanks for all your hard work!

@rahz What do you think about this : Server update changelog ? :smiley:

Magnifique!! Now I can get my assets under control!