Progressively load large number of keys

[tugtugtug]

Redis commander often takes a long time to load large hashset, and can run out of memory and get killed when the hashset is large enough.
I don't think this is a bug, but it probably makes very little sense to have redis commander replicate the entire redis db in memory.
As an enhancement, could redis commander implement progressive loading of large hashset or large number of keys?
i.e. if the hashset is large, then load the first say N entries, and load more when the front-end scrolls to the bottom or user clicks to load more (or load all).
This would allow redis-commander to be configured with a reasonable memory limit.

[sseide]

you are right - such a feature is helpfull.
As i do not have time to implement it any PR is welcome. Will help to discuss a solution and answer questions arise during implementation.

[JESii]

Not sure, but I might have a similar situation with a series of SET/SADD commands. I exported a branch (200 items, some with large JSON objects to be inserted). It's been running for ~10 minutes so far which seems excessive given that a single SET or SADD happens almost instantaneously. Oh... I see "413 Payload too large" in the console. I'll see if I can cut this down and do some more testing.
UPDATE: Had to cut down the 200 items into 5- to 1-record bits to avoid the jquery failure; one record was too large all by itself (85750 chars), but I was able to get the others done this way. Would be nice to have a larger import capability.

[sseide]

@JESii HTTP 413 is a different error - you need to set the config key server.clientMaxBodySize to a bigger value. Exact value depends on your data stored inside Redis. Check docs/configuration.md for more information.

[JESii]

Thanks… I’ll give that a shot!

…

On Apr 29, 2022, at 8:00 AM, Jon Seidel ***@***.***> wrote: Not sure, but I might have a similar situation with a series of SET/SADD commands. I exported a branch (200 items, some with large JSON objects to be inserted). It's been running for ~10 minutes so far which seems excessive given that a single SET or SADD happens almost instantaneously. Oh... I see "413 Payload too large" in the console. I'll see if I can cut this down and do some more testing. — Reply to this email directly, view it on GitHub <#471 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAADZWQPHJSNBEK4QQVFCTVHP2SBANCNFSM5RP6YLDQ>. You are receiving this because you are subscribed to this thread.