client ID already exist

[jvmakine]

Hi, Thank you for the amazing library!

I have a test for running a dragonboat cluster with 2 replicas and nodehosts locally, talking to each other. They are both running in separate local addresses storing data to separate directories.

It works well for most of the time, but very occasionally the test fails with

debug:raft: getting session for shard 1 on replica 1
debug:raft: got clientID 4983173200269798049 for shard 1 on replica 1
debug:raft: proposing event to shard 1 on replica 1
debug:raft: getting session for shard 1 on replica 2
2025-01-23 11:06:53.611344 W | rsm: client ID 4983173200269798049 already exist
    cluster_test.go:64: Did not expect an error but got:
        failed to get session: failed to submit request to shard 1 on replica 2: request rejected
info:raft: stopping replica 1
info:raft: removing replica 1 from shard 1

When I try to create a session for the second replica using

nh.SyncGetSession(ctx, s.shardID)

As a workaround, I retry creating a session if the request is rejected.

I might be well misconfiguring something, but can not figure out what could be causing this?

The test is here if that helps: https://github.com/block/ftl/blob/a3dd47c987368c87a744979a5656cff18176b4c7/internal/raft/cluster_test.go#L50