HDDS-12094.Implement ozone debug replicas verify checksums command #7748
Conversation
| OzoneConfiguration configuration = new OzoneConfiguration(getConf()); | ||
| configuration.setBoolean("ozone.client.verify.checksum", | ||
| !isChecksumVerifyEnabled); | ||
| RpcClient newClient = new RpcClient(configuration, null); |
There was a problem hiding this comment.
This method processKey() is going to be called multiple times for all keys in a bucket.
The RpcClient object can be reused for multiple keys. I'd also suggest move the instantiation of OzoneConfiguration to be only one-time overall.
There was a problem hiding this comment.
also worth noting that the command will process one key at a time with no parallelism so could take a long time for a big bucket.
| @MetaInfServices(DebugSubcommand.class) | ||
| public class ReadReplicas extends KeyHandler implements DebugSubcommand { | ||
| public class ReadReplicas extends Handler implements DebugSubcommand { |
There was a problem hiding this comment.
Suggest to rename this class/file name as well to match the command name.
| File dir = createDirectory(volumeName, bucketName, keyName); | ||
| OzoneKeyDetails keyInfoDetails = checksumClient.getKeyDetails(volumeName, bucketName, keyName); | ||
| Map<OmKeyLocationInfo, Map<DatanodeDetails, OzoneInputStream>> replicas = | ||
| checksumClient.getKeysEveryReplicas(volumeName, bucketName, keyName); |
There was a problem hiding this comment.
Not totally related, but RpcClient.getKeysEveryReplicas() doesn't seems to create input stream that refreshes container cache upon failure. Could that lead problems in such a corner case?
What changes were proposed in this pull request?
This PR:
ozone debug replicas verifycommandread-replicastochecksumscommandchecksumscommand with the ability to walk the file tree and calculate checksums of all filesreplicas verifyNote: This is a WIP change. Further enhancements and refactoring will be done and acceptance tests will be updated.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-12057
How was this patch tested?
Manual testing in docker-compose environment
CI: https://github.com/ptlrs/ozone/actions/runs/12954171235