Add data chunk and task result enums and dtos #2442
Conversation
ypeckstadt
requested review from
komamitsu,
brfrn169,
feeblefakie and
Torch3333
| @Builder | ||
| @Value | ||
| @JsonDeserialize(builder = ImportTaskResult.ImportTaskResultBuilder.class) | ||
| public class ImportTaskResult { |
There was a problem hiding this comment.
Honestly speaking, it's hard to me to get the relationship of these classes and how to use them from this PR. It would be great if there is a simple diagram or something to show the relationship.
There was a problem hiding this comment.
Sorry, my previous comment might be unclear. I just read the design document again. The definition of the following terms are still unclear to me, so adding comments about the definitions of xxxxx would be enough:
batch(I guessImportTransactionBatchis related)task(I guessImportTaskActionandImportTaskResultare related)
There was a problem hiding this comment.
@komamitsu san,
Sorry for the late update, I will provide the general idea for batch amd task.
The import file is split into smaller data chunks for import initially. Each data chunk when the import is in transaction mode is further split into transaction batches. This is indicated as a batch .
In each transaction batch, each individual row of data is imported by a transaction. The process of importing each individual row via a transaction is termed as a task.
There was a problem hiding this comment.
@inv-jishnu Thanks for the explanation! I think the information will be very helpful for future code reviewers and maintainers. Could you add the description to the design doc or as source code comments?
There was a problem hiding this comment.
@komamitsu san,
I will check with @ypeckstadt and add these in the design doc.
Thank you.
There was a problem hiding this comment.
LGTM! 👍
@inv-jishnu Let me know when you address #2442 (comment)
| @@ -0,0 +1,8 @@ | |||
| package com.scalar.db.dataloader.core.dataimport.datachunk; | |||
|
|
|||
| /** * Status of the import data chunk which during the import process */ | |||
There was a problem hiding this comment.
| /** * Status of the import data chunk which during the import process */ | |
| /** * Status of the import data chunk during the import process */ |
Description
This PR adds enum and dtos used in data import for data chunks ssupport (smaller partitions of import data - data chunk details, status and state enums). And the dto for enums for import tasks and import result.
This also contains enums and dtos for transaction batch (to store transaction batch details, transaction batch status and result)
Related issues and/or PRs
NA
Changes made
I have added data chunk and task result enums and dtos for data loader import.
Checklist
Additional notes (optional)
Road map to merge remaining data loader core files. Current status
General
Export
Import
Release notes
NA