Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Untyped dictionary serializer #439

Merged
merged 23 commits into from
Jan 5, 2024
Merged

Conversation

aloneguid
Copy link
Owner

@aloneguid aloneguid commented Dec 13, 2023

Use single codebase from FSC/FAC to understand untyped dictionaries. This will deprecate row API when absolutely stable.

Motivation

  1. Single codebase for class serializer and untyped dictionary serializer.
  2. De-serialization produces JSON-like structures in memory. These can be written back to JSON file as is.
  3. Row API is an old legacy that is somewhat buggy and very hard to evolve and fix.

Release plan

This will be released as an addition to the current class serializer in v4. Row API methods will be marked as deprecated and eventually removed in v5, making it a primary dynamic API.

Todo

  • Atomic fields.
  • Nullable atomic fields.
  • Maps.
  • Structs.
  • Lists.
    • Atomics
    • Lists and Structs
  • Public interface refinement.
  • WRS docs restructure and documenting the above.
  • Add docs for [ParquetRequired] (related to the above).

@aloneguid aloneguid marked this pull request as draft December 13, 2023 15:56
@aloneguid aloneguid added this to the 4.19.0 milestone Dec 22, 2023
@aloneguid aloneguid marked this pull request as ready for review January 5, 2024 09:54
@aloneguid aloneguid merged commit 50ccd64 into master Jan 5, 2024
5 checks passed
@aloneguid aloneguid deleted the dictionary_serializer branch January 5, 2024 09:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant