Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding a use case example 'shuffle' #15

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

JaeseungYeom
Copy link
Contributor

Use case demonstration for input file shuffling with deep learning training.
DYAD needs to be updated to leverage fcntl() to dynamically determine the mode of each individual open file, and handle it accordingly. This code is independent of DYAD and Flux. However, it reuses some of utility functions in DYAD implementation. The source files of those are symbolically linked to avoid duplication.

@ilumsden
Copy link
Collaborator

ilumsden commented Jun 13, 2024

What's left to do:

  • Add script(s) to run with DYAD
  • Integrate into testing/CI

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants