incomplete log error

[shubh0155]

I've generated Darshan logs for Dlio benchmarking on my Luster file system. When I parse these logs using darshan-summary.pl, it generates a PDF, but the output appears to be incomplete. When I try to parse the logs using darshan-parser with the --show-incomplete flag, it gives the full report. I've also attempted to exclude all other directories by passing DARSHAN_EXCLUDE_DIRS=*, but I'm still facing the same issue. Can anyone provide assistance with this? Thank you.

[kevin-harms]

How many files are you trying to track per-rank? You might have rank0 or something that is using lots of files. If you use the --show-incomplete, do you see one rank (or many ranks) with lots of files?

[shanedsnyder]

See this section in darshan-runtime docs about configuring the runtime library: https://www.mcs.anl.gov/research/projects/darshan/docs/darshan-runtime.html#_configuring_darshan_library_at_runtime

Specifically, you have a couple of options to ensure you're not missing something instrumentation:

• simply asking for Darshan to allocate more records for modules that are running out of memory using the MAX_RECORDS setting
• filtering out records that aren't of interest using NAME_EXCLUDE setting

There should be some example config files shown there that illustrate how to set those parameters. Maybe start there and let us know if you have any questions?