Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stacktrace without error message when git fails during --job #4750

Open
Micket opened this issue Jan 21, 2025 · 0 comments
Open

Stacktrace without error message when git fails during --job #4750

Micket opened this issue Jan 21, 2025 · 0 comments
Labels
Milestone

Comments

@Micket
Copy link
Contributor

Micket commented Jan 21, 2025

I had run out of space on my test VM, and trying to

eb --fetch QuantumESPRESSO-7.4-foss-2024a.eb

and it correctly display the fail, and it's easy to see it failed to due disk space in the log

$ eb --fetch QuantumESPRESSO-7.4-foss-2024a.eb 
== Temporary log file in case of crash /dev/shm/eb-t6ieyp0y/easybuild-sgcd3k5r.log
== processing EasyBuild easyconfig /apps/easybuild-easyconfigs/easybuild/easyconfigs/q/QuantumESPRESSO/QuantumESPRESSO-7.4-foss-2024a.eb
== building and installing QuantumESPRESSO/7.4-foss-2024a...
  >> installation prefix: /apps/Arch/software/QuantumESPRESSO/7.4-foss-2024a
== fetching files and verifying checksums...

ERROR: Shell command failed!
    full command              ->  git clone --no-checkout https://github.com/wannier-developers/wannier90.git
    exit code                 ->  128
    called from               ->  'get_source_tarball_from_git' function in /apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/filetools.py (line 2719)
    working directory         ->  /dev/shm/eb-t6ieyp0y/tmpu1t3yx6s
    output (stdout + stderr)  ->  /dev/shm/eb-t6ieyp0y/run-shell-cmd-output/git-3p2mjjk_/out.txt  <-- I can see the error easily
    interactive shell script  ->  /dev/shm/eb-t6ieyp0y/run-shell-cmd-output/git-3p2mjjk_/cmd.sh

but when I did this via --job it crashes hard

$ eb --job QuantumESPRESSO-7.4-foss-2024a.eb 
== Temporary log file in case of crash /dev/shm/eb-uk5cm_ru/easybuild-3hy06wec.log
Traceback (most recent call last):
  File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/main.py", line 801, in <module>
    main_with_hooks()
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/main.py", line 787, in main_with_hooks
    main(args=args, prepared_cfg_data=(init_session_state, eb_go, cfg_settings))
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/main.py", line 742, in main
    do_cleanup = process_eb_args(orig_paths, eb_go, cfg_settings, modtool, testing, init_session_state,
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/main.py", line 555, in process_eb_args
    submit_jobs(ordered_ecs, eb_go.generate_cmd_line(), testing=testing)
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/parallelbuild.py", line 159, in submit_jobs
    return build_easyconfigs_in_parallel(command, ordered_ecs, prepare_first=prepare_first)
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/parallelbuild.py", line 97, in build_easyconfigs_in_parallel
    prepare_easyconfig(easyconfig)
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/parallelbuild.py", line 214, in prepare_easyconfig
    easyblock_instance.fetch_step(skip_checksums=True)
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/easyblocks/generic/configuremake.py", line 231, in fetch_step
    super(ConfigureMake, self).fetch_step(*args, **kwargs)
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/framework/easyblock.py", line 2526, in fetch_step
    self.fetch_sources(self.cfg['sources'], checksums=self.cfg['checksums'])
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/framework/easyblock.py", line 532, in fetch_sources
    src_spec = self.fetch_source(source, checksum=checksum)
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/framework/easyblock.py", line 488, in fetch_source
    path = self.obtain_file(filename, extension=extension, download_filename=download_filename,
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/framework/easyblock.py", line 930, in obtain_file
    return get_source_tarball_from_git(filename, targetdir, git_config)
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/filetools.py", line 2719, in get_source_tarball_from_git
    run_shell_cmd(' '.join(clone_cmd), hidden=True, verbose_dry_run=True, work_dir=tmpdir)
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/run.py", line 192, in cache_aware_func
    res = func(cmd, *args, **kwargs)
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/run.py", line 627, in run_shell_cmd
    raise_run_shell_cmd_error(res)
  File "/apps/Arch/software/EasyBuild/5.0.x-beta/lib/python3.9/site-packages/easybuild/tools/run.py", line 175, in raise_run_shell_cmd_error
    raise RunShellCmdError(cmd_res, caller_info)
easybuild.tools.run.RunShellCmdError: Shell command 'git' failed!

so the preparation that --job does to download the sources isn't handling run_shell_cmd failures the same way. I think it should.

@boegel boegel added the EasyBuild-5.0 EasyBuild 5.0 label Jan 29, 2025
@boegel boegel added this to the 5.0 milestone Jan 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants