Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[UR] Enable hwloc in UMF linux build #15261

Draft
wants to merge 2 commits into
base: sycl
Choose a base branch
from

Conversation

PatKamin
Copy link

@PatKamin PatKamin commented Sep 3, 2024

Update UR to statically link UMF with hwloc on Linux.
This will enable hwloc support in UMF on Linux.

Also, always use the v2.9.3 hwloc version for hwloc builds.

@PatKamin PatKamin marked this pull request as ready for review September 13, 2024 13:35
@PatKamin PatKamin requested a review from a team as a code owner September 13, 2024 13:35
@PatKamin PatKamin force-pushed the enable-hwloc-on-linux branch from d8a76ff to 94c1a73 Compare December 6, 2024 08:47
@PatKamin PatKamin force-pushed the enable-hwloc-on-linux branch from 7aa9690 to dcc6761 Compare December 9, 2024 12:11
@PatKamin PatKamin force-pushed the enable-hwloc-on-linux branch from dcc6761 to 2891f1e Compare December 9, 2024 12:16
@PatKamin PatKamin force-pushed the enable-hwloc-on-linux branch from 2891f1e to c647376 Compare December 9, 2024 12:28
@PatKamin PatKamin marked this pull request as draft December 9, 2024 13:58
@PatKamin
Copy link
Author

PatKamin commented Dec 9, 2024

Drafted until the UMF fix is pulled down to llvm via UMF update in UR.

@PatKamin PatKamin force-pushed the enable-hwloc-on-linux branch from c647376 to c6e5f06 Compare December 12, 2024 09:01
@PatKamin PatKamin marked this pull request as ready for review December 12, 2024 09:01
@PatKamin
Copy link
Author

Drafted until the UMF fix is pulled down to llvm via UMF update in UR.

Fix is now pulled down, ready for review.

@igchor
Copy link
Member

igchor commented Dec 30, 2024

You should probably also update https://github.com/intel/llvm/blob/sycl/sycl/doc/GetStartedGuide.md (remove hwloc as a dependency that needs to be installed by the user)

@PatKamin PatKamin force-pushed the enable-hwloc-on-linux branch from c6e5f06 to f1fa58b Compare December 31, 2024 11:36
@PatKamin PatKamin requested a review from a team as a code owner December 31, 2024 11:36
@PatKamin
Copy link
Author

You should probably also update https://github.com/intel/llvm/blob/sycl/sycl/doc/GetStartedGuide.md (remove hwloc as a dependency that needs to be installed by the user)

Right, now it will automatically be downloaded - deleted user requirement.

Comment on lines -47 to -48
* `hwloc` version 2.3 or later (Linux only)
* libhwloc-dev or hwloc-devel package on linux
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update UR to statically link hwloc v2.9.3 both on linux and Windows.

If I get the change right, the dependency on hwloc still exists. The change removed the runtime dependency, but not the compile time dependency. Am I right?

This document describes the environment for DPC++ compiler developers, not DPC++ compiler users.

Your change seems to raise the version requirement. Please, update the hwloc version to 2.9.3.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hwloc library will always be fetched from sources. Regardless of hwloc-dev packages user has installed, the fetched version will be linked with the UMF library. That's why I remove hwloc from this dependencies list, as the user-installed hwloc-dev package will not be used by UMF.

@bader
Copy link
Contributor

bader commented Dec 31, 2024

You should probably also update https://github.com/intel/llvm/blob/sycl/sycl/doc/GetStartedGuide.md (remove hwloc as a dependency that needs to be installed by the user)

Right, now it will automatically be downloaded - deleted user requirement.

If hwloc dependencies will be automatically resolved by CMake, please, remove installation of these dependencies from CI scripts. Specifically, from

and
sudo apt-get install -y graphviz ssh ninja-build libhwloc-dev
.

@PatKamin PatKamin force-pushed the enable-hwloc-on-linux branch from f1fa58b to d54b9d2 Compare January 2, 2025 08:48
@PatKamin PatKamin force-pushed the enable-hwloc-on-linux branch from bb9405e to 9c4a5d3 Compare January 23, 2025 08:13
@PatKamin PatKamin force-pushed the enable-hwloc-on-linux branch 2 times, most recently from 887ad36 to 12514ae Compare January 23, 2025 12:16
@PatKamin PatKamin force-pushed the enable-hwloc-on-linux branch from 12514ae to fd07145 Compare January 23, 2025 12:28
@PatKamin PatKamin force-pushed the enable-hwloc-on-linux branch from fd07145 to eccd59a Compare January 27, 2025 07:55
Update UR to statically link UMF with hwloc on Linux.
This will enable hwloc support in UMF on Linux.

Also, always use the v2.9.3 hwloc version for hwloc builds.
@PatKamin
Copy link
Author

@bader, I've added an automake dependency for building hwloc.

@bader
Copy link
Contributor

bader commented Jan 27, 2025

Also, always use the v2.9.3 hwloc version for hwloc builds.

I've added an automake dependency for building hwloc.

If we always use fixed version of the library, I prefer CMake to use the pre-built binaries. Building it from sources is unnecessary overhead.

WRT adding dependency on automake for building hwloc, I'd like @AlexeySachkov to approve this.

@PatKamin
Copy link
Author

If we always use fixed version of the library, I prefer CMake to use the pre-built binaries. Building it from sources is unnecessary overhead.

I can make it a fallback: first search for hwloc binaries installed in the system, if not found - fetch hwloc from sources. I would also remove specifying the exact hwloc version in llvm leaving it to UMF to decide on version. @bader, what do you think?

@bader
Copy link
Contributor

bader commented Jan 28, 2025

If we always use fixed version of the library, I prefer CMake to use the pre-built binaries. Building it from sources is unnecessary overhead.

I can make it a fallback: first search for hwloc binaries installed in the system, if not found - fetch hwloc from sources. I would also remove specifying the exact hwloc version in llvm leaving it to UMF to decide on version. @bader, what do you think?

I suggest we fetch binaries instead of sources.
https://www.open-mpi.org/software/hwloc/v2.9/ has links to the binaries for different platforms.

hwloc handling must be done in UMF CMake files.

@PatKamin
Copy link
Author

WRT adding dependency on automake for building hwloc, I'd like @AlexeySachkov to approve this.

@AlexeySachkov, is it ok to add automake dependency?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants