Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make parthenon manager play nicely with other MPI libraries #1172

Merged
merged 3 commits into from
Sep 11, 2024
Merged

make parthenon manager play nicely with other MPI libraries #1172

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ec4bec2
make parthenon manager play nicely with other MPI libraries
jonahm-LANL Sep 10, 2024
cbdc096
changelog
jonahm-LANL Sep 10, 2024
06c6172
Merge branch 'develop' into jmm/dont-init-mpi-twice
Yurlungur Sep 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@


### Changed (changing behavior/API/variables/...)

- [[PR1172]](https://github.com/parthenon-hpc-lab/parthenon/pull/1172) Make parthenon manager robust against external MPI init and finalize calls

### Fixed (not changing behavior/API/variables/...)

Expand Down
24 changes: 9 additions & 15 deletions src/parthenon_manager.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,26 +50,18 @@ ParthenonStatus ParthenonManager::ParthenonInitEnv(int argc, char *argv[]) {

// initialize MPI
#ifdef MPI_PARALLEL
if (MPI_SUCCESS != MPI_Init(&argc, &argv)) {
std::cout << "### FATAL ERROR in ParthenonInit" << std::endl
int mpi_initialized;
PARTHENON_MPI_CHECK(MPI_Initialized(&mpi_initialized));
if (!mpi_initialized && (MPI_SUCCESS != MPI_Init(&argc, &argv))) {
std::cerr << "### FATAL ERROR in ParthenonInit" << std::endl
<< "MPI Initialization failed." << std::endl;
return ParthenonStatus::error;
}
// Get process id (rank) in MPI_COMM_WORLD
if (MPI_SUCCESS != MPI_Comm_rank(MPI_COMM_WORLD, &(Globals::my_rank))) {
std::cout << "### FATAL ERROR in ParthenonInit" << std::endl
<< "MPI_Comm_rank failed." << std::endl;
// MPI_Finalize();
return ParthenonStatus::error;
}
PARTHENON_MPI_CHECK(MPI_Comm_rank(MPI_COMM_WORLD, &(Globals::my_rank)));

// Get total number of MPI processes (ranks)
if (MPI_SUCCESS != MPI_Comm_size(MPI_COMM_WORLD, &Globals::nranks)) {
std::cout << "### FATAL ERROR in main" << std::endl
<< "MPI_Comm_size failed." << std::endl;
// MPI_Finalize();
return ParthenonStatus::error;
}
PARTHENON_MPI_CHECK(MPI_Comm_size(MPI_COMM_WORLD, &Globals::nranks));
#else // no MPI
Globals::my_rank = 0;
Globals::nranks = 1;
Expand Down Expand Up @@ -232,7 +224,9 @@ ParthenonStatus ParthenonManager::ParthenonFinalize() {
pmesh.reset();
Kokkos::finalize();
#ifdef MPI_PARALLEL
MPI_Finalize();
int mpi_finalized;
PARTHENON_MPI_CHECK(MPI_Finalized(&mpi_finalized));
if (!mpi_finalized) PARTHENON_MPI_CHECK(MPI_Finalize());
#endif
return ParthenonStatus::complete;
}
Expand Down
Loading