From 598783bf8c41bc63d640a5a88e6bd8ca8883b264 Mon Sep 17 00:00:00 2001 From: Luc Grosheintz Date: Tue, 3 Dec 2024 14:25:32 +0100 Subject: [PATCH 1/7] Prepare for end of BBP. --- CMakeLists.txt | 5 + README.md | 241 ++----------------------------------------------- 2 files changed, 12 insertions(+), 234 deletions(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index 77ed20190..4aea70998 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -4,6 +4,11 @@ cmake_policy(VERSION 3.13) project(HighFive VERSION 3.0.0) set(HIGHFIVE_VERSION_PRERELEASE 2) +message(WARNING "=================================================================\n" + "HighFive development moved to:\n" + " github.com/highfive-devs/highfive\n" + "=================================================================") + # Configure HighFive # ------------------ option(HIGHFIVE_VERBOSE "Set logging level to verbose." OFF) diff --git a/README.md b/README.md index 3752cf435..542c32062 100644 --- a/README.md +++ b/README.md @@ -1,84 +1,17 @@ -*Note:* In preparation of `v3` of HighFive, we've started merging breaking -changes into the main branch. More information and opportunity to comment can -be found at: -https://github.com/BlueBrain/HighFive/issues/864 +> [!WARNING] +> The BlueBrainProject closed at the end of 2024. This unfortunately required +> archiving all repositories under `BlueBrain` including HighFive. +> +> The development of HighFive will continue at: +> https://github.com/highfive-devs/highfive + # HighFive - HDF5 header-only C++ Library -[![Doxygen -> gh-pages](https://github.com/BlueBrain/HighFive/workflows/gh-pages/badge.svg?branch=master)](https://BlueBrain.github.io/HighFive/actions/workflows/gh-pages.yml?query=branch%3Amaster) -[![codecov](https://codecov.io/gh/BlueBrain/HighFive/branch/master/graph/badge.svg?token=UBKxHEn7RS)](https://codecov.io/gh/BlueBrain/HighFive) -[![HighFive_Integration_tests](https://github.com/BlueBrain/HighFive-testing/actions/workflows/integration.yml/badge.svg)](https://github.com/BlueBrain/HighFive-testing/actions/workflows/integration.yml) [![Zenodo](https://zenodo.org/badge/47755262.svg)](https://zenodo.org/doi/10.5281/zenodo.10679422) -Documentation: https://bluebrain.github.io/HighFive/ - -## Brief - -HighFive is a modern header-only C++14 friendly interface for libhdf5. - -HighFive supports STL vector/string, Boost::UBLAS, Boost::Multi-array and Xtensor. It handles C++ from/to HDF5 with automatic type mapping. -HighFive does not require additional libraries (see dependencies). - -It integrates nicely with other CMake projects by defining (and exporting) a HighFive target. - -### Design -- Simple C++-ish minimalist interface -- Only hard dependency is libhdf5 -- Zero/low overhead, when possible -- RAII for opening/closing files, groups, datasets, etc. -- Written in C++14 - -### Feature support -- create/read/write files, datasets, attributes, groups, dataspaces. -- automatic memory management / ref counting -- automatic conversion of `std::vector` and nested `std::vector` from/to any dataset with basic types -- automatic conversion of `std::string` to/from variable- or fixed-length string dataset -- selection() / slice support -- parallel Read/Write operations from several nodes with Parallel HDF5 -- Advanced types: Compound, Enum, Arrays of Fixed-length strings, References -- half-precision (16-bit) floating-point datasets -- `std::byte` in C++17 mode (with `-DCMAKE_CXX_STANDARD=17` or higher) -- etc... (see [ChangeLog](./CHANGELOG.md)) - -### Dependencies -- HDF5 or pHDF5, including headers -- boost (optional) -- eigen3 (optional) -- xtensor (optional) -- half (optional) - -### Versioning & Code Stability -We use semantic versioning. Currently, we're preparing `v3` which contains a -limited set of breaking changes required to eliminate undesireable behaviour or -modernize outdated patterns. We provide a -[Migration Guide](https://bluebrain.github.io/HighFive/md__2home_2runner_2work_2_high_five_2_high_five_2doc_2migration__guide.html), -please report any missing or incorrect information to help others make the -switch more easily. - -- `v2.x.y` are stable and any API breaking changes are considered bugs. There's - like not going to be very many releases of the `v2` line once `v3` is stable. - -- `v3.0.0-beta?` are pre-releases of `v3.0.0`. We predict that one more - breaking changes might happen: the string handling is confusing to some of the - maintainers and the default encoding is inconsistent (and will likely be made - consistent). - - For codes that either use `std::string` when dealing with strings, or that - don't use strings with HDF5 at all, we don't currently have any additional - breaking changes planned for 3.0.0. - - -### Known flaws -- HighFive is not thread-safe. At best it has the same limitations as the HDF5 library. However, HighFive objects modify their members without protecting these writes. Users have reported that HighFive is not thread-safe even when using the threadsafe HDF5 library, e.g., https://github.com/BlueBrain/HighFive/discussions/675. -- Eigen support in core HighFive was broken until v3.0. See https://github.com/BlueBrain/HighFive/issues/532. H5Easy was not - affected. -- The support of fixed length strings isn't ideal. - - ## Examples -#### Write a std::vector to 1D HDF5 dataset and read it back - ```c++ #include @@ -109,166 +42,6 @@ std::string filename = "/tmp/new_file.h5"; } ``` -**Note:** As of 2.8.0, one can use `highfive/highfive.hpp` to include -everything HighFive. Prior to 2.8.0 one would include `highfive/H5File.hpp`. - -**Note:** For advanced usecases the dataset can be created without immediately -writing to it. This is common in MPI-IO related patterns, or when growing a -dataset over the course of a simulation. - -#### Write a 2 dimensional C double float array to a 2D HDF5 dataset - -See [create_dataset_double.cpp](https://github.com/BlueBrain/HighFive/blob/master/src/examples/create_dataset_double.cpp) - -#### Write and read a matrix of double float (boost::ublas) to a 2D HDF5 dataset - -See [boost_ublas_double.cpp](https://github.com/BlueBrain/HighFive/blob/master/src/examples/boost_ublas_double.cpp) - -#### Write and read a subset of a 2D double dataset - -See [select_partial_dataset_cpp11.cpp](https://github.com/BlueBrain/HighFive/blob/master/src/examples/select_partial_dataset_cpp11.cpp) - -#### Create, write and list HDF5 attributes - -See [create_attribute_string_integer.cpp](https://github.com/BlueBrain/HighFive/blob/master/src/examples/create_attribute_string_integer.cpp) - -#### And others - -See [src/examples/](https://github.com/BlueBrain/HighFive/blob/master/src/examples/) subdirectory for more info. - - -### H5Easy - -For several 'standard' use cases the [highfive/H5Easy.hpp](include/highfive/H5Easy.hpp) interface is available. It allows: - -* Reading/writing in a single line of: - - - scalars (to/from an extendible DataSet), - - strings, - - vectors (of standard types), - - [Eigen::Matrix](http://eigen.tuxfamily.org) (optional), - - [xt::xarray](https://github.com/QuantStack/xtensor) and [xt::xtensor](https://github.com/QuantStack/xtensor) - (optional). - - [cv::Mat_](https://docs.opencv.org/master/df/dfc/classcv_1_1Mat__.html) - (optional). - -* Getting in a single line: - - - the size of a DataSet, - - the shape of a DataSet. - -#### Example - -```cpp -#include - -int main() { - H5Easy::File file("example.h5", H5Easy::File::Overwrite); - - int A = ...; - H5Easy::dump(file, "/path/to/A", A); - - A = H5Easy::load(file, "/path/to/A"); -} -``` - -whereby the `int` type of this example can be replaced by any of the above -types. See [easy_load_dump.cpp](src/examples/easy_load_dump.cpp) for more -details. - -**Note:** Classes such as `H5Easy::File` are just short for the regular -`HighFive` classes (in this case `HighFive::File`). They can thus be used -interchangeably. - - -## CMake integration -There's two common paths of integrating HighFive into a CMake based project. -The first is to "vendor" HighFive, the second is to install HighFive as a -normal C++ library. Since HighFive makes choices about how to integrate HDF5, -sometimes following the third Bailout Approach is needed. - -Regular HDF5 CMake variables can be used. Interesting variables include: - -* `HDF5_USE_STATIC_LIBRARIES` to link statically against the HDF5 library. -* `HDF5_PREFER_PARALLEL` to prefer pHDF5. -* `HDF5_IS_PARALLEL` to check if HDF5 is parallel. - -Please consult `tests/cmake_integration` for examples of how to write libraries -or applications using HighFive. - -### Vendoring HighFive - -In this approach the HighFive sources are included in a subdirectory of the -project (typically as a git submodule), for example in `third_party/HighFive`. - -The projects `CMakeLists.txt` add the following lines -```cmake -add_subdirectory(third_party/HighFive) -target_link_libraries(foo HighFive) -``` - -**Note:** `add_subdirectory(third_party/HighFive)` will search and "link" HDF5 -but wont search or link any optional dependencies such as Boost. - -### Regular Installation of HighFive - -Alternatively, HighFive can be install and "found" like regular software. - -The project's `CMakeLists.txt` should add the following: -```cmake -find_package(HighFive REQUIRED) -target_link_libraries(foo HighFive) -``` - -**Note:** `find_package(HighFive)` will search for HDF5. "Linking" to -`HighFive` includes linking with HDF5. The two commands will not search for or -"link" to optional dependencies such as Boost. - -### Bailout Approach - -To prevent HighFive from searching or "linking" to HDF5 the project's -`CMakeLists.txt` should contain the following: - -```cmake -# Prevent HighFive CMake code from searching for HDF5: -set(HIGHFIVE_FIND_HDF5 Off) - -# Then "find" HighFive as usual: -find_package(HighFive REQUIRED) -# alternatively, when vendoring: -# add_subdirectory(third_party/HighFive) - -# Finally, use the target `HighFive::Include` which -# doesn't add a dependency on HDF5. -target_link_libraries(foo HighFive::Include) - -# Proceed to find and link HDF5 as required. -``` - -### Optional Dependencies - -HighFive does not attempt to find or "link" to any optional dependencies, such -as Boost, Eigen, etc. Any project using HighFive with any of the optional -dependencies must include the respective header: -``` -#include -#include -``` -and add the required CMake code to find and link against the dependencies. For -Boost the required lines might be -``` -find_package(Boost REQUIRED) -target_link_libraries(foo PUBLIC Boost::headers) -``` - -# Questions? - -Do you have questions on how to use HighFive? Would you like to share an interesting example or -discuss HighFive features? Head over to the [Discussions](https://github.com/BlueBrain/HighFive/discussions) -forum and join the community. - -For bugs and issues please use [Issues](https://github.com/BlueBrain/HighFive/issues). - # Funding & Acknowledgment The development of this software was supported by funding to the Blue Brain Project, a research center of the École polytechnique fédérale de Lausanne (EPFL), from the Swiss government's ETH Board of the Swiss Federal Institutes of Technology. From 910c0292c6355d5591e289ea6978fdf124ab2221 Mon Sep 17 00:00:00 2001 From: Luc Grosheintz Date: Tue, 3 Dec 2024 14:42:15 +0100 Subject: [PATCH 2/7] Shorten README.md --- README.md | 16 ++++------------ 1 file changed, 4 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index 542c32062..638b3d411 100644 --- a/README.md +++ b/README.md @@ -13,31 +13,23 @@ ## Examples ```c++ -#include - using namespace HighFive; -std::string filename = "/tmp/new_file.h5"; +File file("foo.h5", File::Truncate); { - // We create an empty HDF55 file, by truncating an existing - // file if required: - File file(filename, File::Truncate); - std::vector data(50, 1); file.createDataSet("grp/data", data); } { - // We open the file as read-only: - File file(filename, File::ReadOnly); auto dataset = file.getDataSet("grp/data"); - // Read back, with allocating: + // Read back, automatically allocating: auto data = dataset.read>(); - // Because `data` has the correct size, this will - // not cause `data` to be reallocated: + // Alternatively, if `data` has the correct + // size, without reallocation: dataset.read(data); } ``` From 52278418bb8ed3bdc6bb801922b83912bf4b7787 Mon Sep 17 00:00:00 2001 From: Luc Grosheintz Date: Wed, 4 Dec 2024 11:19:21 +0100 Subject: [PATCH 3/7] Improve README.md --- README.md | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 638b3d411..2129d37a6 100644 --- a/README.md +++ b/README.md @@ -8,9 +8,14 @@ # HighFive - HDF5 header-only C++ Library -[![Zenodo](https://zenodo.org/badge/47755262.svg)](https://zenodo.org/doi/10.5281/zenodo.10679422) +The releases for versions 2.x.y and two prereleases of v3 can be found at: +* https://github.com/BlueBrain/HighFive/releases +* https://zenodo.org/doi/10.5281/zenodo.10679422 -## Examples +All future development and full releases can be found at: +* https://github.com/highfive-devs/highfive + +## Example ```c++ using namespace HighFive; @@ -35,14 +40,14 @@ File file("foo.h5", File::Truncate); ``` # Funding & Acknowledgment - + The development of this software was supported by funding to the Blue Brain Project, a research center of the École polytechnique fédérale de Lausanne (EPFL), from the Swiss government's ETH Board of the Swiss Federal Institutes of Technology. HighFive releases are uploaded to Zenodo. If you wish to cite HighFive in a scientific publication you can use the DOIs for the [Zenodo records](https://zenodo.org/doi/10.5281/zenodo.10679422). - -Copyright © 2015-2022 Blue Brain Project/EPFL + +Copyright © 2015-2024 Blue Brain Project/EPFL ### License From a02cb8f2ade4ca7f8390c85bc29664f1c58a62db Mon Sep 17 00:00:00 2001 From: Luc Grosheintz Date: Wed, 4 Dec 2024 11:26:48 +0100 Subject: [PATCH 4/7] fix wording --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 2129d37a6..a9ea7e329 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ The releases for versions 2.x.y and two prereleases of v3 can be found at: * https://github.com/BlueBrain/HighFive/releases * https://zenodo.org/doi/10.5281/zenodo.10679422 -All future development and full releases can be found at: +All future development and new releases can be found at: * https://github.com/highfive-devs/highfive ## Example From 842da35b1f25b954c47b3606a5b779e5be2c861c Mon Sep 17 00:00:00 2001 From: Luc Grosheintz Date: Tue, 10 Dec 2024 16:03:17 +0100 Subject: [PATCH 5/7] Refer back. --- README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/README.md b/README.md index a9ea7e329..f11e844f1 100644 --- a/README.md +++ b/README.md @@ -12,6 +12,9 @@ The releases for versions 2.x.y and two prereleases of v3 can be found at: * https://github.com/BlueBrain/HighFive/releases * https://zenodo.org/doi/10.5281/zenodo.10679422 +The state of HighFive immediately before preparing it for archival is: +* https://github.com/BlueBrain/HighFive/tree/v3.0.0-beta2 + All future development and new releases can be found at: * https://github.com/highfive-devs/highfive From af837fd3872bb4df13e87a1a083af8bab0a95f9d Mon Sep 17 00:00:00 2001 From: Pramod Kumbhar Date: Tue, 10 Dec 2024 16:32:38 +0100 Subject: [PATCH 6/7] Update README.md Co-authored-by: Matthias Wolf --- README.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/README.md b/README.md index f11e844f1..868a15b9d 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,5 @@ > [!WARNING] -> The BlueBrainProject closed at the end of 2024. This unfortunately required -> archiving all repositories under `BlueBrain` including HighFive. +> The Blue Brain Project concluded in December 2024, so the HighFive development is ceased under the BlueBrain GitHub organization. > > The development of HighFive will continue at: > https://github.com/highfive-devs/highfive From 84ec830ade2968654a579518c62b2000e4962c25 Mon Sep 17 00:00:00 2001 From: Pramod Kumbhar Date: Thu, 9 Jan 2025 10:14:21 +0100 Subject: [PATCH 7/7] Nitpick - re-add some README information for repo completeness --- README.md | 38 +++++++++++++++++++++++++++++++++++++- 1 file changed, 37 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 868a15b9d..dee6c73db 100644 --- a/README.md +++ b/README.md @@ -4,9 +4,45 @@ > The development of HighFive will continue at: > https://github.com/highfive-devs/highfive - # HighFive - HDF5 header-only C++ Library +Documentation: https://bluebrain.github.io/HighFive/ + +## Brief + +HighFive is a modern header-only C++14 friendly interface for libhdf5. + +HighFive supports STL vector/string, Boost::UBLAS, Boost::Multi-array and Xtensor. It handles C++ from/to HDF5 with automatic type mapping. +HighFive does not require additional libraries (see dependencies). + +It integrates nicely with other CMake projects by defining (and exporting) a HighFive target. + +### Design +- Simple C++-ish minimalist interface +- Only hard dependency is libhdf5 +- Zero/low overhead, when possible +- RAII for opening/closing files, groups, datasets, etc. +- Written in C++14 + +### Feature support +- create/read/write files, datasets, attributes, groups, dataspaces. +- automatic memory management / ref counting +- automatic conversion of `std::vector` and nested `std::vector` from/to any dataset with basic types +- automatic conversion of `std::string` to/from variable- or fixed-length string dataset +- selection() / slice support +- parallel Read/Write operations from several nodes with Parallel HDF5 +- Advanced types: Compound, Enum, Arrays of Fixed-length strings, References +- half-precision (16-bit) floating-point datasets +- `std::byte` in C++17 mode (with `-DCMAKE_CXX_STANDARD=17` or higher) +- etc... (see [ChangeLog](./CHANGELOG.md)) + +### Dependencies +- HDF5 or pHDF5, including headers +- boost (optional) +- eigen3 (optional) +- xtensor (optional) +- half (optional) + The releases for versions 2.x.y and two prereleases of v3 can be found at: * https://github.com/BlueBrain/HighFive/releases * https://zenodo.org/doi/10.5281/zenodo.10679422