EFS is a high-performance, multithreaded file search utility written in Rust. It's designed to quickly scan large directory structures and locate files based on various criteria.
This came from Windows search being too slow in our petabyte scale file share.
This concurrent file search program utilizes several key components:
- VecDeque: Used for the directory stack, allowing efficient push and pop operations from both ends.
- Vec: Stores the found files matching the target criteria.
- PathBuf: Represents file and directory paths throughout the program.
The SearchState struct encapsulates the shared state across threads:
struct SearchState {
directory_stack: Arc<Mutex<VecDeque<PathBuf>>>,
found_files: Arc<Mutex<Vec<PathBuf>>>,
target_file: String,
}| Dir Size (GB) | Recurrsive (ms) | Concurent (ms) | CI |
|---|---|---|---|
| 3.5 | 273 | 165 | +-40 |
| 113 | 2771 | 1340 | +- 150 |
| 250 | 165890 | 91348 | +-350 |
- 🚀 Fast: Utilizes multithreading for parallel directory traversal
- 🌳 Efficient: Implements a tree-based search algorithm
- 💻 Cross-platform: Works on Windows, macOS, and Linux
Ensure you have Rust installed on your system. If not, install it from rustup.rs.
Then, clone this repository and build the project:
git clone https://github.com/yen936/file_sys_search.git
cd file-system-search
cargo build --releaseThe compiled binary will be available in target/.
Run the program with
cd file_sys_search
touch search_dirs.txt
echo "<your-target-file>" >> search_dirs.txt
cargo run
EFS is designed to be significantly faster than traditional single-threaded search utilities, especially on systems with multiple CPU cores and when searching across large directory structures or network-attached storage.
Contributions are welcome! Please feel free to submit a Pull Request.
This project is licensed under the MIT License - see the LICENSE file for details.
- Implement regex support for file name matching
- Add support for content-based search
- Create a simple GUI interface
Made with ❤️ and Rust