A Hand Gesture Control Module based on Google's Mediapipe and a custom model build on top of it for gesture classification.
View Demo
·
Report Bug
·
Request Feature
Table of Contents
HGC is an easy to use module for hand gesture control in any application. the module facilitates creation of a custom dataset, training a model on it, and using it to control any application. The module is based on Google's Mediapipe for hand detection and tracking, and uses a custom trained model build on top of it for gesture classification.
Mediapipe is an amazing base for such an application since its incredably fast and optimizied for CPUs and Edge devices. The demo below shows Mediapipe in action.
This demo shows the module in action. The module is able to detect and track the hand, classify the gesture, and control the drone all in real time.
Here is a graph of how the application for drone control is constructed.
The project uses the following frameworks and packages: \
-
Core:
-
Dataset Collection & Training:
-
Applications:
- djitellopy, For controlling the Dji Tello Drone Github link
- pycaw, For controlling Audio for windows Github link
To get started you need to install the module dependencies. If you will use the pre-trained model on 8 gestures, you can skip the training part. If you want to train your own custom model, you need to collect a dataset first. The module provides a simple way to collect a dataset, and train a model on it.
- Install Requirements:
pip install -r requirements.txt
Please check the comments in the requirements file and remove what you don't need before installing to avoid unnecessary installations.
- Step 1: Clone the repo using
or simply click on "Download ZIP" from the repo page.
git clone https://github.com/FaragSeif/HGCv1.git
- Step 2: Run the dataset collection script if you want to collect your own custom gestures dataset.
where:
python create_dataset.py -n 3 -s 1000 -l label1 label2 label3 -p path/to/dataset
- -n is the number of gestures you want to collect
- -s is the number of samples you want to collect for each gesture
- -l is the labels of the gestures
- -p is the path to the dataset folder
- Step 3: Create your application that uses and HGClassifier object and give it the path to the TFlite model you want to use (either custom or pre-trained), then specify your video stream source (0 is the defualt for most laptops). Then, use the HGClassifier object to classify the gesture and control your desired application.
from HGCv1.models import HGClassifier classifier = HGClassifier(model_path='path/to/model', src=0) command, image = hg_classifier.detect(draw_on_image=True)
In the future, we are planning:
- Add more applications examples to the module
- Publish the module on PyPi.
- Make it more user friendly.
- Add a GUI for the dataset collection and training process...?
Contributions are what make the open source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature
) - Commit your Changes (
git commit -m 'Add some AmazingFeature'
) - Push to the Branch (
git push origin feature/AmazingFeature
) - Open a Pull Request
Distributed under the MIT License. See LICENSE.txt
for more information.
Seif Farag - @FaragSeif - [email protected]
Nabila Adawy - @NabilaAdawy - [email protected]
Project Link: https://github.com/FaragSeif/HGCv1