Self driving excavator project

This project uses Ardiuno motor shield with Raspberry Pi 4 and raspberry camera. Excavator will move near to given object attempt with take it with excavator bucket.

Set up your hardware

Before you begin, you need to set up your Raspberry Pi with Raspbian (preferably updated to Buster).

You also need to connect and configure the Pi Camera.

And to see the results from the camera, you need a monitor connected to the Raspberry Pi. It's okay if you're using SSH to access the Pi shell (you don't need to use a keyboard connected to the Pi)—you only need a monitor attached to the Pi to see the camera stream.

Setup your Ardiuno motor shield

1. Connect Ardiuno motor shield to Raspberry Pi 4 as following schema.

2. Connect your Excavator motors with Ardiuno motor shield motor like above schema.

3. Test with python3 test_motors.py If everything get connected your motors will move one by one.

4. Connect your camera to Raspberry pi 4 board.

5. Test with camera with python3 detect_picamera.py

6. Working on how to detect distances and catch the object with bucket

Prepare your custom model

Tensorflow lite has some models which already have common object (person, apple, ) detections. But if you need to detect some custom object you need to train custom models.

For you need to go through following steps

1. Data preparation

You need to prepare pictures that contain you custom object. In this project I need to train shovel detection.

I took lot of pictures (around 70) with my phone.

2. Labeling your object

You need to identify your custom object from all of your pictures.

I used following simple and small application.

https://github.com/tzutalin/labelImg

3. Prepare more data with converted images.

Roboflow is very good service that you can do lot of things for free https://roboflow.com/

I uploaded all my labeled images and generated custom data set. This will allow you that your trained model detect your object very precisely with different conditions. After this step you will have more pictures for your data set.

4. Convert Pascal VOC formatted (XML) label to Google Auto ML format (CSV)

Google Auto ML is train your model very easily and very cheap.

You need to upload all labeled images (with label) to Google Cloud bucket. I recommended to choose region "us-central-1" region. because when you export trained model you only able to export to "us-central-1" region.

Google Auto ML (Vision) is requires CSV files to import your data set from Google Cloud bucket.

Above LabelImg or free account of Roboflow not allow to generate that CSV file for Google AutoML.

So, you need to prepare data set with Pascal VOC format (XML) convert python script which in "helper" folder of this project.

Open voc_to_csv.py file copy it under your data set folder in your local and put your correct name for following variables.

imgFolderPath_gs = 'gs://<your Google cloud bucket>/<folder of your data set>/'

After this you need to copy CSV file that generated by voc_to_csv.py script and copy to Google Cloud bucket.

Import CSV file from your CSV file to Google AutoML.

5. Train your model for TFLITE

In Google AutoML you can see "Train" tab and you can train model from there.

You might worry about how many node should I use. I would recommended "1" for less than 1000 images.

And wait until your model training get finishes.

6. Put your model train in ./trained_model folder define model arguments

TODO: Update this section once script get finished