Inference with multiple GPUs

[AndrewGuo0930]

Hi! I have to say that this tool is so powerful and easy to use with my own MMDetection model, but I still have some problems about inferencing with multiple GPUs. I wonder whether I can run the predicting procedure with multiple GPUs or not.
I've tried the code below to infer with multiple GPUs, but it seems that when I changed the number of GPU, the inference speed didn't change.
Could you please tell me how can I infer with multiple GPUs? I would appreciate it if you could give me some advice. Thank you!

#!/bin/bash
#SBATCH -J sahi
#SBATCH -p gpu
#SBATCH -N 1
#SBATCH --output=log.%j.out
#SBATCH --error=log.%j.err 
#SBATCH --gres=gpu:4

sahi predict \
    --source ../datasets/val_images \
    --model_type mmdet \
    --model_path ../mmdetection/work_dirs/atss_r50_fpn_1x_sat_baseline/epoch_1.pth \
    --model_config_path ../mmdetection/work_dirs/atss_r50_fpn_1x_sat_baseline/atss_r50_fpn_1x_sat_baseline.py \
    --model_device cuda \
    --project runs/VISO \
    --name baseline
    --export_visual true \
    --dataset_json_path runs/VISO \
    --postprocess_type NMS
    --slice_width 256 \
    --slice_height 256 \
    --overlap_height_ratio 0.1 \
    --overlap_width_ratio 0.1 \
    --model_confidence_threshold 0.25

[fcakyon]

@AndrewGuo0930 sahi currently doesn't support multi GPU inference, we are open to any PR on that :)