Using onnxruntime-gpu

[luzhenyv]

I have installed onnxruntime-gpu to do SAM inference. It was OK and seemed faster, but I am not sure if it's a good idea or not. I am doing some tests. My environments: Win 10, Python 3.10.11, onnxruntime-gpu 1.14.1

As onnxruntime Docs says:

The GPU package encompasses most of the CPU functionality

My Code

    def __init__(self):
        self._image_size = 1024

        if onnxruntime.get_device() == "GPU":
            logger.info("Using GPU for inference.")
            providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
        else:
            logger.info("Using CPU for inference.")
            providers = ["CPUExecutionProvider"]
......
        encoder_path = gdown.cached_download(
            url="https://github.com/wkentaro/labelme/releases/download/sam-20230416/sam_vit_l_0b3195.quantized.encoder.onnx",  # NOQA
            md5="080004dc9992724d360a49399d1ee24b",
        )
        decoder_path = gdown.cached_download(
            url="https://github.com/wkentaro/labelme/releases/download/sam-20230416/sam_vit_l_0b3195.quantized.decoder.onnx",  # NOQA
            md5="851b7faac91e8e23940ee1294231d5c7",
        )

        self._encoder_session = onnxruntime.InferenceSession(encoder_path, providers=providers)
        self._decoder_session = onnxruntime.InferenceSession(decoder_path, providers=providers)