Would like a FileQueryTool that can describe any file type

[shhlife]

• I have read and agree to the contributing guidelines.

Is your feature request related to a problem? Please describe.
We have an ImageQueryTool that gives an agent the ability to describe an image. As more models get multi-modal, it'd be nice to have a general purpose FileQueryTool that understands how to handle more artifact types - text, image, audio, video. Then we could simply use that tool and it would choose the appropriate loader

[collindutter]

We can accomplish this behavior with a couple structures like so:

from griptape.drivers import LocalStructureRunDriver
from griptape.structures import Agent, Pipeline
from griptape.tasks import PromptTask, ToolTask
from griptape.tools import FileManagerTool, StructureRunTool

file_query = Pipeline(
    tasks=[
        ToolTask(tool=FileManagerTool(), id="file"),
        PromptTask(lambda task: task.parent_outputs["file"]),
    ],
)

agent = Agent(
    tools=[
        StructureRunTool(
            description="Can be used to describe files",
            structure_run_driver=LocalStructureRunDriver(
                create_structure=lambda: file_query
            ),
        )
    ]
)


agent.run("Describe this file: assets/mountain.jpg")

This actually has me thinking we might not even need an ImageQueryTool. While I admit it is a bit clunkier to set up, I'd rather invest the time in improving structure calling. What do you think?

Relevant: #1432 (comment)

[shhlife]

Interesting.. I do like this pattern, but if I hadn't seen it, I wouldn't have thought about using it. There's a discovery challenge here.. this is super cool, but new users won't know to think about things this way.

when I'm hunting around in code I see ImageQueryTool and I go - oh! I can query an image! That's a feature of Griptape!

Whereas this is way powerful.. but until I'm a lego master builder I won't know I can do this kind of thing. :)

[collindutter]

Totally understood, this pattern didn't come to me immediately either. Maybe this could be a docs recipe? I'd prefer to keep solutions built off of composable elements rather than be bespoke.

[shhlife]

yeah.. I feel like there's something to that. These patterns.. we could put them in recipes.. or is there a way to create scaffolding or something as part of Griptape? do other frameworks do those sorts of things?

like discoverable templates?

[shhlife]

Curious if you've thought any more about these? Like published snippets?