This repository has been archived by the owner on Apr 16, 2024. It is now read-only.
Add Image Query docs #219
Merged
Add Image Query docs #219
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
andrewfrench
force-pushed
the
french/240213_image-query-docs
branch
from
687a157 to
36c4491
Compare
|
|
||
| ## OpenAiVisionImageQueryDriver | ||
|
|
||
| The [OpenAiVisionImageQueryDriver](../../reference/griptape/drivers/image_query/openai_vision_image_query_driver.md) is used to query images using the OpenAI Vision API. Here is an example of how to use it: |
There was a problem hiding this comment.
Here is an example of how to use it: this should probably be placed right before the example.
|
|
||
| The [OpenAiVisionImageQueryDriver](../../reference/griptape/drivers/image_query/openai_vision_image_query_driver.md) is used to query images using the OpenAI Vision API. Here is an example of how to use it: | ||
|
|
||
| This Driver defaults to using the `gpt-4-vision-preview` model. As other multimodal models are released, they can be specified using the `model` field. While the `max_tokens` field is optional, it is recommended to set this to a value that corresponds to the desired response length. Without an explicit value, the model will default to very short responses. See [OpenAI's documentation](https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them) for more information on how to relate token count to response length. |
There was a problem hiding this comment.
Maybe put this in a !!! info?
collindutter
force-pushed
the
french/240213_image-query-docs
branch
from
19db397 to
57cde8e
Compare
collindutter
previously approved these changes
Feb 28, 2024
collindutter
approved these changes
Feb 28, 2024
collindutter
added a commit
that referenced
this pull request
Feb 29, 2024
* Multi-agent shared vector database (#202) * add example * create env variables * add md file * Update pull-request-links.yml * testing action * testing action * testing action * testing action * action works * PR updates * update to shared memory * vectors * Remove reference to `create_index()` for `VectorStoreDriver`s (#205) * remove reference to create_index() * Update image tool documentation for task memory support (#206) * Update ImageLoader interface (#208) * Add example mappings for Amazon OpenSearch index creation (#209) * Add example mappings for Amazon OpenSearch index creation * Removed carryover comment * Refactor header structure, fix a couple headings (#211) * Fix marqo query example (#214) * Fix directory structure to match new layout (#215) * Updated ToS (#213) * Updated Terms of Service Added instructions for generating from .docx with Pandoc * Updated Terms of Service Added instructions for generating from .docx with Pandoc * Switch to rtf format * Fix pandoc errors * More pandoc errors Maybe Pandox was a bad idea * Hack to exclude code snippets from pytest (#216) * Run integration tests on PRs (#217) * Run integration tests on PRs * Fix env vars * Update deps * Update readthedocs build def (#223) * Fix integration tests (#220) * Run integration tests on PRs * Fix env vars * Fix some examples * test code blocks in specific path/file with CODE_PATH env var * update engine tests to define prompt driver * Fix test failures in vector store drivers * Fix task memory example execution * Fix tasks example execution * Update prompt driver * Update env var thing * Add pgvector service container * Wire up pgvector vars * save workflow config * Replicate .csv assets locally * Test workflow updates * Update dependencies * Update workflow * Update spec * Widen test scope to all integration tests * misc and example fixes * Remove dev dependency * wire env var --------- Co-authored-by: Andrew French <[email protected]> * Add Image Query docs (#219) * Update docs for new config classes (#218) * Update docs for new config classes * Fix links * Revert poetry lock to main * Remove docs gruop * Ignore assets * Update env * Update config * Add back assets * Revert poetry * Revert group * Regerate lock file * Regenerate lockfile * Fixes * Fix tests * Version bump v0.23.0 --------- Co-authored-by: Matt Vallillo <[email protected]> Co-authored-by: Collin Dutter <[email protected]> Co-authored-by: cjkindel <[email protected]>
collindutter
added a commit
that referenced
this pull request
Mar 27, 2024
* Multi-agent shared vector database (#202) * add example * create env variables * add md file * Update pull-request-links.yml * testing action * testing action * testing action * testing action * action works * PR updates * update to shared memory * vectors * Remove reference to `create_index()` for `VectorStoreDriver`s (#205) * remove reference to create_index() * Update image tool documentation for task memory support (#206) * Update ImageLoader interface (#208) * Add example mappings for Amazon OpenSearch index creation (#209) * Add example mappings for Amazon OpenSearch index creation * Removed carryover comment * Refactor header structure, fix a couple headings (#211) * Fix marqo query example (#214) * Fix directory structure to match new layout (#215) * Updated ToS (#213) * Updated Terms of Service Added instructions for generating from .docx with Pandoc * Updated Terms of Service Added instructions for generating from .docx with Pandoc * Switch to rtf format * Fix pandoc errors * More pandoc errors Maybe Pandox was a bad idea * Hack to exclude code snippets from pytest (#216) * Run integration tests on PRs (#217) * Run integration tests on PRs * Fix env vars * Update deps * Update readthedocs build def (#223) * Fix integration tests (#220) * Run integration tests on PRs * Fix env vars * Fix some examples * test code blocks in specific path/file with CODE_PATH env var * update engine tests to define prompt driver * Fix test failures in vector store drivers * Fix task memory example execution * Fix tasks example execution * Update prompt driver * Update env var thing * Add pgvector service container * Wire up pgvector vars * save workflow config * Replicate .csv assets locally * Test workflow updates * Update dependencies * Update workflow * Update spec * Widen test scope to all integration tests * misc and example fixes * Remove dev dependency * wire env var --------- Co-authored-by: Andrew French <[email protected]> * Add Image Query docs (#219) * Update docs for new config classes (#218) * Update docs for new config classes * Fix links * Revert poetry lock to main * Remove docs gruop * Ignore assets * Update env * Update config * Add back assets * Revert poetry * Revert group * Regerate lock file * Regenerate lockfile * Fixes * Fix tests * Update docs for loader extras (#210) * Update docs for loader extras * Add extras to loaders, add missing loader docs * Remove cloud links (#225) * Add Image Query Task docs (#226) * Remove references to default tokenizers (#212) * Update test deps (#233) * Update test deps * Fix tokenizer examples * Update simple tokenizer * Fix missing args * Update lock * Fix import * Fix missing models * Ignore marqo * Fix model names * Add docs for voyageai (#230) * Add docs for voyageai * Fix example, add key * Fix key * Regenerate lock * Add docs for gemini (#231) * Add docs for gemini * Fix reference links * Claude 3 image query docs (#234) * adding anthropic and bedrock image query docs * updating max_token name * fixing typo * updating to remove openai default model for image query driver * updated wording * updating path for image file * docs: max_tokens documentation for vision * updating engines page for max_tokens specification --------- Co-authored-by: Michael Bianco <[email protected]> * Add web scraper driver docs (#227) * Update docs for Anthropic (#229) * Update docs for Anthropic * Link to example of overriding * Update link * Fix url * Update deps * Version bump v0.24.0 * Update poetry.lock --------- Co-authored-by: Matt Vallillo <[email protected]> Co-authored-by: Andrew French <[email protected]> Co-authored-by: Collin Dutter <[email protected]> Co-authored-by: Zach Giordano <[email protected]> Co-authored-by: Emily Danielson <[email protected]> Co-authored-by: Michael Bianco <[email protected]> Co-authored-by: dylanholmes <[email protected]>
collindutter
added a commit
that referenced
this pull request
Mar 27, 2024
* Release version v0.23.0 (#224) * Multi-agent shared vector database (#202) * add example * create env variables * add md file * Update pull-request-links.yml * testing action * testing action * testing action * testing action * action works * PR updates * update to shared memory * vectors * Remove reference to `create_index()` for `VectorStoreDriver`s (#205) * remove reference to create_index() * Update image tool documentation for task memory support (#206) * Update ImageLoader interface (#208) * Add example mappings for Amazon OpenSearch index creation (#209) * Add example mappings for Amazon OpenSearch index creation * Removed carryover comment * Refactor header structure, fix a couple headings (#211) * Fix marqo query example (#214) * Fix directory structure to match new layout (#215) * Updated ToS (#213) * Updated Terms of Service Added instructions for generating from .docx with Pandoc * Updated Terms of Service Added instructions for generating from .docx with Pandoc * Switch to rtf format * Fix pandoc errors * More pandoc errors Maybe Pandox was a bad idea * Hack to exclude code snippets from pytest (#216) * Run integration tests on PRs (#217) * Run integration tests on PRs * Fix env vars * Update deps * Update readthedocs build def (#223) * Fix integration tests (#220) * Run integration tests on PRs * Fix env vars * Fix some examples * test code blocks in specific path/file with CODE_PATH env var * update engine tests to define prompt driver * Fix test failures in vector store drivers * Fix task memory example execution * Fix tasks example execution * Update prompt driver * Update env var thing * Add pgvector service container * Wire up pgvector vars * save workflow config * Replicate .csv assets locally * Test workflow updates * Update dependencies * Update workflow * Update spec * Widen test scope to all integration tests * misc and example fixes * Remove dev dependency * wire env var --------- Co-authored-by: Andrew French <[email protected]> * Add Image Query docs (#219) * Update docs for new config classes (#218) * Update docs for new config classes * Fix links * Revert poetry lock to main * Remove docs gruop * Ignore assets * Update env * Update config * Add back assets * Revert poetry * Revert group * Regerate lock file * Regenerate lockfile * Fixes * Fix tests * Version bump v0.23.0 --------- Co-authored-by: Matt Vallillo <[email protected]> Co-authored-by: Collin Dutter <[email protected]> Co-authored-by: cjkindel <[email protected]> * Version bump v0.24.0 (#237) * Multi-agent shared vector database (#202) * add example * create env variables * add md file * Update pull-request-links.yml * testing action * testing action * testing action * testing action * action works * PR updates * update to shared memory * vectors * Remove reference to `create_index()` for `VectorStoreDriver`s (#205) * remove reference to create_index() * Update image tool documentation for task memory support (#206) * Update ImageLoader interface (#208) * Add example mappings for Amazon OpenSearch index creation (#209) * Add example mappings for Amazon OpenSearch index creation * Removed carryover comment * Refactor header structure, fix a couple headings (#211) * Fix marqo query example (#214) * Fix directory structure to match new layout (#215) * Updated ToS (#213) * Updated Terms of Service Added instructions for generating from .docx with Pandoc * Updated Terms of Service Added instructions for generating from .docx with Pandoc * Switch to rtf format * Fix pandoc errors * More pandoc errors Maybe Pandox was a bad idea * Hack to exclude code snippets from pytest (#216) * Run integration tests on PRs (#217) * Run integration tests on PRs * Fix env vars * Update deps * Update readthedocs build def (#223) * Fix integration tests (#220) * Run integration tests on PRs * Fix env vars * Fix some examples * test code blocks in specific path/file with CODE_PATH env var * update engine tests to define prompt driver * Fix test failures in vector store drivers * Fix task memory example execution * Fix tasks example execution * Update prompt driver * Update env var thing * Add pgvector service container * Wire up pgvector vars * save workflow config * Replicate .csv assets locally * Test workflow updates * Update dependencies * Update workflow * Update spec * Widen test scope to all integration tests * misc and example fixes * Remove dev dependency * wire env var --------- Co-authored-by: Andrew French <[email protected]> * Add Image Query docs (#219) * Update docs for new config classes (#218) * Update docs for new config classes * Fix links * Revert poetry lock to main * Remove docs gruop * Ignore assets * Update env * Update config * Add back assets * Revert poetry * Revert group * Regerate lock file * Regenerate lockfile * Fixes * Fix tests * Update docs for loader extras (#210) * Update docs for loader extras * Add extras to loaders, add missing loader docs * Remove cloud links (#225) * Add Image Query Task docs (#226) * Remove references to default tokenizers (#212) * Update test deps (#233) * Update test deps * Fix tokenizer examples * Update simple tokenizer * Fix missing args * Update lock * Fix import * Fix missing models * Ignore marqo * Fix model names * Add docs for voyageai (#230) * Add docs for voyageai * Fix example, add key * Fix key * Regenerate lock * Add docs for gemini (#231) * Add docs for gemini * Fix reference links * Claude 3 image query docs (#234) * adding anthropic and bedrock image query docs * updating max_token name * fixing typo * updating to remove openai default model for image query driver * updated wording * updating path for image file * docs: max_tokens documentation for vision * updating engines page for max_tokens specification --------- Co-authored-by: Michael Bianco <[email protected]> * Add web scraper driver docs (#227) * Update docs for Anthropic (#229) * Update docs for Anthropic * Link to example of overriding * Update link * Fix url * Update deps * Version bump v0.24.0 * Update poetry.lock --------- Co-authored-by: Matt Vallillo <[email protected]> Co-authored-by: Andrew French <[email protected]> Co-authored-by: Collin Dutter <[email protected]> Co-authored-by: Zach Giordano <[email protected]> Co-authored-by: Emily Danielson <[email protected]> Co-authored-by: Michael Bianco <[email protected]> Co-authored-by: dylanholmes <[email protected]> --------- Co-authored-by: Andrew French <[email protected]> Co-authored-by: Matt Vallillo <[email protected]> Co-authored-by: Collin Dutter <[email protected]> Co-authored-by: Zach Giordano <[email protected]> Co-authored-by: Emily Danielson <[email protected]> Co-authored-by: Michael Bianco <[email protected]> Co-authored-by: dylanholmes <[email protected]>
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add documentation for Image Query Driver, Engine, and Client.
📚 Documentation preview 📚: https://griptape--219.org.readthedocs.build//219/