Add Headful browsing to Inspect's Web Tooling #1
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR contains:
What is the current behavior? (You can also link to an open issue here)
The web_browser tooling only makes use of headless browser, it is not possible to view this browser graphically in real-time.
What is the new behavior?
This PR implements headful browsing, allowing users to view an agent's interactions with the browser in real time via a VNC viewer.
Tests have been added for this new behaviour in
test_playwright_crawler
. These are essentially just parameterising playwright's headless flag to test bothTrue
andFalse
, instead of justTrue
originally.Does this PR introduce a breaking change? (What changes might users need to make in their application due to this PR?)
This PR should be accompanied by an update to the image (aisiuk/inspect-web-browser-tool) hosted at Dockerhub for web_browser tooling. This PR's implementation defaults to headless mode. For users making use of web_browser tooling who are not aware of this PR, there should be no downstream impacts.
Other information:
This PR's implementation makes use of a layered Dockerfile approach, the user selects a build target in their task's docker-compose file. For example, in the
examples/browser/compose.yaml
file, the user can choose headful browsing with:If a target is not specified, or it is specified as
headless
the Dockerfile will build the headless image. In this instance, it is not necessary to specify the 127.0.0.1:5900:5900 port mapping either (this is only needed for VNC viewing).This PR achieves this broadly by:
True
(i.e. the "HEADLESS" environment variable).Note: When running tests, a chromium browser will open/close for the headful tests, if necessary I'm open to adding decorators to these tests so they are only run when specified.