Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent reporting of UI Automation elements #3530

Open
DaveTryon opened this issue Jun 1, 2023 · 3 comments
Open

Inconsistent reporting of UI Automation elements #3530

DaveTryon opened this issue Jun 1, 2023 · 3 comments
Assignees
Labels
bug Something isn't working

Comments

@DaveTryon
Copy link

Description
The WebView2 control is inconsistent in what UIA elements it reports. The first time its UIA elements are enumerated, only a partial set is returned. On subsequent enumerations, the full set is consistently returned. This occurs in Edge, in the WebView2 samples, and with a tiny test app that we created separately. This behavior does not occur in Chrome, so it seems unlikely that this is a Chromium bug.

Version
SDK: Unknown
Runtime: 113.0.1774.57 (as reported by Control Panel)
Framework: WinForms (also happens with WPF)
OS: Win11 22H2

Repro Steps

  1. Install the Axe.Windows CLI from https://github.com/microsoft/axe-windows/releases/tag/v2.1.2
  2. Build the WebView2WindowsFormsBrowser test app from https://github.com/MicrosoftEdge/WebView2Samples
  3. Run WebView2WindowsFormsBrowser.exe
  4. Open task manager to get the main process. This will be listed as WebView2WindowsFormsBrowser in the UI--in this screenshot, the processID we want is 9196
    image
  5. Run the Axe.Windows CLI tool and note how many errors are reported
  6. Run the Axe.Windows CLI tool again and note that the error count has changed
  7. You can run the Axe.Windows CLI tool additional times and note that the error count remains constant
  8. The error count should remain constant, but the first call doesn't enumerate the entire tree. This is demonstrable by opening the generated .a11ytest files in Accessibility Insight for Windows and comparing the reported hierarchies. As demonstrated below, the contents of the Document control are completely absent from the first test.

Screenshots

Iteration Command line (exe is on path) Errors reported
1 AxeWindowsCLI.exe --outputdirectory %userprofile%\desktop --scanid 1 --processid 9196 23
2 AxeWindowsCLI.exe --outputdirectory %userprofile%\desktop --scanid 2 --processid 9196 195
3 AxeWindowsCLI.exe --outputdirectory %userprofile%\desktop --scanid 3 --processid 9196 195

Comparisons of 1st iteration and 2nd iteration in Accessibility Insights for Windows. The selected element is identical in each, but the first set of results omit the Document control containing the HTML.

Call # Returned UIA hierarchy
1 image
2 image

Additional context

@DaveTryon DaveTryon added the bug Something isn't working label Jun 1, 2023
DaveTryon added a commit to microsoft/axe-windows that referenced this issue Jun 2, 2023
#### Details

microsoft/accessibility-insights-windows#1625
points out that the `ChromiumComponentsShouldUseWebScanner` rule is
flagging items that are outside the context of a web page, which goes
against the rule's primary purpose. This change limits the
`ChromiumComponentsShouldUseWebScanner` rule to elements within a HTML
document (including the document itself).

While working on this, we discovered a bug in WebView2, which is tracked
at MicrosoftEdge/WebView2Feedback#3530.

##### Motivation

Address
microsoft/accessibility-insights-windows#1625

##### Before & after

###### Text results using Edge

As expected, only the `ChromiumComponentsShouldUseWebScanner` rule is
impacted
Rule | Count Before | Count After
--- | --- | ---
An element of the given ControlType must not support the invoke pattern
| 1 | 1
An on-screen element must not have a null BoundingRectangle property | 7
| 7
Chromium components should be scanned with a web-based scanner | 211 |
135
The Name must not include the same text as the LocalizedControlType | 5
| 5
The Name property must not contain only whitespace | 3 | 3
The Name property must not include the element's control type | 1 | 1
The Name property of a focusable element must not be null | 10 | 10

###### Screenshots using Edge

These screenshots are limited to just the
`ChromiumComponentsShouldUseWebScanner` rule:

State | Screenshot
--- | ---
Before change |
![image](https://github.com/microsoft/axe-windows/assets/45672944/13040b65-3ac7-475e-b6ef-6ca2d2a98907)
After change |
![image](https://github.com/microsoft/axe-windows/assets/45672944/07b7faac-d0fe-46c7-a6b4-7991f9f88838)


##### Context

This includes an opportunistic optimization to some of the code in
`Relationships`, because the unit test code looked odd with arbitrary
multipliers to compensate for hoe `Relationships` is coded. This could
be a separate PR if that's preferred.

<!-- Were there any alternative approaches you considered? What
tradeoffs did you consider? -->

#### Pull request checklist
<!-- If a checklist item is not applicable to this change, write "n/a"
in the checkbox -->
- [x] Addresses an existing issue:
microsoft/accessibility-insights-windows#1625
@codeofdusk
Copy link

I'm also able to reproduce this issue in Google Chrome Canary 116.0.5806 when --enable-experimental-ui-automation is passed on the command line, so I suspect this is a bug in Chromium's native UIA provider.

@yunate
Copy link
Contributor

yunate commented Jun 5, 2023

@codeofdusk, thank for your reply!
Since both Chrome and Edge have this issue, it should be an upstream issue, and I will continue to follow up on this issue too.

@KiYugadgeter
Copy link

Is there any update?
I have same problem too

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

5 participants