Closes #56

Remaining work:

• Raise more events (children changed, bounds changed...),
• Get rid of the strum crate and push more event related types to the atspi crate,
• Implement keyboard event handling.

This took me longer than expected.

I am not really proud of the UnignoredChildren iterator implementation, I don't find it elegant but I can't think of another way to do it right now.

Questions

• Should we implement UnignoredChildren::len()? I can't find a good value to return for UnignoredChildren::size_hint because you'll have to iterate to the end to get the correct size, so maybe the default is fine.
• I'm not sure about the unignored_children_reverse test first assertion. The root node clearly has more unignored children, but given that one descendant got an unignored parent, it's a stopping condition. I couldn't find such test case inside Chromium's codebase.

:robot: I have created a release *beep* *boop*

This PR was generated with Release Please. See documentation.

:robot: I have created a release *beep* *boop*

This PR was generated with Release Please. See documentation.

The really ugly part of this is the dynamic subclassing (in platforms/macos/src/subclass.rs). But since winit doesn't currently provide a direct way to use a subclass of WinitView, and I have to implement some accessibility methods on the view (starting with accessibilityChildren, but there will be others), I have to do this. And I've got other projects in the works that will require this hack, retrofitting accessibility onto Unity and a GLFW-based GUI. I'm already doing something similar on Windows, though Win32 subclassing is better documented and (I think) more widely used. I'm open to a cleaner solution in winit, but in order to show results immediately, I'm going with this hack for now.

This allows screen readers and other ATs to provide the expected feedback when editing text on Windows.

The easiest way to try it out with a working application is to try one of the examples in the corresponding egui branch.

The primary missing feature is support for rich text, including text formatting information.

This ought to have tests, but I figured I'd get the implementation into code review.

Steps to reproduce:

1. Check out and build this commit of AccessKit.
2. On x64 Windows 10 build 19043.1237 (21H1), run NVDA 2021.2.
3. Run the hello_world example program under platforms/windows/examples.

Expected result: NVDA should say "Hello world window". This indicates that the UIA provider is properly exposed and NVDA is recognizing the window as a UIA window.

Observed: NVDA says "Hello world". This is what it says for a window for which it can't detect a UIA provider. However, if you reverse the order of the steps (start the program and then start NVDA), it says "Hello world window", indicating that it detected the UIA provider.

My analysis: I know that NVDA is behaving as it is because when it calls UiaHasServerSideProvider on the newly created window, that function is returning false. But I don't know why UiaHasServerSideProvider is returning false for that window. The window procedure handles WM_GETOBJECT by creating the UIA provider and passing that along with the WPARAM and LPARAM to UiaReturnRawElementProvider. It doesn't do anything else with the WPARAM and LPARAM, since I want to let UIA Core handle both UIA and MSAA requests. So I guess there's some kind of race condition where, immediately after the window is created, UIA Core running in NVDA can't connect to UIA Core running in the example program. FWIW, NVDA is 32-bit x86 while the example program is x64.

I have a workaround. If I call CoInitializeEx with COINIT_MULTITHREADED at the start of the program, then this unexpected behavior doesn't happen. But if I either omit a call to CoInitializeEx or call it with COINIT_APARTMENTTHREADED, then I get this problem. So for now, I can just use COINIT_MULTITHREADED, but since AccessKit is a library aiming for wide adoption, I want it to work in any of these cases if possible. Also, it doesn't seem to matter whether I use ProviderOptions_UseComThreading or not, though I'd prefer not to.

I've reported this to one of my former colleagues at Microsoft and his manager, since this looks like it might be a bug in UIA Core.

UIA itself normally does this automatically. But for an AT that uses the UIA Remote Operations API introduced in Windows 11, such as the Windows 11 version of Narrator, this fallback isn't available for providers that don't use COM threading, such as ours. As a result, Windows 11 Narrator was unable to find the window context for the current focus if something other than the window is immediately focused. We don't want to switch to COM threading, since the ability to run UIA methods independently of the UI thread is a key benefit of the AccessKit architecture. We also don't want to require integrators to provide the window title as the name of the root node if we can avoid it. So, this PR implements our own fallback. It also modifies the Windows example and test code to clarify that the window title doesn't need to be explicitly provided in the AccessKit tree.

:robot: I have created a release *beep* *boop*

This PR was generated with Release Please. See documentation.

:robot: I have created a release *beep* *boop*

This PR was generated with Release Please. See documentation.

:robot: I have created a release beep boop

This PR was generated with Release Please. See documentation.

This is a massive breaking change for all users. But I'm afraid it's unavoidable.

This is still a work in progress. The technique is proven to work, but I still need to do the following:

• Use macros to eliminate as much getter/setter boilerplate as possible
• Get serialization, deserialization, and JSON schema support working again
• Use packed bit flags instead of individual bool fields

fixes #199

Switching from our own AppKit bindings to the new ones in icrate isn't strictly necessary to fix the bug, but I figured it was a good idea anyway.

@madsmtm Please review.

I'm in the process of doing several things more or less at the same time:

• documenting the dbus XML interfaces in at-spi2-core
• Fixing incorrect method signatures as part of making the XML the single source of truth

The result is that the dbus XML files in at-spi2-core are changing to reflect what the C code actually does. Bindings that use the XML as a basis will of course want to be notified of this.

So, this issue is me asking you for how to best communicate this.

I can file a bug report here every time I make a change, but this doesn't help other projects (e.g. GTK4, WebKit) that also use DBus directly instead of atk.

(Concretely, my latest fix is in this commit - not merged yet, hopefully in a merge request soon.)

The patterns! macro previously both generated the ResolvedPlatformNode::pattern_provider and ResolvedPlatformNode::enqueue_pattern_property_changes methods, and also generated some of the boilerplate methods for the #[implement(_)] derivations from the windows crate.

In windows 0.32 the implement macro was reworked to use traits (https://github.com/microsoft/windows-rs/pull/1450). As a result, we can't just derive the methods for all patterns in the existing patterms! macro invocation, because:

• The per-trait methods need to go into separate impl IFooBarProvider_Impl for FooBarProvider trait impls.
• The patterns are a subset of the trait methods that need to be implemented.

In #102 I took the simple approach of duplication: we now manually implement the boilerplate methods in addition to specifying their details inside the patterns! macro. It would be nice to figure out how to rework the patterns! macro to re-introduce the previous de-duplication.

UIA doesn't have direct control type equivalents for all AccessKit roles. For that matter, AccessKit doesn't have roles for all control types I've seen, e.g. in the Druid widget gallery. But UIA does have a LocalizedControlType property that we can use to provide custom control types as strings. One question is whether AccessKit should provide its own localized control types, or delegate that to the application. Different applications and higher-level frameworks will have their own approaches to localization, and that might not be something that we want to take on ourselves.

For some control types, such as a slider, it can be useful to compute the node's value field from one or more labels, as we can already do with name. ARIA doesn't seem to have a property that we can emulate here, like with labelled_by, but I think we should still have this.