Skip to content

Field editors are force-closed by an irrelevant visibility change in assistive tech browser. #11022

@kmcnaught

Description

@kmcnaught

Describe the bug

When accessing MakeCode via an embedded browser in an assistive tech interface, loading a new 'page' in the AT software causes a visibilitychange event, and this closes the field editor. The result is that if you need to change 'page' to access a qwerty keyboard (or even, a subset of a qwerty keyboard if you require large targets) then you keep having to resume edit focus, and/or get tied in knots due to accidentally issuing a single-character shortcut when you thought the field was being edited but it wasn't.

Background context

  • I am using Grid3 which is an extremely common interface used for kids using eye gaze and switch access
  • My users access the interface using eye gaze, others might use switch access
  • The interface uses keyboard controls to code (so we are currently using micro:bit, but in the future will use other platforms when keyboard accessibility is added. The bug is in pxt core.)
  • The interface ("pageset") has an embedded browser in it, in which the website is loaded
  • In order to provide large targets for eye gaze access, we only have a limited subset of shortcuts available at any one time
  • When you need to type, you change the page to load a typing keyboard
  • In the act of changing page, the field is closed

To Reproduce

Using a Grid 3 interface (I can provide files and a demo on request but I'm hoping the fix is non-controversial so it won't be required):

  • Navigate to a text field using arrows
  • Press Enter
    • (you are now editing the text field)
  • Change to a typing keyboard pageset, to type
    • you are NOT editing the text field any more
    • you have to press Enter again to edit

Note that the problem does not occur at the blockly-keyboard-experimentation test page as it's caused by the MakeCode simulator state management. It is not an issue within the keyboard controls themselves.

Expected behavior

Changing page in Grid 3 should not close a field you are in the middle of editing.

Screenshots

Here's a two minute video demo explaining the problem.

In pictures... in the following screenshot I have just moved to an editable field and pressed Enter to edit it

moved to an editable field and pressed Enter to edit it

And now i've changed to a page where I have numbers available for typing, and the field has been automatically closed:

changed to a page where I have numbers available for typing, and the field has been automatically closed

Desktop (please complete the following information):

  • OS: Win11
  • Browser: Embedded browser in Grid 3, I believe it is based on Edge.
  • Version: live makecode.microbit.org.

Proposed fix

The issue occurs here:

pxt/webapp/src/app.tsx

Lines 489 to 494 in 0b5a27c

} else if (this.state.resumeOnVisibility) {
this.setState({ resumeOnVisibility: false });
// We did a save when the page was hidden, no need to save again.
this.runSimulator();
cmds.maybeReconnectAsync(false, true);
} else if (!this.state.home) {

If we change it to:

- this.runSimulator();
+ this.runSimulator({ background: true });

Then the simulator re-runs without beforeCompile force closing the editor. This is the same approach taken by autoRunSimulatorDebounced which also needs to re-run without assuming that the user has finished their edits.

I'm opening a PR in parallel with the fix.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions