[No QA] Make it easier for AI agents to interact with the app#81583
[No QA] Make it easier for AI agents to interact with the app#81583neil-marcellini merged 14 commits intomainfrom
Conversation
…vent SVG click interception Browser automation tools (and potentially users) have trouble clicking on certain UI elements because SVG icons inside buttons/pressables intercept clicks. The SVG path elements become the click target instead of the parent pressable. This fix adds pointerEvents="none" to: - Icon component's View wrapper and ImageSVG (both main and inline rendering) - FloatingActionButton's Svg component Co-authored-by: Cursor <cursoragent@cursor.com>
melvin-bot
bot
requested review from
joekaufmanexpensify and
youssef-lr
and removed request for
a team
|
@youssef-lr Please copy/paste the Reviewer Checklist from here into a new comment on this PR and complete it. If you have the K2 extension, you can simply click: [this button] |
|
Tests failing @neil-marcellini |
|
No product review needed |
|
Moving this to a draft, it's quite untested. |
Julesssss
left a comment
Julesssss
left a comment
There was a problem hiding this comment.
Nice improvement. Would love to see a test demonstrating this, but NAB
neil-marcellini
dismissed stale reviews from Julesssss and youssef-lr
via
4477295
neil-marcellini
changed the title
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
Hey! I see that you made changes to our Form component. Make sure to update the docs in FORMS.md accordingly. Cheers! |
Co-authored-by: Cursor <cursoragent@cursor.com>
Codecov Report❌ Looks like you've decreased code coverage for some files. Please write tests to increase, or at least maintain, the existing level of code coverage. See our documentation here for how to interpret this table.
|
Co-authored-by: Cursor <cursoragent@cursor.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5768b015f0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
src/components/TextInput/BaseTextInput/implementation/index.tsx
Outdated
Show resolved
Hide resolved
Codex correctly noted that unconditionally dropping role would break callers like SearchAutocompleteInput that pass role=SEARCHBOX. Now only role="presentation" is filtered out; all other roles pass through to the actual <input> element. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
- Remove explicit tool name listing (AI can infer available tools) - Use environment-agnostic language instead of specific tool names - Keep "Playwright MCP tools" naming for consistency with Melvin - Condense wait strategy to a single concise instruction Co-authored-by: Cursor <cursoragent@cursor.com>
|
No reviewer checklist is needed, regressions will be caught in staging in the rare chance that there are any. |
|
🚧 @neil-marcellini has triggered a test Expensify/App build. You can view the workflow run here. |
|
🧪🧪 Use the links below to test this adhoc build on Android, iOS, and Web. Happy testing! 🧪🧪
|
|
✋ This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release. |
|
🚀 Deployed to staging by https://github.com/neil-marcellini in version: 9.3.22-0 🚀
|
|
🚀 Deployed to production by https://github.com/mountiny in version: 9.3.22-4 🚀
|
|
🚀 Deployed to production by https://github.com/mountiny in version: 9.3.22-4 🚀
|
6 participants
Explanation of Change
This PR addresses three separate issues that make it difficult for browser automation tools (AI agents) to interact with the app on web, while also improving real accessibility:
1. SVG click interception (
Icon,FloatingActionButton)SVG
<path>elements inside buttons/pressables become the click target instead of the parent pressable on web. AddedpointerEvents="none"to theIconcomponent's wrapper Views andImageSVGelements, and to theSvgelement inFloatingActionButton, so clicks pass through to the parent pressable. This is web-only behavior; native platforms handle SVG hit-testing differently and are unaffected.2. TextInput
roleprop leaking to<input>elements (BaseTextInput)Many TextInput usages pass
role={CONST.ROLE.PRESENTATION}, intended for the wrapper, but this leaked through...inputPropsonto the actual<input>element — stripping its nativetextboxsemantics and making it invisible to screen readers and automation tools. Fixed by destructuringroleinBaseTextInputso it no longer reachesinputProps. The wrapperPressableWithoutFeedbackalready hasrole="presentation"hardcoded. This fixes all 40+ callers at once and improves real screen reader accessibility.3. Missing accessibility labels on onboarding radio buttons (
RadioButtonWithLabel,BaseOnboardingAccounting)Added an
accessibilityLabelprop toRadioButtonWithLabelthat passes through to the innerRadioButton, falling back tolabelif not provided. InBaseOnboardingAccounting, passaccessibilityLabel={item.text}so each radio button is labeled (e.g., "QuickBooks Online"). Also removed a deadaccessibilityLabelfrom the outerPressableWithoutFeedbackwhich hadaccessible={false}(contradictory no-op), and added requiredsentryLabelprops.Fixed Issues
$ #82681
Tests
Offline tests
N/A — these changes only affect pointer-events and accessibility attributes, which don't involve network behavior.
QA Steps
No QA
PR Author Checklist
### Fixed Issuessection aboveTestssectionOffline stepssectionQA stepssectioncanBeMissingparam foruseOnyxtoggleReportand notonIconClick)src/languages/*files and using the translation methodSTYLE.md) were followedAvatar, I verified the components usingAvatarare working as expected)StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))npm run compress-svg)Avataris modified, I verified thatAvataris working as expected in all cases)Designlabel and/or tagged@Expensify/designso the design team can review the changes.ScrollViewcomponent to make it scrollable when more elements are added to the page.mainbranch was merged into this PR after a review, I tested again and verified the outcome was still expected according to theTeststeps.Screenshots/Videos
On main: AI agent gets stuck trying to click the accounting radio button.
2026-02-12_08-20-17.mp4
On this branch: The test completes fully automated in about 4 minutes (video at 2x speed).
automation-2x.mp4
Only relevant on web. Other platforms will be tested for regressions on staging.
Android: Native
N/A — web-only changes (pointerEvents, role, accessibilityLabel)
Android: mWeb Chrome
iOS: Native
N/A — web-only changes (pointerEvents, role, accessibilityLabel)
iOS: mWeb Safari
MacOS: Chrome / Safari
See videos above