Skip to content

Conversation

@kirillzyusko
Copy link
Contributor

@kirillzyusko kirillzyusko commented Jan 2, 2025

To Do

  • [$250] iOS - Chat - Big gap in the background when context menu is opened #54713 (fixed in 5b05167)
  • iOS - Edit composer is hidden behind keyboard when editing message #54712 <-- I didn't fix it yet, but useful observations:
    • it's reproducible in release builds only and it's better to use a physical device (I assemble release but with a connection to the dev server)
    • internally root cause is the problem, that we update paddingTop in useAnimatedStyles. It triggers onLayout events when we perform animation. In some corner case sometimes onLayout on CellItem skips some last events (when bottom sheet gets hidden we expect, that it will return to its initial position and onLayout will produce the same event as when the view was just mounted. But instead we start scrolling when view is far away about ~80-100px till its expected position). The simplest solution is to postpone a call to scrollTo method, but in this case it produces significant delay).

Details

A successor of #42143

The first step of react-native-keyboard-controller migration plan

Important

This PR relies on react-native-keyboard-controller (useKeyboardHandler) hook and KeyboardAvoidingView component).

How it works 🤔

This PR is iOS only for now. Other platforms are not affected. Also, in this explanation, we are talking about iOS only since on Android, iOS web and Android Web keyboard handling works differently.

Note

We're talking about default platform behavior. If we start to use react-native-keyboard-controller on Android - it'll disable all default keyboard handling by OS and will delegate the keyboard handling to us - so we will have identical behavior across both iOS/Android platforms.

But it may introduce additional issues (because we need to enter edge-to-edge mode - that would be good to do on entire app level, but it may bring some issues with StatusBar paddings, so everything has to be verified very carefully). That's why I think it would be better to handle in separate PR.

When we long press on a message, we want to be able to see it. It's easy when we have the keyboard closed. But gets tricky when the keyboard is open.

But first of all let's focus on the case when the keyboard is closed. User long press on the message, we render the bottom sheet (action sheet) with some options. If the message is higher than the height of bottom sheet - the message is visible. When the message is the last one (first from the bottom), it can be fully or partially covered by the bottom sheet.

Press to see videos
Message above bottom sheet Message and bottom sheet overlap
300481023-9dc8be61-799d-470b-997f-ee5c5995fe19.mp4
300481277-0d0570cd-a85f-4cf5-8a91-6a3d5ad72c58.mp4

Also everything is tricky when we have keyboard open 😓

Press to see videos with keyboard
emoji picker instead of the keyboard (the messages should not jump) showing a message which is covered by the keyboard and the bottom sheet
300482884-cbe3b610-fa85-4071-a3dd-bbc1a9d56e05.mp4
300483006-2e5ba833-c0fa-4fad-b383-b3854e5fa112.mp4

To achieve this goal we had to:

  • use our own scroll component on iOS for FlatList of ReportListView (it uses reanimated to move the view by the offset)
  • use KeyboardAvoiding so we're in sync with keyboard-avoiding style updates
  • implement animated style handling using a state machine
  • create ActionSheetAwareScrollViewContext to pass the transition function, so we can transition from one state to another by action triggered from different parts of the application

Below you can find more use cases that involves keyboard and other UI elements interactions:

Interactions with other UI elements
Add attachment Call popover
300483536-04991f52-a41b-4d3e-abaf-12712b94f1aa.mp4
300483723-91b076a9-4c7a-4fbb-bc50-1e8194763e9d.mp4

Note

Call popover, bottom sheet and emoji picker are implemented as Modal windows and on iOS keyboard instantly gets hidden when modal window is shown. But "Attachment" popup is implemented not as modal, so for this type of transition we are using progress-based animations. When we press on plus -> popup is already shown under keyboard and we dismiss the keyboard. Overall the mechanism is similar to what we've used in the past, except the moment where we substitute translationY value - in case when keyboard disappear/appears we interpolate it by progress variable to achieve smooth transitions.

Technical details ⚒️

So in order to calculate the offset for the message, we:

  1. Measure the message item in the window using ref.current.measureInWindow, so we can have height and y coordinate of the image
  2. Measure the height of the bottom sheet using onLayout callback
  3. Get the window height from Dimensions
  4. Get safeArea.top since y coordinate doesn't get into account safe area
  5. Calculate offset as elementOffset = (y + safeArea.top + itemHeight) - (windowHeight - popoverHeight)
    In order to animate item when they keyboard was open, we need to also know its height which is provided by useAnimatedKeyboard hook from react-native-reanimated, but we should always subtract safeArea.bottom from it since it's not part of the calculation for the offset, but only keyboardHeight.

Overall, the keyboard is the tricky part. On iOS, the keyboard doesn't resize the viewport. So for the content to not be covered by the keyboard, KeyboardAvoidingView is used. In React Native, KeyboardAvoidingView is implemented as a view that changes its height or resizes the children's view by the keyboard height. It's done using RN's Layout Animations. This results in the animation for the keyboard being applied using the native event emitter callback which uses the bridge. So open/close events of the keyboard will always be delayed that resulting in the delayed keyboard avoiding animation and, which is more important, scheduling layout animations.

In order to be consistent 100% of the time with animation and timings, we had to:

  • use KeyboardAvoidingView from react-native-keyboard-controller for iOS. It reacts on keyboard height/state change on UI thread in the same frame, so we can schedule an update for styles in the next frame
  • use just useDerivedValue for all the logic, since introducing useAnimatedReaction or having multiple shared values will delay updates for 1 frame, but we want to be 100% in sync with keyboard updates
  • use paddingTop: translateY (because ScrollView is inverted).

Another important aspect of the implementation is the usage of the state machine for the animation. State machine allows us to specify the chain of events and how to handle them, and more importantly what actions to ignore, for example:

  1. Initial state is idle. it can react to KEYBOARD_OPEN action
  2. we open emoji picker. it sends EMOJI_PICKER_OPEN action
  3. there is no handling for this action in idle state so we do nothing
  4. we close emoji picker and it sends EMOJI_PICKER_CLOSE action which again does nothing
  5. we open keyboard. it sends KEYBOARD_OPEN action. idle can react to this action by transitioning into keyboardOpen state
  6. our state is keyboardOpen. it can react to KEYBOARD_CLOSE, EMOJI_PICKER_OPEN actions
  7. we open emoji picker again. it sends EMOJI_PICKER_OPEN action which transitions our state into emojiPickerOpen state. now we react only to EMOJI_PICKER_CLOSE action
  8. before rendering the emoji picker, the app hides the keyboard. it sends KEYBOARD_CLOSE action. but we ignore it since our emojiPickerOpen state can only handle EMOJI_PICKER_CLOSE action. so we write the logic for handling 9. hiding the keyboard but maintaining the offset based on the keyboard state shared value
  9. we close the picker and send EMOJI_PICKER_CLOSE action which transitions us back into keyboardOpen state.

Fixed Issues

$ #10632
PROPOSAL: N/A

Tests

These steps were performed on all the platforms. The new behavior is iOS only for now.

  1. Open chat
  2. Long press on the first message - message should be pushed up to be displayed above the bottom sheet
  3. Long press on the top message - if the message is not covered by the bottom sheet, it should stay in place (without empty space between bottom-sheet and message)
  4. Long press on any attachment (document, message) should result in the same behavior

Before each next we make sure the keyboard is open:

  1. Long press on the message should either animate the message above the bottom sheet or keep it in place
  2. Messages should not "jump" because of keyboard hide/open when showing bottom sheet, emoji picker
  3. Press on emoji picker in the composer, messages should stay in place even though the keyboard was closer - emoji picker is rendered instead of keyboard
  4. Long press on the mesage, click on add reactions - close emoji reactions
  • Verify that no errors appear in the JS console

Offline tests

Should not affect the offline state.

QA Steps

Same as Tests

  • Verify that no errors appear in the JS console

PR Author Checklist

  • I linked the correct issue in the ### Fixed Issues section above
  • I wrote clear testing steps that cover the changes made in this PR
    • I added steps for local testing in the Tests section
    • I added steps for the expected offline behavior in the Offline steps section
    • I added steps for Staging and/or Production testing in the QA steps section
    • I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
    • I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
    • I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).
  • I included screenshots or videos for tests on all platforms
  • I ran the tests on all platforms & verified they passed on:
    • Android: Native
    • Android: mWeb Chrome
    • iOS: Native
    • iOS: mWeb Safari
    • MacOS: Chrome / Safari
    • MacOS: Desktop
  • I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
  • I followed proper code patterns (see Reviewing the code)
    • I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
    • I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
    • I verified that comments were added to code that is not self explanatory
    • I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
    • I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
      • If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:
    • I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
    • I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
    • I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
    • I verified the JSDocs style guidelines (in STYLE.md) were followed
  • If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
  • I followed the guidelines as stated in the Review Guidelines
  • I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
  • I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
  • I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
  • I verified that if a function's arguments changed that all usages have also been updated correctly
  • If any new file was added I verified that:
    • The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory
  • If a new CSS style is added I verified that:
    • A similar style doesn't already exist
    • The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))
  • If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
  • If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
  • If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
  • If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
  • If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:
    • I verified that all the inputs inside a form are aligned with each other.
    • I added Design label and/or tagged @Expensify/design so the design team can review the changes.
  • If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
  • If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native
telegram-cloud-document-2-5460627828225625066.mp4
Android: mWeb Chrome
telegram-cloud-document-2-5460627828225625138.mp4
iOS: Native
1104ff6c-6eba-4295-a905-6caaaf3edf28.mp4
iOS: mWeb Safari
6eff007f-3c61-4f19-a60e-43f46d2f0e61.mp4
MacOS: Chrome / Safari
e5e28bd8-529f-4da3-8ae7-bff03bd14499.mp4
MacOS: Desktop
2c813bc9-6988-4fbc-8f36-3282c9cecd82.mp4

@luacmartins
Copy link
Contributor

@kirillzyusko is the next step here to fix the blockers before we can put this in review again?

@kirillzyusko
Copy link
Contributor Author

is the next step here to fix the blockers before we can put this in review again?

Yes, it's the plan 🙂

@luacmartins
Copy link
Contributor

@kirillzyusko is there a rough ETA to get this PR ready for review again?

@kirillzyusko
Copy link
Contributor Author

@luacmartins there is one deploy blocker to be resolved. I was planning to discuss how to tackle that one tomorrow with @hannojg 👀

I'll get back to you tomorrow with the answer, but I hope this/next week is the target 😊

@luacmartins
Copy link
Contributor

@kirillzyusko did you manage to discuss the fix with Hanno? Looking forward to getting this one in review again

@hannojg
Copy link
Contributor

hannojg commented Jan 14, 2025

Sorry for providing no update earlier, Kiryl was blocked with some other tasks and I believe @perunt wanted to fix the remaining issues?

@luacmartins
Copy link
Contributor

luacmartins commented Jan 14, 2025

Np, all good. Let me know if I should assign @perunt to the issue as well

@perunt
Copy link
Contributor

perunt commented Jan 15, 2025

@luacmartins yes, please. You can assign it to me

Also, after some testing, I found two bugs that I am going to address as well:

  1. Jumpy behaviour when editing message
RPReplay_Final1736867201.MP4
  1. The context menu is automatically skipped
RPReplay_Final1736940087.MP4

@perunt
Copy link
Contributor

perunt commented Jan 21, 2025

@rojiphil, may I ask you to review this in the next round?

@perunt
Copy link
Contributor

perunt commented Jan 21, 2025

@luacmartins, the new ESLint rules introduce a lot of changes. I'm also not sure if it's a good idea to address them in this PR. Let me explain why:
These changes are not related to the current PR. Moreover, the current PR was reverted and is critical to merge sooner as it’s a release blocker.
If I make all the changes suggested by ESLint, it would result in a lot of random function names like close, open, or check, which were originally part of general imports such as import * as Module. Without context, these names don't make their purpose clear at first glance.
I think it would be better to handle this in a separate PR, where we can implement these changes more thoughtfully. Hopefully, we can find a cleaner solution, as the current approach introduces unnecessary clutter to the code

@luacmartins
Copy link
Contributor

@perunt I agree that we can address those in a follow up, so we don't distract from the current changes here, avoid conflicts and make things easier to debug later on if reverts are needed. Let's go ahead without addressing those for now.

@perunt
Copy link
Contributor

perunt commented May 20, 2025

@perunt I am not sure if this is intentional or not. But on closing the editing composer, nothing (no composer) gets highlighted.

I see this on the main as well. I suggest raising an issue/feature request separately
Apart from this, does this work well for you?

@shubham1206agra
Copy link
Contributor

@perunt Yes

@melvin-bot melvin-bot bot requested a review from roryabraham May 20, 2025 17:34
@luacmartins
Copy link
Contributor

@roryabraham did you wanna review again or are we good to merge?

import useWindowDimensions from '@hooks/useWindowDimensions';
import {Actions, ActionSheetAwareScrollViewContext, States} from './ActionSheetAwareScrollViewContext';

const KeyboardState = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NAB:

diff --git a/src/components/ActionSheetAwareScrollView/ActionSheetKeyboardSpace.tsx b/src/components/ActionSheetAwareScrollView/ActionSheetKeyboardSpace.tsx
index cf96a0823b0..c773c74c30b 100644
--- a/src/components/ActionSheetAwareScrollView/ActionSheetKeyboardSpace.tsx
+++ b/src/components/ActionSheetAwareScrollView/ActionSheetKeyboardSpace.tsx
@@ -3,6 +3,7 @@ import type {ViewProps} from 'react-native';
 import {useKeyboardHandler} from 'react-native-keyboard-controller';
 import Reanimated, {useAnimatedReaction, useAnimatedStyle, useDerivedValue, useSharedValue, withSequence, withSpring, withTiming} from 'react-native-reanimated';
 import type {SharedValue} from 'react-native-reanimated';
+import type {ValueOf} from 'type-fest';
 import useSafeAreaPaddings from '@hooks/useSafeAreaPaddings';
 import useThemeStyles from '@hooks/useThemeStyles';
 import useWindowDimensions from '@hooks/useWindowDimensions';
@@ -14,16 +15,16 @@ const KeyboardState = {
     OPEN: 2,
     CLOSING: 3,
     CLOSED: 4,
-};
+} as const;
 
 const SPRING_CONFIG = {
     mass: 3,
     stiffness: 1000,
     damping: 500,
-};
+} as const;
 
 const useAnimatedKeyboard = () => {
-    const state = useSharedValue(KeyboardState.UNKNOWN);
+    const state = useSharedValue<ValueOf<typeof KeyboardState>>(KeyboardState.UNKNOWN);
     const height = useSharedValue(0);
     const lastHeight = useSharedValue(0);
     const heightWhenOpened = useSharedValue(0);
@@ -74,7 +75,7 @@ function ActionSheetKeyboardSpace(props: ActionSheetKeyboardSpaceProps) {
     const {position} = props;
 
     // Similar to using `global` in worklet but it's just a local object
-    const syncLocalWorkletState = useSharedValue(KeyboardState.UNKNOWN);
+    const syncLocalWorkletState = useSharedValue<ValueOf<typeof KeyboardState>>(KeyboardState.UNKNOWN);
     const {windowHeight} = useWindowDimensions();
     const {currentActionSheetState, transitionActionSheetStateWorklet: transition, resetStateMachine} = useContext(ActionSheetAwareScrollViewContext);

const {windowHeight} = useWindowDimensions();
const {currentActionSheetState, transitionActionSheetStateWorklet: transition, resetStateMachine} = useContext(ActionSheetAwareScrollViewContext);

// Reset state machine when component unmounts
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NAB:

diff --git a/src/components/ActionSheetAwareScrollView/ActionSheetKeyboardSpace.tsx b/src/components/ActionSheetAwareScrollView/ActionSheetKeyboardSpace.tsx
index cf96a0823b0..76c9bb3182b 100644
--- a/src/components/ActionSheetAwareScrollView/ActionSheetKeyboardSpace.tsx
+++ b/src/components/ActionSheetAwareScrollView/ActionSheetKeyboardSpace.tsx
@@ -79,10 +79,7 @@ function ActionSheetKeyboardSpace(props: ActionSheetKeyboardSpaceProps) {
     const {currentActionSheetState, transitionActionSheetStateWorklet: transition, resetStateMachine} = useContext(ActionSheetAwareScrollViewContext);
 
     // Reset state machine when component unmounts
-    // eslint-disable-next-line arrow-body-style
-    useEffect(() => {
-        return () => resetStateMachine();
-    }, [resetStateMachine]);
+    useEffect(() => resetStateMachine, [resetStateMachine]);
 
     useAnimatedReaction(
         () => keyboard.state.get(),

@luacmartins
Copy link
Contributor

@perunt let's resolve conflicts here too

@roryabraham roryabraham merged commit a6af6fa into Expensify:main May 27, 2025
17 checks passed
@roryabraham
Copy link
Contributor

I think main changed such that there were no conflicts anymore. So I went ahead and merged this

@OSBotify
Copy link
Contributor

✋ This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

}

// save previous payload or merge the new payload with the previous payload
const nextPayload = typeof action.payload === 'undefined' ? state.current.payload : fastMerge(state.current.payload, action.payload);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We created an app error due to this. We should convert fastMerge to worklet before using it here.

Should I create a PR for this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, it seems like you already did 😄

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed here #62873

@github-actions
Copy link
Contributor

🚀 Deployed to staging by https://github.com/roryabraham in version: 9.1.53-0 🚀

platform result
🖥 desktop 🖥 success ✅
🕸 web 🕸 success ✅
🤖 android 🤖 failure ❌
🍎 iOS 🍎 success ✅

@github-actions
Copy link
Contributor

🚀 Deployed to staging by https://github.com/roryabraham in version: 9.1.53-0 🚀

platform result
🖥 desktop 🖥 success ✅
🕸 web 🕸 success ✅
🤖 android 🤖 cancelled 🔪
🍎 iOS 🍎 success ✅

@nlemma
Copy link

nlemma commented May 28, 2025

@kirillzyusko This PR is failing because of issue #62955

@github-actions
Copy link
Contributor

🚀 Deployed to production by https://github.com/Beamanator in version: 9.1.53-7 🚀

platform result
🖥 desktop 🖥 success ✅
🕸 web 🕸 success ✅
🤖 android 🤖 failure ❌
🍎 iOS 🍎 success ✅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.