fix: optimistically build next step after resolving violation by dominictb · Pull Request #47543 · Expensify/App

dominictb · 2024-08-16T05:42:35Z

Details

Build fix the issue(s) next step optimistically when the transaction has violations.

Fixed Issues

$ #46914
PROPOSAL: #46914 (comment)

Tests

Create a "Control" workspace
Set the workspace currency to USD
Enable expense violation on OD (Settings > Workspaces > Choose the workspace > Expenses > Enable "Expense Violations")
On New Dot, Go to workspace settings > Categories > Settings
Enable "Members must categorize all expenses"
Submit an expense without a category and with an amount greater than the "Receipt Required Amount"
You will get a "Waiting for to fix the issue(s)" report next step
Go offline
Add a category to the expense
Change the expense amount to a value less than the "Receipt Required Amount"
Verify the next step message should update from "Waiting for to fix the issue(s)" to the corresponding next step ("Waiting for to submit expense(s)" for example).
Go online
Verify the next step does not revert to "Waiting for to fix the issue(s)"

Verify that no errors appear in the JS console

Offline tests

See Tests

QA Steps

See Tests

Verify that no errors appear in the JS console

PR Author Checklist

Screenshots/Videos

Android: Native

android.webm

Android: mWeb Chrome

compressed_aweb.webm.mp4

iOS: Native

Screen.Recording.2025-02-10.at.03.49.24-compressed.mov

iOS: mWeb Safari

Screen.Recording.2025-02-10.at.03.53.10.mov

MacOS: Chrome / Safari

Screen.Recording.2025-02-10.at.03.27.54-compressed.mov

MacOS: Desktop

Screen.Recording.2025-02-10.at.03.33.42-compressed.mov

melvin-bot · 2024-08-16T09:58:21Z

@rayane-djouah Please copy/paste the Reviewer Checklist from here into a new comment on this PR and complete it. If you have the K2 extension, you can simply click: [this button]

dominictb · 2024-08-16T09:59:06Z

Note: After reload, the BE still returns Waitng for XXX to fix the issues, even though there's no violations indicated. So it seems like that's BE issue.

rayane-d · 2024-08-19T10:55:01Z

Note: After reload, the BE still returns Waitng for XXX to fix the issues, even though there's no violations indicated. So it seems like that's BE issue.

cc @dangrous ^^

I believe we can move forward with fixing the optimistic next step on frontend and then @dangrous will handle them on backend side as well.

rayane-d · 2024-08-21T20:10:45Z

@dominictb - Could you please update the QA steps including how to get the optimistic / server side violation? We need to make them clear for QA team.

dominictb · 2024-08-22T09:45:50Z

@rayane-djouah updated!

rayane-d · 2024-08-23T09:31:25Z

Nice! for optimistic violations like "missing category" violation. We're now clearing the violation optimistically. But the backend still returns the Waiting for to fix the issue(s) next step after disabling offline mode.

Screen.Recording.2024-08-23.at.12.36.28.AM.mov

rayane-d · 2024-08-23T09:35:55Z

For server-side violations, I think we can't clear them optimistically because they need to be recalculated on backend.

Screen.Recording.2024-08-23.at.12.34.25.AM.mov

But the backend is returning the Waiting for to fix the issue(s) next step after resolving the violation online.

Screen.Recording.2024-08-23.at.12.33.24.AM.mov

rayane-d · 2024-08-23T09:47:35Z

@dominictb - I think the QA steps to test clearing the optimistic violations can be:

Enable expense violation on OD
On New Dot, Go to workspace settings > category > settings
Enable "Members must categorize all expenses"
Submit an expense without a category
You will get a "Waiting for to fix the issue(s)" report next step
Go offline
Add a category to the expense
Verify that the "Waiting for to fix the issue(s)" report next step is changed to "Waiting for to add expense(s)"

wdyt?

rayane-d · 2024-08-23T09:50:36Z

@dominictb - Bug: If we create a report with an optimistic violation, it displays "Waiting for to add expense(s)" instead of "Waiting for to fix the issue(s)" optimistic next step.

Screen.Recording.2024-08-23.at.10.46.22.AM.mov

rayane-d · 2024-08-23T10:01:39Z

@dominictb - Can we add automated tests in tests/unit/NextStepUtilsTest.ts for the new "Waiting for to add expense(s)" optimistic next step?

rayane-d · 2024-08-23T10:05:50Z

Backend bug not clearing "Waiting for to fix the issue(s)" next step after fixing server-side violations:

Screen.Recording.2024-08-23.at.11.02.57.AM.mov

cc @dangrous

dangrous · 2024-08-23T19:08:07Z

Okay so this is a bit of a nightmare from the backend side, since we build the report next step dynamically when we call OpenReport. We can send an onyx update with a new next step to the front end, BUT in order for it to work we'd need to check - in EVERY command that changes a request - if there was a violation before, and, if so, check if there isn't one now, and then build the onyx update for the next step.

I'm not actually sure what the best option for this is. Do you have any thoughts @vit?

dominictb · 2024-08-26T08:06:07Z

Will check this today.

dominictb · 2024-08-27T12:31:00Z

@rayane-djouah added the tests and fixed this: #47543 (comment)

dangrous · 2024-08-27T14:43:06Z

Okay so this is a bit of a nightmare from the backend side, since we build the report next step dynamically when we call OpenReport. We can send an onyx update with a new next step to the front end, BUT in order for it to work we'd need to check - in EVERY command that changes a request - if there was a violation before, and, if so, check if there isn't one now, and then build the onyx update for the next step.

I'm not actually sure what the best option for this is. Do you have any thoughts @vit?

Oops I mentioned the wrong person whoops! Let's try @mountiny

mountiny · 2024-08-27T22:23:23Z

@dangrous fun, do we optimistically clear the violations? We could update the next step when there are no violations left.

Otherwise, I agree that we might have to make a change to a bunch of the API commands. We can add an optional parameter that would be true only if the local report's next steps say to fix issues. In that case, the API command would also re-compute and return the report's next steps.

dangrous · 2024-08-28T16:46:08Z

So yeah, we're clearing the violations optimistically here. @dominictb do you want to take a shot at seeing if we can call OpenReport here, or some other way to re-pull the next steps from the backend?

That seems less intrusive than the backend solution, but I can work on that over time if we feel like it's best.

dominictb · 2024-08-29T06:51:25Z

I will take a look at this later and let you all know.

dangrous · 2024-08-30T15:47:03Z

thanks! happy to implement something on the backend too if we feel like that's the best solution, whether that's the checking every update transaction api command (hopefully not haha) or another option!

rayane-d · 2024-08-30T16:19:49Z

We can add an optional parameter that would be true only if the local report's next steps say to fix issues. In that case, the API command would also re-compute and return the report's next steps.

I like this approach, but I think it will not work with server-side violations #47543 (comment)

The solution of checking every update transaction api command will work well for both optimistic and server-side violations.

mountiny · 2024-09-02T11:28:18Z

I am mildly afraid the right solution is to update all the commands to recompute the next steps, but also this should wait for the auth migration @dangrous what do you think? Anything else is a hack/ workaround

dominictb · 2024-09-04T10:36:22Z

@dominictb do you want to take a shot at seeing if we can call OpenReport here, or some other way to re-pull the next steps from the backend?

Sorry @dangrous but the OpenReport API is the problem here I think (I mentioned #47543 (comment) here). It seems like the optimal solution is to fix for every BE endpoint.

One naive solution is for the BE to fix in the OpenReport API, and in the FE, if we can check if the transactionViolation in Onyx has changed (from null to non-empty array and vice-versa), and if yes, trigger OpenReport or a separate API to re-compute the next step.

mountiny · 2024-09-04T15:59:49Z

if yes, trigger OpenReport or a separate API to re-compute the next step.

I am afraid that would be against our 1:1:1 principle

…ext-step

…ation if they had receipt required violation

dominictb · 2025-02-09T20:47:58Z

@rayane-d Ready for review again. I also fixed 2 related issues:

6106e77: Trying to fix any violation would also clear the receiptRequired violation. The root cause is that transaction.modifiedAmount is 0 by default so the amount is always 0 causing shouldShowReceiptRequiredViolation to be false and consequently receiptRequired to be removed:

App/src/libs/Violations/ViolationsUtils.ts

Line 233 in 631006e

const amount = updatedTransaction.modifiedAmount ?? updatedTransaction.amount;

App/src/libs/Violations/ViolationsUtils.ts

Lines 234 to 237 in 631006e

    
           const shouldShowReceiptRequiredViolation = 
        
               !isInvoiceTransaction && 
        
               policy.maxExpenseAmountNoReceipt && 
        
               Math.abs(amount) > policy.maxExpenseAmountNoReceipt &&

App/src/libs/Violations/ViolationsUtils.ts

Lines 263 to 265 in 631006e

    
           if (hasReceiptRequiredViolation && !shouldShowReceiptRequiredViolation) { 
        
               newTransactionViolations = reject(newTransactionViolations, {name: CONST.VIOLATIONS.RECEIPT_REQUIRED}); 
        
           }

Screen.Recording.2025-02-10.at.02.55.17-compressed.mov

3678f27: The current user is not shown as you in next step message.

rayane-d · 2025-02-10T17:45:01Z

@dominictb Could you please fill out the Explanation of Change section in the PR?

dominictb · 2025-02-11T16:21:49Z

@rayane-d Updated.

dangrous · 2025-02-13T19:20:45Z

let me know when this is ready for a look @rayane-d - thank you!

dominictb · 2025-02-15T14:26:20Z

Bump @rayane-d

rayane-d · 2025-02-16T21:58:44Z

Reviewer Checklist

I have verified the author checklist is complete (all boxes are checked off).
I verified the correct issue is linked in the ### Fixed Issues section above
I verified testing steps are clear and they cover the changes made in this PR
- I verified the steps for local testing are in the Tests section
- I verified the steps for Staging and/or Production testing are in the QA steps section
- I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
- I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
I checked that screenshots or videos are included for tests on all platforms
I included screenshots or videos for tests on all platforms
I verified tests pass on all platforms & I tested again on:
- Android: Native
- Android: mWeb Chrome
- iOS: Native
- iOS: mWeb Safari
- MacOS: Chrome / Safari
- MacOS: Desktop
If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
I verified proper code patterns were followed (see Reviewing the code)
- I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
- I verified that comments were added to code that is not self explanatory
- I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
- I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
- I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
- I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
- I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
- I verified the JSDocs style guidelines (in STYLE.md) were followed
If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
I verified that this PR follows the guidelines as stated in the Review Guidelines
I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
I verified any variables that can be defined as constants (ie. in CONST.ts or at the top of the file that uses the constant) are defined as such
If a new component is created I verified that:
- A similar component doesn't exist in the codebase
- All props are defined accurately and each prop has a /** comment above it */
- The file is named correctly
- The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
- The only data being stored in the state is data necessary for rendering and nothing else
- For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
- Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
- All JSX used for rendering exists in the render method
- The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions
If any new file was added I verified that:
- The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory
If a new CSS style is added I verified that:
- A similar style doesn't already exist
- The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG)
If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:
- I verified that all the inputs inside a form are aligned with each other.
- I added Design label and/or tagged @Expensify/design so the design team can review the changes.
If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
For any bug fix or new feature in this PR, I verified that sufficient unit tests are included to prevent regressions in this flow.
If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native

Screen.Recording.2025-02-23.at.5.14.42.PM.mov

Android: mWeb Chrome

Screen.Recording.2025-02-23.at.5.06.05.PM.mov

iOS: Native

Simulator.Screen.Recording.-.iPhone.15.Pro.Max.-.2025-02-23.at.17.29.30.mp4

iOS: mWeb Safari

Simulator.Screen.Recording.-.iPhone.15.Pro.Max.-.2025-02-23.at.17.12.42.mp4

MacOS: Chrome / Safari

Screen.Recording.2025-02-23.at.4.37.22.PM.mov

MacOS: Desktop

Screen.Recording.2025-02-23.at.5.30.10.PM.mov

rayane-d · 2025-02-23T16:55:03Z

@dominictb let's update the QA steps (and Tests steps) to:

1. Create a "Control" workspace
2. Set the workspace currency to USD
3. Enable expense violation on OD (Settings > Workspaces > Choose the workspace > Expenses > Enable "Expense Violations")
4. On New Dot, Go to workspace settings > Categories > Settings
5. Enable "Members must categorize all expenses"
6. Submit an expense without a category and with an amount greater than the "Receipt Required Amount"
7. You will get a "Waiting for to fix the issue(s)" report next step
8. Go offline
9. Add a category to the expense
10. Change the expense amount to a value less than the "Receipt Required Amount"
11. Verify the next step message should update from "Waiting for to fix the issue(s)" to the corresponding next step ("Waiting for to submit expense(s)" for example).
12. Go online
13. Verify the next step does not revert to "Waiting for to fix the issue(s)"

src/libs/NextStepUtils.ts

rayane-d

LGTM and tests well 👍

rayane-d · 2025-02-23T17:11:52Z

@dominictb please address #47543 (comment) and #47543 (comment) when you get a chance

@dangrous this is ready for your review

dangrous

Let's update that jsdoc but otherwise looks good!

src/libs/NextStepUtils.ts

dangrous · 2025-02-26T15:58:56Z

bump for adding the param to the doc, then we're good to go!

…ext-step

OSBotify · 2025-02-27T16:38:00Z

✋ This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

github-actions · 2025-02-27T21:07:11Z

🚀 Deployed to staging by https://github.com/dangrous in version: 9.1.7-1 🚀

platform	result
🤖 android 🤖	success ✅
🖥 desktop 🖥	success ✅
🍎 iOS 🍎	success ✅
🕸 web 🕸	success ✅
🤖🔄 android HybridApp 🤖🔄	success ✅
🍎🔄 iOS HybridApp 🍎🔄	success ✅

github-actions · 2025-03-03T16:43:04Z

🚀 Deployed to production by https://github.com/puneetlath in version: 9.1.7-2 🚀

platform	result
🤖 android 🤖	true ❌
🖥 desktop 🖥	success ✅
🍎 iOS 🍎	success ✅
🕸 web 🕸	success ✅
🤖🔄 android HybridApp 🤖🔄	failure ❌
🍎🔄 iOS HybridApp 🍎🔄	failure ❌

dominictb added 2 commits August 16, 2024 12:40

fix: optimistically build next step after resolving violation

e50bb97

fix: update next step to reflect fixing violations

941ebc8

dominictb marked this pull request as ready for review August 16, 2024 09:58

dominictb requested a review from a team as a code owner August 16, 2024 09:58

melvin-bot bot requested review from rayane-d and removed request for a team August 16, 2024 09:58

fix: add unit tests for violation fix next step

0087920

dominictb added 5 commits February 10, 2025 02:33

Merge branch 'main' of github.com:dominictb/epsf-app into fix/46914-n…

5dc6087

…ext-step

remove redundant changes

cdf030e

fix: optimistic transaction violations are cleared when fixing a viol…

6106e77

…ation if they had receipt required violation

fix: does not recognize current user in next step

3678f27

fix lint

e41c2e1

dominictb changed the title ~~[HOLD] fix: optimistically build next step after resolving violation~~ fix: optimistically build next step after resolving violation Feb 9, 2025

rayane-d reviewed Feb 23, 2025

View reviewed changes

src/libs/NextStepUtils.ts Show resolved Hide resolved

rayane-d approved these changes Feb 23, 2025

View reviewed changes

melvin-bot bot requested a review from dangrous February 23, 2025 16:59

dangrous requested changes Feb 24, 2025

View reviewed changes

src/libs/NextStepUtils.ts Show resolved Hide resolved

dominictb added 2 commits February 27, 2025 02:58

Merge branch 'main' of github.com:dominictb/epsf-app into fix/46914-n…

d56f445

…ext-step

add jsdoc comment for params

2c96c61

rayane-d approved these changes Feb 26, 2025

View reviewed changes

melvin-bot bot requested a review from dangrous February 26, 2025 20:05

dangrous approved these changes Feb 27, 2025

View reviewed changes

dangrous merged commit a75fbdb into Expensify:main Feb 27, 2025
18 checks passed

github-actions bot mentioned this pull request Feb 27, 2025

Deploy Checklist: New Expensify 2025-02-27 #57563

Closed

40 tasks

FitseTLT mentioned this pull request May 8, 2025

[Due for payment 2025-05-30] [Due for payment 2025-05-28] [$250] Chat- Next step changes when changing category #59626

Closed

8 tasks

Conversation

dominictb commented Aug 16, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Details

Fixed Issues

Tests

Offline tests

QA Steps

PR Author Checklist

Screenshots/Videos

Uh oh!

melvin-bot bot commented Aug 16, 2024

Uh oh!

dominictb commented Aug 16, 2024

Uh oh!

rayane-d commented Aug 19, 2024

Uh oh!

rayane-d commented Aug 21, 2024

Uh oh!

dominictb commented Aug 22, 2024

Uh oh!

rayane-d commented Aug 23, 2024

Uh oh!

rayane-d commented Aug 23, 2024

Uh oh!

rayane-d commented Aug 23, 2024

Uh oh!

rayane-d commented Aug 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rayane-d commented Aug 23, 2024

Uh oh!

rayane-d commented Aug 23, 2024

Uh oh!

dangrous commented Aug 23, 2024

Uh oh!

dominictb commented Aug 26, 2024

Uh oh!

dominictb commented Aug 27, 2024

Uh oh!

dangrous commented Aug 27, 2024

Uh oh!

mountiny commented Aug 27, 2024

Uh oh!

dangrous commented Aug 28, 2024

Uh oh!

dominictb commented Aug 29, 2024

Uh oh!

dangrous commented Aug 30, 2024

Uh oh!

rayane-d commented Aug 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mountiny commented Sep 2, 2024

Uh oh!

dominictb commented Sep 4, 2024

Uh oh!

mountiny commented Sep 4, 2024

Uh oh!

dominictb commented Feb 9, 2025

Uh oh!

rayane-d commented Feb 10, 2025

Uh oh!

dominictb commented Feb 11, 2025

Uh oh!

dangrous commented Feb 13, 2025

Uh oh!

dominictb commented Feb 15, 2025

Uh oh!

rayane-d commented Feb 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviewer Checklist

Screenshots/Videos

Uh oh!

rayane-d commented Feb 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

rayane-d left a comment

Choose a reason for hiding this comment

dominictb commented Aug 16, 2024 •

edited

Loading

rayane-d commented Aug 23, 2024 •

edited

Loading

rayane-d commented Aug 30, 2024 •

edited

Loading

rayane-d commented Feb 16, 2025 •

edited

Loading

rayane-d commented Feb 23, 2025 •

edited

Loading