Skip to content

Implement JSON_PATCH() in Onyx.merge() in react-native-onyx(native)#238

Merged
marcaaron merged 90 commits intoExpensify:mainfrom
chrispader:@chrispader/feat/use-json-patch-for-merge
Jul 18, 2023
Merged

Implement JSON_PATCH() in Onyx.merge() in react-native-onyx(native)#238
marcaaron merged 90 commits intoExpensify:mainfrom
chrispader:@chrispader/feat/use-json-patch-for-merge

Conversation

@chrispader
Copy link
Contributor

@chrispader chrispader commented Mar 2, 2023

Details

JSON_PATCH is an algorithm in SQLite we can use on iOS and Android to merge delta changes natively (in the storage provider) instead of manually "pre-merging" in the "Onyx layer .

The original goal of this PR was to utilise JSON_PATCH on native platforms, whilst still retaining the current merge logic on web.

(By "pre-merging" i mean using the current implementation of applyMerge which uses a mergeQueue to batch and merge delta changes, after which data will be written to storage with Storage.setItem instead of Storage.mergeItem)

Related Issues

Expensify/App#13972

Automated Tests

Linked PRs

Copy link
Contributor

@marcaaron marcaaron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just leaving a couple comments. I think it would be good to lay out this approach somewhere and get more feedback.

@tgolen or @kidroca might also be able to help validate the correct approach.

The existing code (merge queue stuff) is far from perfect and could likely be improved - but something about the notifyAfterWrite() direction is making me uneasy and I would appreciate some more eyes on this problem.

Copy link
Contributor

@kidroca kidroca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some notes


I have a question:

  • We need to perform the merge in order to update memory cache
  • Then why do we move the merge logic to storage providers

If we need to extract the merge logic we can extract it to a lib and import the storage provider there (this would import the Storage provider we're currently using)

chrispader and others added 2 commits March 9, 2023 09:45
Co-authored-by: Peter Velkov <kidroca@gmail.com>
@chrispader chrispader marked this pull request as ready for review March 15, 2023 13:54
@chrispader chrispader requested a review from a team as a code owner March 15, 2023 13:54
@melvin-bot melvin-bot bot requested review from MariaHCD and removed request for a team March 15, 2023 13:55
@marcaaron
Copy link
Contributor

I think maybe this one is on hold pending the AsyncStorage removal?

@chrispader chrispader changed the title Implement JSON_PATCH() in Onyx.merge() in react-native-onyx(native) [HOLD on AsyncStorage removal] Implement JSON_PATCH() in Onyx.merge() in react-native-onyx(native) Mar 27, 2023
@marcaaron
Copy link
Contributor

Still holding

@chrispader chrispader changed the title [HOLD on AsyncStorage removal] Implement JSON_PATCH() in Onyx.merge() in react-native-onyx(native) Implement JSON_PATCH() in Onyx.merge() in react-native-onyx(native) Apr 5, 2023
@chrispader
Copy link
Contributor Author

@marcaaron @kidroca change requests resolved 👍

@marcaaron
Copy link
Contributor

Can we fix the lint errors?

@chrispader
Copy link
Contributor Author

Can we fix the lint errors?

Done, though i'm not sure if this eslint rule should trigger here. Basically it gave errors because the withOnyx HOC calls in the tests don't have propTypes, although the component that it wraps has those defined...

@marcaaron
Copy link
Contributor

Basically it gave errors because the withOnyx HOC calls in the tests don't have propTypes, although the component that it wraps has those defined...

I think it's ok to disable it in this case.

@chrispader
Copy link
Contributor Author

chrispader commented Jul 18, 2023

Basically it gave errors because the withOnyx HOC calls in the tests don't have propTypes, although the component that it wraps has those defined...

I think it's ok to disable it in this case.

done 👍

Copy link
Contributor

@marcaaron marcaaron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we clean up the diff? There are a bunch of what look like new changes, but are actually recent changes to main.

@chrispader
Copy link
Contributor Author

@marcaaron just updated the diff and replied to your questions :)

Copy link
Contributor

@marcaaron marcaaron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@marcaaron marcaaron merged commit 6a142d0 into Expensify:main Jul 18, 2023
@marcaaron
Copy link
Contributor

@chrispader Let's raise a PR in App to bump the version of Onyx there and do any final testing? Thanks so much for your patience and continued effort on this one. And thank you @kidroca for the additional thoughts and review 🙇

@chrispader
Copy link
Contributor Author

Some issues that arose from this PR are fixed in this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants