Setup a Macrobenchmark for RNTester

[janicduplessis]

Summary:

Benchmark to test changes from #49449

Might be nice to have some version of this in the repo.

Changelog:

[INTERNAL] [ADDED] - Setup a Macrobenchmark for RNTester

Test Plan:

Methodology

Picked various JS file from websites (facebook, instagram) to artificially grow RN tester bundle somewhat realistically. The files are required lazily from a button press callback to simulate the code being included, but not executed, as it would be in a large app that uses lazy requires for the different screens.

I've also made the RN tester screens lazy so all their code is not loaded initially. This is more representative of real apps. Note this is implemented in a hacky way just for the purpose of this test. It would actually be nice to implement this properly.

The tests were made using low end device Samsung Galaxy A03s.

Compression ON with 10.5 mb bundle

Peak allocated memory

60.9 mb

ReactInstance.loadJSBundler

148.64 ms

Benchmark

timeToFullDisplayMs min 1,825.0, median 1,911.1, max 1,994.8
timeToInitialDisplayMs min 834.9, median 860.9, max 903.9

APK

Size: 22.9 mb
Download size: 14.5 mb

Compression OFF with 10.5 mb bundle

Peak allocated memory

51.5 mb

ReactInstance.loadJSBundler

946 us

Benchmark

timeToFullDisplayMs min 1,752.8, median 1,827.2, max 1,977.5
timeToInitialDisplayMs min 837.7, median 881.3, max 937.2

APK

Size: 28 mb
Download size: 14.5 mb

[cortinico]

Thanks for looking into those @janicduplessis 🙏

[janicduplessis]

@cortinico Do you think we should try to merge this? I was thinking it could be useful to manually test perf improvements. Would be nice to run on ci too, but might be hard with the current setup since it seems to only make sense to run on physical device.

If so I can remove the changes to rn-tester and fake bundle size increase.

[cortinico]

In general, I'd like to have such a thing merged. The problem is that this might slow down all the rn-tester gradle builds (because it adds a new build variant that we use like once in a while). So I'd rather keep it as a branch or PR around for when we need it

[janicduplessis]

@cortinico Let me check the impact on gradle build time, if its negligible I think we can merge it.

[cortinico]

@cortinico Let me check the impact on gradle build time, if its negligible I think we can merge it.

I think that as we enabled Gradle Config Cache today, this is probably going to be small (and ok to maintain)

[janicduplessis]

@cortinico Should we try to avoid having the benchmark variants built on CI or that's fine? I think it gets built because of this line here that includes all projects that use the gradle-plugin https://github.com/janicduplessis/react-native/blob/224fe93fecc17d98170888060b2b91872e1e2bf8/build.gradle.kts#L90.

[cortinico]

@cortinico Should we try to avoid having the benchmark variants built on CI or that's fine? I think it gets built because of this line here that includes all projects that use the gradle-plugin janicduplessis/react-native@224fe93/build.gradle.kts#L90.

We'll have to do something like this: https://stackoverflow.com/a/34337260
and filter everything benchmark related till we actually need it (so we never build it)

[janicduplessis]

Turns out we can just remove the benchmark variant and it still works fine using the hermesRelease variant instead.

[janicduplessis]

From looking at ci logs it looks like the benchmark project does get built, but takes around 2 seconds so should be fine.

[cortinico]

Thanks for cleaning this up @janicduplessis

[janicduplessis]

I addressed the comments, once the PR to bump appcompat version lands I will rebase this and should be good to go!

[janicduplessis]

Rebased

[facebook-github-bot]

@cortinico has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[cortinico]

@janicduplessis can we make sure the JS tests are green also?

[janicduplessis]

Fixed!

[facebook-github-bot]

@cortinico has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[janicduplessis]

@cortinico ReportFullyDrawnView actually doesn't need to wrap the app, I moved it and hopefully that solves the issue.

[facebook-github-bot]

@cortinico has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[facebook-github-bot]

@cortinico merged this pull request in b614c96.

[react-native-bot]

This pull request was successfully merged by @janicduplessis in b614c96

^{When will my fix make it into a release? | How to file a pick request?}