Skip to content

Conversation

@sheetalkamat
Copy link
Member

@sheetalkamat sheetalkamat commented Jun 3, 2024

This makes --outFile behave similar to not specifying it.
Previously we use to store the bundle section information in the outFileBuildInfo which means we could not emit buildInfo without emitting the other output files and had to be always in sync. But now that we are not storing that information with no support for prepend, we can write buildInfo even if the other outputs like js or dts were not emitted. This helps with storing semantic diagnostics information and dts errors information in the buildInfo so we can re-use it just like multiFile scenario.
The changes are:

  • Cache semantic diagnostics in --outFile scenario
  • Ensuring all places that checked for outFile not present when emitting buildInfo explicitly to disable that
  • Ensuring emitter can emit only outFile's js, dts, map etc without emitting buildInfo so that buildInfo can be synchronized correctly with the builder state for changeFileSet, dts diagnostics etc (this is similar to multiFileScenario where we emit all affected files one by one and then emit build info. In --outFile we have to emit program without emitting buildInfo and then emit buildInfo in later stage)

@typescript-bot typescript-bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Jun 3, 2024
@sheetalkamat sheetalkamat force-pushed the outFileChanges branch 5 times, most recently from 0247216 to 79d728f Compare June 3, 2024 22:50
@sheetalkamat sheetalkamat marked this pull request as ready for review June 3, 2024 23:01
@sheetalkamat
Copy link
Member Author

@typescript-bot test this
@typescript-bot test top100
@typescript-bot test tsserver top100
@typescript-bot user test this
@typescript-bot user test tsserver
@typescript-bot run dt
@typescript-bot perf test this
@typescript-bot pack this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 3, 2024

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
test top100 ✅ Started
test tsserver top100 ✅ Started
user test this ✅ Started
user test tsserver ✅ Started
run dt ✅ Started ✅ Results
perf test this ✅ Started 👀 Results
pack this ✅ Started ✅ Results

@typescript-bot
Copy link
Collaborator

typescript-bot commented Jun 3, 2024

Hey @sheetalkamat, I've packed this into an installable tgz. You can install it for testing by referencing it in your package.json like so:

{
    "devDependencies": {
        "typescript": "https://typescript.visualstudio.com/cf7ac146-d525-443c-b23c-0d58337efebc/_apis/build/builds/162085/artifacts?artifactName=tgz&fileId=7B842EEB1F7BF781F93AA4CD2C86FA0CF1E415F91791071EBE52BDDF104D9E5F02&fileName=/typescript-5.6.0-insiders.20240603.tgz"
    }
}

and then running npm install.


There is also a playground for this build and an npm module you can use via "typescript": "npm:@typescript-deploys/pr-build@5.6.0-pr-58760-2".;

@sheetalkamat sheetalkamat merged commit 9edddc7 into main Jun 4, 2024
@sheetalkamat sheetalkamat deleted the outFileChanges branch June 4, 2024 00:06
@typescript-bot
Copy link
Collaborator

@sheetalkamat
The results of the perf run you requested are in!

Here they are:

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - node (v18.15.0, x64)
Errors 30 30 ~ ~ ~ p=1.000 n=6
Symbols 62,153 62,153 ~ ~ ~ p=1.000 n=6
Types 50,242 50,242 ~ ~ ~ p=1.000 n=6
Memory used 195,274k (± 0.74%) 193,551k (± 0.95%) ~ 192,316k 195,949k p=0.471 n=6
Parse Time 1.58s (± 0.89%) 1.57s (± 0.67%) ~ 1.56s 1.59s p=0.564 n=6
Bind Time 0.87s (± 1.13%) 0.86s (± 0.94%) ~ 0.85s 0.87s p=0.498 n=6
Check Time 11.39s (± 0.40%) 11.41s (± 0.45%) ~ 11.36s 11.48s p=0.520 n=6
Emit Time 3.27s (± 0.57%) 3.28s (± 1.71%) ~ 3.17s 3.32s p=0.126 n=6
Total Time 17.10s (± 0.18%) 17.13s (± 0.57%) ~ 16.98s 17.26s p=0.686 n=6
angular-1 - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 944,109 944,109 ~ ~ ~ p=1.000 n=6
Types 407,049 407,049 ~ ~ ~ p=1.000 n=6
Memory used 1,222,101k (± 0.01%) 1,222,156k (± 0.01%) ~ 1,222,040k 1,222,242k p=0.173 n=6
Parse Time 8.08s (± 0.28%) 8.09s (± 0.42%) ~ 8.06s 8.15s p=0.804 n=6
Bind Time 2.24s (± 0.54%) 2.23s (± 0.78%) ~ 2.22s 2.26s p=0.620 n=6
Check Time 36.50s (± 0.37%) 36.46s (± 0.28%) ~ 36.32s 36.58s p=0.574 n=6
Emit Time 18.12s (± 0.41%) 18.10s (± 0.46%) ~ 17.97s 18.18s p=0.936 n=6
Total Time 64.94s (± 0.29%) 64.89s (± 0.19%) ~ 64.73s 65.04s p=0.574 n=6
mui-docs - node (v18.15.0, x64)
Errors 5 5 ~ ~ ~ p=1.000 n=6
Symbols 2,054,061 2,054,061 ~ ~ ~ p=1.000 n=6
Types 899,813 899,813 ~ ~ ~ p=1.000 n=6
Memory used 2,067,168k (± 0.01%) 2,067,158k (± 0.00%) ~ 2,067,026k 2,067,267k p=0.810 n=6
Parse Time 8.10s (± 0.30%) 8.08s (± 0.52%) ~ 8.03s 8.15s p=0.373 n=6
Bind Time 2.73s (± 0.89%) 2.73s (± 0.50%) ~ 2.71s 2.75s p=0.871 n=6
Check Time 81.49s (± 0.21%) 81.09s (± 0.84%) ~ 79.82s 81.89s p=0.093 n=6
Emit Time 0.17s (± 3.32%) 0.34s (±124.05%) ~ 0.16s 1.19s p=0.476 n=6
Total Time 92.47s (± 0.18%) 92.23s (± 0.36%) ~ 91.90s 92.86s p=0.065 n=6
self-build-src - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,229,637 1,229,789 +152 (+ 0.01%) ~ ~ p=0.001 n=6
Types 260,905 260,912 +7 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 2,375,597k (± 2.63%) 2,376,172k (± 2.59%) ~ 2,349,142k 2,501,777k p=0.689 n=6
Parse Time 4.97s (± 0.79%) 4.95s (± 0.55%) ~ 4.90s 4.98s p=0.295 n=6
Bind Time 1.91s (± 0.29%) 1.91s (± 0.88%) ~ 1.90s 1.94s p=1.000 n=6
Check Time 34.01s (± 0.32%) 34.01s (± 0.29%) ~ 33.85s 34.15s p=0.936 n=6
Emit Time 2.59s (± 3.48%) 2.60s (± 2.64%) ~ 2.53s 2.70s p=0.748 n=6
Total Time 43.50s (± 0.35%) 43.50s (± 0.23%) ~ 43.35s 43.60s p=0.810 n=6
self-build-src-public-api - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 1,229,637 1,229,789 +152 (+ 0.01%) ~ ~ p=0.001 n=6
Types 260,905 260,912 +7 (+ 0.00%) ~ ~ p=0.001 n=6
Memory used 2,452,242k (± 2.52%) 2,426,564k (± 0.05%) ~ 2,424,792k 2,428,138k p=0.378 n=6
Parse Time 6.24s (± 0.55%) 6.24s (± 0.84%) ~ 6.16s 6.31s p=0.936 n=6
Bind Time 2.02s (± 1.02%) 2.01s (± 0.87%) ~ 1.98s 2.03s p=0.744 n=6
Check Time 40.47s (± 0.28%) 40.55s (± 0.13%) ~ 40.47s 40.61s p=0.148 n=6
Emit Time 3.13s (± 3.03%) 3.15s (± 3.31%) ~ 2.97s 3.28s p=1.000 n=6
Total Time 51.87s (± 0.28%) 51.97s (± 0.27%) ~ 51.76s 52.17s p=0.199 n=6
self-compiler - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 258,611 258,657 +46 (+ 0.02%) ~ ~ p=0.001 n=6
Types 105,018 105,029 +11 (+ 0.01%) ~ ~ p=0.001 n=6
Memory used 428,357k (± 0.02%) 428,422k (± 0.01%) ~ 428,371k 428,485k p=0.066 n=6
Parse Time 3.33s (± 0.67%) 3.31s (± 0.70%) ~ 3.28s 3.35s p=0.139 n=6
Bind Time 1.32s (± 0.83%) 1.33s (± 0.95%) ~ 1.32s 1.35s p=0.145 n=6
Check Time 18.09s (± 0.52%) 18.18s (± 0.25%) ~ 18.09s 18.22s p=0.198 n=6
Emit Time 1.36s (± 1.00%) 1.38s (± 1.43%) ~ 1.35s 1.41s p=0.284 n=6
Total Time 24.11s (± 0.43%) 24.20s (± 0.27%) ~ 24.12s 24.28s p=0.199 n=6
ts-pre-modules - node (v18.15.0, x64)
Errors 35 35 ~ ~ ~ p=1.000 n=6
Symbols 224,565 95,979 🟩-128,586 (-57.26%) ~ ~ p=0.001 n=6
Types 93,734 86 🟩-93,648 (-99.91%) ~ ~ p=0.001 n=6
Memory used 369,877k (± 0.04%) 228,504k (± 0.01%) 🟩-141,373k (-38.22%) 228,459k 228,534k p=0.005 n=6
Parse Time 2.84s (± 0.76%) 2.94s (± 0.61%) +0.10s (+ 3.34%) 2.92s 2.97s p=0.005 n=6
Bind Time 1.59s (± 0.92%) 1.42s (± 0.36%) 🟩-0.17s (-10.58%) 1.42s 1.43s p=0.004 n=6
Check Time 15.76s (± 0.82%) 0.00s 🟩-15.76s (-100.00%) ~ ~ p=0.003 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 20.19s (± 0.60%) 4.36s (± 0.40%) 🟩-15.83s (-78.39%) 4.35s 4.39s p=0.005 n=6
vscode - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 2,845,118 2,845,118 ~ ~ ~ p=1.000 n=6
Types 963,977 963,977 ~ ~ ~ p=1.000 n=6
Memory used 3,016,958k (± 0.00%) 3,016,896k (± 0.00%) ~ 3,016,771k 3,016,997k p=0.230 n=6
Parse Time 13.92s (± 0.32%) 13.92s (± 0.47%) ~ 13.86s 14.04s p=0.872 n=6
Bind Time 4.15s (± 0.43%) 4.20s (± 2.40%) ~ 4.14s 4.40s p=0.329 n=6
Check Time 73.51s (± 0.25%) 73.44s (± 0.31%) ~ 73.15s 73.82s p=0.422 n=6
Emit Time 23.68s (± 0.43%) 23.81s (± 0.70%) ~ 23.54s 23.99s p=0.172 n=6
Total Time 115.26s (± 0.13%) 115.37s (± 0.34%) ~ 114.91s 116.00s p=0.748 n=6
webpack - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 266,361 266,361 ~ ~ ~ p=1.000 n=6
Types 108,554 108,554 ~ ~ ~ p=1.000 n=6
Memory used 411,490k (± 0.03%) 411,530k (± 0.02%) ~ 411,352k 411,625k p=0.630 n=6
Parse Time 4.77s (± 1.02%) 4.79s (± 0.59%) ~ 4.73s 4.80s p=1.000 n=6
Bind Time 2.08s (± 0.74%) 2.07s (± 0.66%) ~ 2.05s 2.09s p=1.000 n=6
Check Time 21.12s (± 0.35%) 21.17s (± 0.16%) ~ 21.14s 21.23s p=0.199 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 27.98s (± 0.27%) 28.03s (± 0.22%) ~ 27.94s 28.12s p=0.222 n=6
xstate-main - node (v18.15.0, x64)
Errors 0 0 ~ ~ ~ p=1.000 n=6
Symbols 526,436 526,436 ~ ~ ~ p=1.000 n=6
Types 179,425 179,425 ~ ~ ~ p=1.000 n=6
Memory used 463,770k (± 0.07%) 464,222k (± 0.07%) +452k (+ 0.10%) 463,582k 464,467k p=0.031 n=6
Parse Time 3.15s (± 0.65%) 3.14s (± 0.77%) ~ 3.12s 3.19s p=0.466 n=6
Bind Time 1.18s (± 0.69%) 1.19s (± 0.69%) ~ 1.18s 1.20s p=0.666 n=6
Check Time 18.15s (± 0.22%) 18.16s (± 0.76%) ~ 17.95s 18.31s p=0.872 n=6
Emit Time 0.00s 0.00s ~ ~ ~ p=1.000 n=6
Total Time 22.49s (± 0.17%) 22.50s (± 0.60%) ~ 22.29s 22.65s p=0.936 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • Compiler-Unions - node (v18.15.0, x64)
  • angular-1 - node (v18.15.0, x64)
  • mui-docs - node (v18.15.0, x64)
  • self-build-src - node (v18.15.0, x64)
  • self-build-src-public-api - node (v18.15.0, x64)
  • self-compiler - node (v18.15.0, x64)
  • ts-pre-modules - node (v18.15.0, x64)
  • vscode - node (v18.15.0, x64)
  • webpack - node (v18.15.0, x64)
  • xstate-main - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

tsserver

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-UnionsTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 3,426ms (± 0.28%) 3,423ms (± 0.47%) ~ 3,397ms 3,439ms p=0.936 n=6
Req 2 - geterr 7,615ms (± 0.73%) 7,632ms (± 0.71%) ~ 7,546ms 7,689ms p=0.575 n=6
Req 3 - references 430ms (± 0.28%) 434ms (± 1.14%) +5ms (+ 1.09%) 429ms 441ms p=0.042 n=6
Req 4 - navto 339ms (± 0.83%) 340ms (± 0.62%) ~ 336ms 342ms p=0.241 n=6
Req 5 - completionInfo count 1,357 1,357 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 122ms (± 7.17%) 121ms (± 6.15%) ~ 111ms 129ms p=0.810 n=6
CompilerTSServer - node (v18.15.0, x64)
Req 1 - updateOpen 3,610ms (± 0.63%) 3,621ms (± 0.52%) ~ 3,588ms 3,644ms p=0.378 n=6
Req 2 - geterr 5,706ms (± 0.44%) 5,714ms (± 0.33%) ~ 5,693ms 5,736ms p=0.810 n=6
Req 3 - references 446ms (± 0.74%) 447ms (± 1.20%) ~ 443ms 458ms p=1.000 n=6
Req 4 - navto 342ms (± 1.47%) 342ms (± 1.41%) ~ 340ms 352ms p=0.607 n=6
Req 5 - completionInfo count 1,519 1,519 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 117ms (± 7.52%) 122ms (± 5.18%) ~ 109ms 125ms p=0.402 n=6
xstate-main-1-tsserver - node (v18.15.0, x64)
Req 1 - updateOpen 7,636ms (± 0.27%) 7,620ms (± 0.65%) ~ 7,537ms 7,685ms p=0.575 n=6
Req 2 - geterr 1,688ms (± 0.88%) 1,700ms (± 0.80%) ~ 1,681ms 1,714ms p=0.148 n=6
Req 3 - references 136ms (± 3.13%) 135ms (± 4.75%) ~ 129ms 144ms p=0.629 n=6
Req 4 - navto 589ms (± 1.32%) 590ms (± 1.89%) ~ 574ms 601ms p=1.000 n=6
Req 5 - completionInfo count 3,413 3,413 ~ ~ ~ p=1.000 n=6
Req 5 - completionInfo 1,268ms (± 1.95%) 1,265ms (± 2.48%) ~ 1,228ms 1,303ms p=1.000 n=6
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • CompilerTSServer - node (v18.15.0, x64)
  • Compiler-UnionsTSServer - node (v18.15.0, x64)
  • xstate-main-1-tsserver - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

startup

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
tsc-startup - node (v18.15.0, x64)
Execution time 187.19ms (± 0.16%) 187.09ms (± 0.15%) -0.10ms (- 0.05%) 185.43ms 191.60ms p=0.002 n=600
tsserver-startup - node (v18.15.0, x64)
Execution time 294.18ms (± 0.28%) 294.28ms (± 0.28%) +0.10ms (+ 0.03%) 286.99ms 300.01ms p=0.004 n=600
tsserverlibrary-startup - node (v18.15.0, x64)
Execution time 236.25ms (± 0.14%) 236.16ms (± 0.13%) -0.09ms (- 0.04%) 234.76ms 238.46ms p=0.011 n=600
typescript-startup - node (v18.15.0, x64)
Execution time 287.72ms (± 0.29%) 287.84ms (± 0.29%) +0.12ms (+ 0.04%) 280.52ms 295.01ms p=0.032 n=600
System info unknown
Hosts
  • node (v18.15.0, x64)
Scenarios
  • tsc-startup - node (v18.15.0, x64)
  • tsserver-startup - node (v18.15.0, x64)
  • tsserverlibrary-startup - node (v18.15.0, x64)
  • typescript-startup - node (v18.15.0, x64)
Benchmark Name Iterations
Current pr 6
Baseline baseline 6

Developer Information:

Download Benchmarks

@typescript-bot
Copy link
Collaborator

Hey @sheetalkamat, the results of running the DT tests are ready.

Everything looks the same!

You can check the log here.

@AlQubaiisi

This comment was marked as spam.

@jakebailey
Copy link
Member

Uh, why does that perf run look like that?

@sheetalkamat
Copy link
Member Author

To me it seems more like a noise. #57196 didnt change anything in compiler and still shows increase in symbols etc
Also Req 3 - references increase makes no sense, no change in tsserver at all

@jakebailey
Copy link
Member

jakebailey commented Jun 4, 2024

I was referring to the bit where ts-pre-modules stopped compiling, so something's not quite right when when running on that last codebase.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants