Supports incremental build in tsc --b --w mode

[sheetalkamat]

Some notes:

• Since we use builder, the first time program is created, its going to emit all the files and not just changed - just like it does in tsc --watch mode
• We do not cache source files yet and that's something to be looked at later.
  Two things can be done:
  • To keep source files for program thats being updated in last few cycles
  • To release the source files if size exceeds or program is not used for few build cycles
  • Adding this to Create tsbuild public API and use incremental build in tsbuild --watch mode #28640
• We use hash from source file's text as the version.
• Because in tsbuild we get semantic diagnostics before emit, the EmitAndSemanticDiagnosticsBuilderProgram's existing assumption was not satisfied (that is changed files are already emitted before getting semantic diagnostics). So in such case, we store list of files to emit later so as to help with that case.
• In this we also have a new Builder method to release the program and its source files to ensure that we aren't holding on program after emit and/or semantic diagnostics are complete. That way the builder state can be in memory for comparison and finding out what are new changes that need emitting while creating new program but we aren't holding onto these costly (memory wise) resources. (all the program methods are thus available only till program is released.

Here are the perf numbers:
Local change::

File change detected. Starting incremental compilation... Project 'SubProject 4' is out of date because oldest output 'someOutputFile.js' is older than newest input 'SubProject 2' Building project 'SubProject 4'... Updating unchanged output timestamps of project 'SubProject 4'... Before After Files: 8305 Files: 8305 Lines: 509633 Lines: 509633 Nodes: 1741390 Nodes: 1741390 Identifiers: 563590 Identifiers: 563590 Symbols: 597260 Symbols: 316035 Types: 175190 Types: 232 Memory used: 1418992K Memory used: 1859282K I/O Read time: 2.83s I/O Read time: 3.85s Parse time: 2.43s Parse time: 3.42s Program time: 23.45s Program time: 33.07s Bind time: 1.56s Bind time: 1.71s Check time: 11.86s Check time: 0.04s transformTime time: 13.72s transformTime time: 0.25s Source Map time: 2.58s Source Map time: 0.01s commentTime time: 1.46s commentTime time: 0.00s printTime time: 27.08s printTime time: 0.28s Emit time: 27.08s Emit time: 0.28s I/O Write time: 16.16s I/O Write time: 0.04s Total time: 63.96s Total time: 35.09s Building project 'SubProject 5'... Updating unchanged output timestamps of project 'SubProject 5'... Before After Files: 1875 Files: 1875 Lines: 207731 Lines: 207731 Nodes: 632955 Nodes: 632955 Identifiers: 212573 Identifiers: 212573 Symbols: 162334 Symbols: 125245 Types: 22975 Types: 71 Memory used: 1095817K Memory used: 311574K I/O Read time: 1.13s I/O Read time: 0.74s Parse time: 2.03s Parse time: 0.87s Program time: 9.64s Program time: 5.97s Bind time: 0.67s Bind time: 0.48s Check time: 2.17s transformTime time: 1.22s Source Map time: 0.77s commentTime time: 0.34s printTime time: 4.16s Emit time: 4.16s I/O Write time: 1.92s Total time: 16.64s Total time: 6.45s Building project 'SubProject 6'... Updating unchanged output timestamps of project 'SubProject 6'... Before After Files: 2396 Files: 2396 Lines: 221677 Lines: 221677 Nodes: 669898 Nodes: 669898 Identifiers: 221017 Identifiers: 221017 Symbols: 189874 Symbols: 143939 Types: 30412 Types: 71 Memory used: 1442324K Memory used: 333909K I/O Read time: 0.92s I/O Read time: 0.91s Parse time: 1.40s Parse time: 1.17s Program time: 8.91s Program time: 7.73s Bind time: 0.43s Bind time: 0.39s Check time: 1.74s transformTime time: 0.67s Source Map time: 0.18s commentTime time: 0.10s printTime time: 1.67s Emit time: 1.67s I/O Write time: 1.40s Total time: 12.74s Total time: 8.12s Building project 'SubProject 7'... Updating unchanged output timestamps of project 'SubProject 7'... Before After Files: 9326 Files: 9326 Lines: 552828 Lines: 552830 Nodes: 1896877 Nodes: 1896879 Identifiers: 605603 Identifiers: 605605 Symbols: 608250 Symbols: 330625 Types: 173986 Types: 71 Memory used: 2367385K Memory used: 953536K I/O Read time: 4.22s I/O Read time: 3.84s Parse time: 3.18s Parse time: 2.02s Program time: 36.41s Program time: 27.79s Bind time: 2.73s Bind time: 1.19s Check time: 13.15s transformTime time: 15.36s Source Map time: 2.35s commentTime time: 1.52s printTime time: 28.95s Emit time: 28.95s I/O Write time: 13.41s Total time: 81.23s Total time: 28.98s Building project 'SubProject 8'... Updating unchanged output timestamps of project 'SubProject 8'... Before After Files: 6146 Files: 6146 Lines: 367180 Lines: 367182 Nodes: 1201703 Nodes: 1201705 Identifiers: 386452 Identifiers: 386454 Symbols: 406182 Symbols: 232990 Types: 94077 Types: 71 Memory used: 1704251K Memory used: 1333457K I/O Read time: 3.06s I/O Read time: 2.50s Parse time: 3.61s Parse time: 2.06s Program time: 31.86s Program time: 19.05s Bind time: 1.76s Bind time: 0.77s Check time: 10.90s transformTime time: 7.64s Source Map time: 0.96s commentTime time: 0.91s printTime time: 13.81s Emit time: 13.81s I/O Write time: 6.67s Total time: 58.33s Total time: 19.82s Building project 'SubProject 9'... Updating unchanged output timestamps of project 'SubProject 9'... Before After Files: 3781 Files: 3781 Lines: 264899 Lines: 264899 Nodes: 834360 Nodes: 834360 Identifiers: 278052 Identifiers: 278052 Symbols: 261052 Symbols: 178924 Types: 52797 Types: 71 Memory used: 2175398K Memory used: 1620896K I/O Read time: 1.67s I/O Read time: 1.51s Parse time: 1.98s Parse time: 1.19s Program time: 16.03s Program time: 11.75s Bind time: 0.77s Bind time: 0.49s Check time: 4.86s transformTime time: 4.24s Source Map time: 0.40s commentTime time: 0.21s printTime time: 6.63s Emit time: 6.63s I/O Write time: 2.03s Total time: 28.29s Total time: 12.24s Solution Total Time: Before: 261.19, After: 110.7 

Local change:

File change detected. Starting incremental compilation... Project 'SubProject 4' is out of date because oldest output 'someOutputFile.js' is older than newest input 'SubProject 2' Building project 'SubProject 4'... Updating unchanged output timestamps of project 'SubProject 4'... Before After Files: 8305 Files: 8305 Lines: 509635 Lines: 509635 Nodes: 1741406 Nodes: 1741406 Identifiers: 563597 Identifiers: 563597 Symbols: 597262 Symbols: 315957 Types: 175191 Types: 114 Memory used: 3243603K Memory used: 690192K I/O Read time: 3.23s I/O Read time: 3.28s Parse time: 4.00s Parse time: 2.31s Program time: 35.91s Program time: 24.59s Bind time: 1.75s Bind time: 1.27s Check time: 15.76s Check time: 0.01s transformTime time: 15.10s transformTime time: 0.00s Source Map time: 2.93s Source Map time: 0.00s commentTime time: 1.85s commentTime time: 0.00s printTime time: 30.68s printTime time: 0.00s Emit time: 30.68s Emit time: 0.00s I/O Write time: 14.47s I/O Write time: 0.00s Total time: 84.10s Total time: 25.87s Project 'SubProject 5' is up to date with .d.ts files from its dependencies Updating output timestamps of project 'SubProject 5'... Project 'SubProject 6' is up to date with .d.ts files from its dependencies Updating output timestamps of project 'SubProject 6'... Project 'SubProject 7' is up to date with .d.ts files from its dependencies Updating output timestamps of project 'SubProject 7'... Project 'SubProject 8' is up to date with .d.ts files from its dependencies Updating output timestamps of project 'SubProject 8'... Project 'SubProject 9' is up to date with .d.ts files from its dependencies Updating output timestamps of project 'SubProject 9'... Solution Total Time: Before: 84.1, After: 25.87 Found 0 errors. Watching for file changes. 

[sheetalkamat]

Ping @sandersn @rbuckton @weswigham @DanielRosenwasser Would like to get this in soon.

[sandersn]

I looked over everything but I need to learn more about watch mode + tsbuild in person before signing off.

[sheetalkamat]

@sandersn comments?

[xaviergonz]

Just wondering, wouldn't it be even faster if the cache was also based on file size as first check before doing the actual hash?