⚡ Performance: Project service doesn't cache all fs.statSync

[JoshuaKGoldberg]

Acknowledgement

• I acknowledge that issues using this template may be closed without further explanation at the maintainer's discretion.

Comment

🔎 Search Terms

project service fs stat statSync

🙁 Actual behavior

When using typescript-eslint's parserOptions.projectService, type checking APIs switch from the traditional manual TypeScript ts.Program approach to the editor-style ts.ProjectService. We're observing excess calls to the ts.sys.statSync function on some paths in node_modules/ - up to a few dozen for some paths (!).

🙂 Expected behavior

There should be no uncached statSync calls, I'd think? Even if in a persistent session, I'd expect them to be debounced in some way.

As a draft, I added a basic caching Map to statSync and ran a before & after comparison with hyperfine. The results showed a ~7-12% improvement in lint time:

Variant	Measurement	User Time
Baseline	3.112 s ± 0.033 s	4.382
Caching	2.740 s ± 0.030 s	4.032

diff patch to switch to the Caching variant...

diff --git a/node_modules/typescript/lib/typescript.js b/node_modules/typescript/lib/typescript.js
index 4baad59..44639d5 100644
--- a/node_modules/typescript/lib/typescript.js
+++ b/node_modules/typescript/lib/typescript.js
@@ -8546,9 +8546,15 @@ var sys = (() => {
         }
       }
     };
+    const statCache = new Map();
     return nodeSystem;
     function statSync(path) {
-      return _fs.statSync(path, { throwIfNoEntry: false });
+      if (statCache.has(path)) {
+        return statCache.get(path);
+      }
+      const result = _fs.statSync(path, { throwIfNoEntry: false });
+      statCache.set(path, result);
+      return result;
     }
     function enableCPUProfiler(path, cb) {
       if (activeSession) {

Additional information about the issue

On the typescript-eslint side:

• https://github.com/typescript-eslint/performance is the performance repository showing how to create these profiles.
  • https://github.com/typescript-eslint/performance/blob/2f8e7b33c25ce445dc1a645e7e5b95cb79a5c756/README.md#comparison-project-service-uncached-file-system-stats has the measurements for this specific investigation.
• feat(typescript-estree): add EXPERIMENTAL_useProjectService option to use TypeScript project service typescript-eslint/typescript-eslint#6754 is the PR that first added usage of the project service in typescript-eslint.

These are the top 10 most common paths called by statSync...

32 Error: /Users/josh/repos/performance/node_modules/execa/types/arguments
28 Error: /Users/josh/repos/performance/node_modules/execa/types/stdio
28 Error: /Users/josh/repos/performance/node_modules/execa/types/return
26 Error: /Users/josh/repos/performance/node_modules/execa/types
17 Error: /Users/josh/repos/performance/node_modules/execa/types/methods
14 Error: /Users/josh/repos/performance/node_modules/execa/types/subprocess
9 Error: /Users/josh/repos/performance/node_modules/prettier
9 Error: /Users/josh/repos/performance/node_modules/execa/types/arguments/options.d.ts
8 Error: /Users/josh/repos/performance/node_modules/execa/types/transform
7 Error: /Users/josh/repos/performance/node_modules/execa/types/stdio/type.d.ts

Here's an example call stack from the most common one...

Error
    at statSync (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:8568:19)
    at fileSystemEntryExists (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:8794:22)
    at Object.directoryExists (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:8816:14)
    at _AutoImportProviderProject.directoryExists (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:182483:40)
    at directoryProbablyExists (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:20914:40)
    at tryAddingExtensions (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:44501:29)
    at loadModuleFromFileNoImplicitExtensions (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:44484:10)
    at loadModuleFromFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:44460:40)
    at nodeLoadModuleByRelativeName (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:44409:30)
    at tryResolve (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:44365:25)
    at nodeModuleNameResolverWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:44283:14)
    at nodeNextModuleNameResolverWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:44166:10)
    at nodeNextModuleNameResolver (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:44148:10)
    at resolveModuleName (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:43972:18)
    at resolveModuleNameUsingGlobalCache (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:129141:25)
    at Object.resolve (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:129128:45)
    at resolveNamesWithLocalCache (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:129434:29)
    at Object.resolveModuleNameLiterals (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:129523:12)
    at _AutoImportProviderProject.resolveModuleNameLiterals (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:182461:33)
    at resolveModuleNamesWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:124514:20)
    at resolveModuleNamesReusingOldState (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:124600:14)
    at processImportedModules (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:126117:118)
    at findSourceFileWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125895:7)
    at findSourceFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125746:20)
    at processImportedModules (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:126143:11)
    at findSourceFileWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125895:7)
    at findSourceFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125746:20)
    at /Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125695:22
    at getSourceFileFromReferenceWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125666:26)
    at processSourceFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125693:5)
    at processRootFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:125485:5)
    at /Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:124210:41
    at forEach (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:2387:22)
    at createProgram (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:124210:5)
    at synchronizeHostDataWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:148957:15)
    at synchronizeHostData (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:148853:7)
    at Object.getProgram (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:149029:5)
    at _AutoImportProviderProject.updateGraphWorker (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:183153:41)
    at _AutoImportProviderProject.updateGraph (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:183009:32)
    at _AutoImportProviderProject.updateGraph (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:184155:37)
    at updateProjectIfDirty (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:184785:36)
    at ConfiguredProject2.getPackageJsonAutoImportProvider (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:183741:9)
    at ConfiguredProject2.updateGraph (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:183033:12)
    at ConfiguredProject2.updateGraph (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:184311:24)
    at updateWithTriggerFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:184794:11)
    at _ProjectService.reloadConfiguredProject (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:186410:5)
    at ConfiguredProject2.updateGraph (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:184307:29)
    at updateWithTriggerFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:184794:11)
    at updateConfiguredProject (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:184802:9)
    at _ProjectService.findCreateOrReloadConfiguredProject (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:187271:44)
    at _ProjectService.tryFindDefaultConfiguredProjectForOpenScriptInfo (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:187295:25)
    at _ProjectService.tryFindDefaultConfiguredProjectAndLoadAncestorsForOpenScriptInfo (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:187341:25)
    at _ProjectService.assignProjectToOpenedScriptInfo (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:187226:27)
    at _ProjectService.openClientFileWithNormalizedPath (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:187421:48)
    at _ProjectService.openClientFile (/Users/josh/repos/performance/node_modules/typescript/lib/typescript.js:187129:17)
    at useProgramFromProjectService (/Users/josh/repos/performance/node_modules/@typescript-eslint/typescript-estree/dist/useProgramFromProjectService.js:61:28)
    at getProgramAndAST (/Users/josh/repos/performance/node_modules/@typescript-eslint/typescript-estree/dist/parser.js:44:100)
    at parseAndGenerateServices (/Users/josh/repos/performance/node_modules/@typescript-eslint/typescript-estree/dist/parser.js:155:11)
    at Object.parseForESLint (/Users/josh/repos/performance/node_modules/@typescript-eslint/parser/dist/parser.js:101:80)
    at Object.parse (/Users/josh/repos/performance/node_modules/eslint/lib/languages/js/index.js:186:26)
    at parse (/Users/josh/repos/performance/node_modules/eslint/lib/linter/linter.js:931:29)
    at Linter._verifyWithFlatConfigArrayAndWithoutProcessors (/Users/josh/repos/performance/node_modules/eslint/lib/linter/linter.js:1696:33)
    at Linter._verifyWithFlatConfigArray (/Users/josh/repos/performance/node_modules/eslint/lib/linter/linter.js:2062:21)
    at Linter.verify (/Users/josh/repos/performance/node_modules/eslint/lib/linter/linter.js:1528:61)
    at Linter.verifyAndFix (/Users/josh/repos/performance/node_modules/eslint/lib/linter/linter.js:2299:29)
    at verifyText (/Users/josh/repos/performance/node_modules/eslint/lib/eslint/eslint.js:498:48)
    at /Users/josh/repos/performance/node_modules/eslint/lib/eslint/eslint.js:939:40
    at async Promise.all (index 1)
    at async ESLint.lintFiles (/Users/josh/repos/performance/node_modules/eslint/lib/eslint/eslint.js:880:25)
    at async Object.execute (/Users/josh/repos/performance/node_modules/eslint/lib/cli.js:521:23)
    at async main (/Users/josh/repos/performance/node_modules/eslint/bin/eslint.js:153:22)

cc @sheetalkamat as FYI, after a pairing with @jakebailey.

[sheetalkamat]

I will take a look but do you really need AutoImportProviderProject ? If not you should probably set the options to disable that ?
Currently auto import provider cannot reuse the resolutions from configured project (i am looking into sharing those as part of #55968 and may be that will help but i will investigate the stacks soon.

[jakebailey]

What options should they be setting to make that project not used? They definitely don't need auto import anything.

[sheetalkamat]

preferences.includePackageJsonAutoImports need to be set with setHostConfiguration

[sheetalkamat]

@jakebailey can you try and see if un cached statsync and realpath calls reduce after that?

[jakebailey]

Hm, I sent typescript-eslint/typescript-eslint#9586 but it's saying:

    Received message: "service.setHostConfiguration is not a function"
          68 |   });
          69 |
        > 70 |   service.setHostConfiguration({
             |           ^
          71 |     preferences: {
          72 |       includePackageJsonAutoImports: 'off',
          73 |     },
          at setHostConfiguration (src/create-program/createProjectService.ts:70:11)
          at tests/lib/createProjectService.test.ts:53:27

Have not tested locally (will do that shortly); maybe this func is new?

[sheetalkamat]

Seems like atleast 5 year old

[JoshuaKGoldberg]

Oh, the unit tests in typescript-eslint mock out the service - see https://github.com/typescript-eslint/typescript-eslint/blob/5542aeb9440280dd4c0be529e9842f12221737ff/packages/typescript-estree/tests/lib/createProjectService.test.ts#L8

[jakebailey]

Ah, yeah, so I'll definitely have to real-world test it.

[sheetalkamat]

I think the eslint change to disable auto import provider should help with the stats cache issue. With that change those high number of stats cache hitting goes down significantly. I confirmed that by running tsserver that just by default returns "off" for that option.

The other two locations where we get from cache are:

• FileExists call that happens at project service level instead of project since its for getting the script info. I have to see if we can do something here but these should be only happening for all files in the project that are not open only once.
• One for typeAquisition that was not needed since its disabled. - i have fix for that and will file PR for that Dont filter type aquisition when its not enabled #59351

[jakebailey]

Before:

┌───────┬──────────────────────┬──────────────────────┬──────────────────────┬──────────────────────┐
│ files │ project (even)       │ project (references) │ service (even)       │ service (references) │
┼───────┼──────────────────────┼──────────────────────┼──────────────────────┼──────────────────────┤
│ 128   │ '1.788 s ±  0.040 s' │ '1.788 s ±  0.041 s' │ '1.861 s ±  0.053 s' │ '2.721 s ±  0.040 s' │
│ 512   │ '2.747 s ±  0.032 s' │ '2.766 s ±  0.042 s' │ '2.974 s ±  0.036 s' │ '4.032 s ±  0.041 s' │
│ 1024  │ '3.943 s ±  0.044 s' │ '4.008 s ±  0.040 s' │ '4.644 s ±  0.053 s' │ '5.925 s ±  0.180 s' │
┴───────┴──────────────────────┴──────────────────────┴──────────────────────┴──────────────────────┘

After typescript-eslint/typescript-eslint#9586:

┌───────┬──────────────────────┬──────────────────────┬──────────────────────┬──────────────────────┐
│ files │ project (even)       │ project (references) │ service (even)       │ service (references) │
┼───────┼──────────────────────┼──────────────────────┼──────────────────────┼──────────────────────┤
│ 128   │ '1.753 s ±  0.019 s' │ '1.761 s ±  0.019 s' │ '1.804 s ±  0.036 s' │ '2.478 s ±  0.046 s' │
│ 512   │ '2.697 s ±  0.049 s' │ '2.707 s ±  0.031 s' │ '2.936 s ±  0.039 s' │ '3.837 s ±  0.050 s' │
│ 1024  │ '3.880 s ±  0.044 s' │ '3.916 s ±  0.028 s' │ '4.511 s ±  0.030 s' │ '5.493 s ±  0.065 s' │
┴───────┴──────────────────────┴──────────────────────┴──────────────────────┴──────────────────────┘

So, clearly better by a little bit.

I haven't yet done the same stat counting augmentation; I'm not sure which command to run to exactly reproduce what @JoshuaKGoldberg was doing anymore.

[jakebailey]

The test was just me in the linked performance repo, modifying node_modules with: