RA uses 2.4 GB RSS on docs.rs

[jyn514]

Steps to replicate:

1. Install VSCode with the rust-analyzer extension
2. git clone https://github.com/rust-lang/docs.rs/
3. code docs.rs

Rust-analyzer uses 2.4g RSS, which kills most of the other programs I have running at the same time. I've started using vim instead.

[jyn514]

I'm happy to help debug this if you have suggestions.

[bjorn3]

It is expected that rust-analyzer uses a lot of memory. If you want to see what it is used for, you can run the "Rust Analyzer: Memory Usage (Clears Database)" command. After a few seconds, a new editor pane should open with the memory usage info.

[jyn514]

That gives a panic:

Panic context:
> request: textDocument/codeLens CodeLensParams {
    text_document: TextDocumentIdentifier {
        uri: Url {
            scheme: "file",
            host: None,
            port: None,
            path: "/home/joshua/src/rust/docs.rs/src/web/releases.rs",
            query: None,
            fragment: None,
        },
    },
    work_done_progress_params: WorkDoneProgressParams {
        work_done_token: None,
    },
    partial_result_params: PartialResultParams {
        partial_result_token: None,
    },
}

thread '<unnamed>' panicked at 'no value set for FileTextQuery(FileId(0))', /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/salsa-0.16.0/src/input.rs:104:32
stack backtrace:
   0: rust_begin_unwind
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:483
   1: std::panicking::begin_panic_fmt
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:437
   2: <salsa::input::InputStorage<Q> as salsa::plumbing::QueryStorageOps<Q>>::try_fetch::{{closure}}
   3: <salsa::input::InputStorage<Q> as salsa::plumbing::QueryStorageOps<Q>>::try_fetch
   4: <DB as base_db::SourceDatabaseExt>::file_text::__shim
   5: salsa::runtime::Runtime::execute_query_implementation
   6: salsa::derived::slot::Slot<Q,MP>::read_upgrade
   7: salsa::derived::slot::Slot<Q,MP>::read
   8: <salsa::derived::DerivedStorage<Q,MP> as salsa::plumbing::QueryStorageOps<Q>>::try_fetch
   9: <DB as ide_db::LineIndexDatabase>::line_index::__shim
  10: ide::Analysis::file_line_index
  11: rust_analyzer::handlers::handle_code_lens
  12: <F as threadpool::FnBox>::call_box
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Panic context:
> request: textDocument/hover HoverParams {
    text_document_position_params: TextDocumentPositionParams {
        text_document: TextDocumentIdentifier {
            uri: Url {
                scheme: "file",
                host: None,
                port: None,
                path: "/home/joshua/src/rust/docs.rs/src/web/releases.rs",
                query: None,
                fragment: None,
            },
        },
        position: Position {
            line: 459,
            character: 65,
        },
    },
    work_done_progress_params: WorkDoneProgressParams {
        work_done_token: None,
    },
}

thread '<unnamed>' panicked at 'no value set for FileTextQuery(FileId(0))', /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/salsa-0.16.0/src/input.rs:104:32
stack backtrace:
   0: rust_begin_unwind
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:483
   1: std::panicking::begin_panic_fmt
             at /rustc/7eac88abb2e57e752f3302f02be5f3ce3d7adfb4/library/std/src/panicking.rs:437
   2: <salsa::input::InputStorage<Q> as salsa::plumbing::QueryStorageOps<Q>>::try_fetch::{{closure}}
   3: <salsa::input::InputStorage<Q> as salsa::plumbing::QueryStorageOps<Q>>::try_fetch
   4: <DB as base_db::SourceDatabaseExt>::file_text::__shim
   5: salsa::runtime::Runtime::execute_query_implementation
   6: salsa::derived::slot::Slot<Q,MP>::read_upgrade
   7: salsa::derived::slot::Slot<Q,MP>::read
   8: <salsa::derived::DerivedStorage<Q,MP> as salsa::plumbing::QueryStorageOps<Q>>::try_fetch
   9: <DB as ide_db::LineIndexDatabase>::line_index::__shim
  10: ide::Analysis::file_line_index
  11: rust_analyzer::from_proto::file_position
  12: rust_analyzer::handlers::handle_hover
  13: <F as threadpool::FnBox>::call_box
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

and the following report:

Per-query memory usage:
   387mb ItemTreeQuery
    87mb CrateDefMapQueryQuery
    52mb MacroArgTextQuery
    34mb ImplDataQuery
    34mb SourceRootQuery (purge)
    24mb InternEagerExpansionQuery
    18mb GenericParamsQuery
    15mb MacroDefQuery
    13mb ParseQuery
    10mb GenericParamsQuery (purge)
     9mb ImplSelfTyQuery (purge)
     9mb ImplTraitQuery (purge)
     8mb ParseMacroExpansionQuery (purge)
     8mb FileSourceRootQuery (purge)
     8mb AstIdMapQuery
     8mb MacroExpandQuery (purge)
     8mb AstIdMapQuery (purge)
     7mb ItemTreeQuery (purge)
     7mb AttrsQuery
     7mb InternFunctionQuery
     7mb ImplDataQuery (purge)
     7mb InternMacroQuery (purge)
     6mb InternMacroQuery
     6mb ImplTraitQuery
     6mb CrateGraphQuery (purge)
     6mb MacroArgTextQuery (purge)
     6mb InternFunctionQuery (purge)
     4mb FunctionDataQuery
     4mb TraitImplsInCrateQuery
     4mb ImplDataQuery (deps)
     4mb ImplTraitQuery (deps)
     4mb ImplSelfTyQuery (deps)
  3955kb TraitImplsInDepsQuery
  3786kb InternImplQuery
  3136kb InternImplQuery (purge)
  2716kb GenericParamsQuery (deps)
  2625kb AttrsQuery (purge)
  2603kb ParseMacroExpansionQuery (deps)
  2233kb MacroExpandQuery (deps)
  2121kb TyQuery (purge)
  1985kb GenericDefaultsQuery (purge)
  1973kb ItemTreeQuery (deps)
  1880kb ImplDatumQuery
  1871kb MacroArgTextQuery (deps)
  1659kb CrateDefMapQueryQuery (deps)
  1487kb InternConstQuery
  1440kb InternConstQuery (purge)
  1394kb BodyQuery
  1332kb AstIdMapQuery (deps)
  1030kb InferQueryQuery
   915kb InternTypeAliasQuery
   848kb InternTypeAliasQuery (purge)
   825kb GenericPredicatesQuery (purge)
   817kb ParseQuery (purge)
   764kb TraitSolveQuery (purge)
   697kb TraitImplsInCrateQuery (deps)
   656kb InternStructQuery (purge)
   646kb InternStructQuery
   557kb ParseMacroExpansionQuery
   555kb GenericDefaultsQuery
   554kb ImplDatumQuery (purge)
   508kb BodyWithSourceMapQuery
   471kb GenericDefaultsQuery (deps)
   448kb TyQuery
   443kb AttrsQuery (deps)
   409kb EnumDataQuery
   388kb InternEagerExpansionQuery (purge)
   356kb ImplSelfTyQuery
   299kb TyQuery (deps)
   276kb ImplDatumQuery (deps)
   253kb GenericPredicatesQuery
   251kb MacroExpandQuery
   235kb CrateDefMapQueryQuery (purge)
   199kb GenericPredicatesQuery (deps)
   193kb TypeAliasDataQuery
   190kb MacroDefQuery (purge)
   172kb FunctionDataQuery (purge)
   156kb ExprScopesQuery
   144kb ParseQuery (deps)
   140kb InternEnumQuery
   136kb InferQueryQuery (deps)
   116kb TraitSolveQuery (deps)
   103kb InternTraitQuery
   102kb StructDataQuery
   100kb InherentImplsInCrateQuery (deps)
    98kb InternEnumQuery (purge)
    98kb InternTraitQuery (purge)
    77kb InherentImplsInCrateQuery
    71kb CrateLangItemsQuery (deps)
    68kb AssociatedTyValueQuery
    61kb TraitImplsInCrateQuery (purge)
    52kb TypeAliasDataQuery (purge)
    49kb InternStaticQuery (purge)
    45kb CallableItemSignatureQuery
    44kb InternStaticQuery
    44kb TraitSolveQuery
    42kb FunctionDataQuery (deps)
    39kb MacroDefQuery (deps)
    36kb GenericPredicatesForParamQuery (purge)
    35kb GenericPredicatesForParamQuery
    34kb ValueTyQuery (purge)
    34kb FnDefDatumQuery
    31kb TraitDataQuery
    29kb AssociatedTyValueQuery (purge)
    29kb CallableItemSignatureQuery (purge)
    27kb EnumDataQuery (purge)
    21kb BodyWithSourceMapQuery (deps)
    19kb TraitDatumQuery
    19kb StructDatumQuery
    17kb StructDatumQuery (purge)
    15kb ConstDataQuery
    14kb TypeAliasDataQuery (deps)
    14kb CallableItemSignatureQuery (deps)
    13kb GenericPredicatesForParamQuery (deps)
    13kb AssociatedTyDataQuery
    13kb TraitDataQuery (purge)
    11kb ConstDataQuery (purge)
    11kb SourceRootCratesQuery (deps)
    10kb AssociatedTyValueQuery (deps)
    10kb FieldTypesQuery
     9kb ValueTyQuery
     8kb TraitDatumQuery (purge)
     8kb TraitDataQuery (deps)
     8kb StructDataQuery (purge)
     8kb BodyWithSourceMapQuery (purge)
     8kb BodyQuery (purge)
     8kb ExprScopesQuery (purge)
     8kb InferQueryQuery (purge)
     8kb EnumDataQuery (deps)
     8kb LineIndexQuery
     7kb ValueTyQuery (deps)
     6kb FnDefDatumQuery (purge)
     5kb LangItemQuery (purge)
     4kb CrateLangItemsQuery
     4kb InternCallableDefQuery (purge)
     4kb StructDatumQuery (deps)
     4kb InternClosureQuery (purge)
   3696b AssociatedTyDataQuery (purge)
   3536b FieldTypesQuery (purge)
   3392b TraitDatumQuery (deps)
   3312b InherentImplsInCrateQuery (purge)
   3088b ConstDataQuery (deps)
   2976b TraitImplsInDepsQuery (deps)
   2848b LibraryRootsQuery (purge)
   2704b FnDefDatumQuery (deps)
   2480b StructDataQuery (deps)
   2400b AssociatedTyDataQuery (deps)
   2320b FieldTypesQuery (deps)
   1616b BodyQuery (deps)
   1520b ExprScopesQuery (deps)
   1120b LangItemQuery (deps)
    560b InternUnionQuery
    528b CrateLangItemsQuery (purge)
    448b InternUnionQuery (purge)
    320b LocalRootsQuery (purge)
    272b SourceRootCratesQuery (purge)
    272b TraitImplsInDepsQuery (purge)
    272b LineIndexQuery (purge)
    128b SourceRootCratesQuery
      0b CrateGraphQuery
      0b CrateGraphQuery (deps)
      0b FileTextQuery
      0b FileTextQuery (deps)
      0b FileSourceRootQuery
      0b FileSourceRootQuery (deps)
      0b SourceRootQuery
      0b SourceRootQuery (deps)
      0b UnionDataQuery
      0b UnionDataQuery (deps)
      0b UnionDataQuery (purge)
      0b StaticDataQuery
      0b StaticDataQuery (deps)
      0b StaticDataQuery (purge)
      0b LangItemQuery
      0b ImportMapQuery
      0b ImportMapQuery (deps)
      0b ImportMapQuery (purge)
      0b AssociatedTyDataQuery
      0b AssociatedTyDataQuery (deps)
      0b AssociatedTyDataQuery (purge)
      0b ReturnTypeImplTraitsQuery
      0b ReturnTypeImplTraitsQuery (deps)
      0b ReturnTypeImplTraitsQuery (purge)
      0b InternCallableDefQuery
      0b InternCallableDefQuery (deps)
      0b InternTypeParamIdQuery
      0b InternTypeParamIdQuery (deps)
      0b InternTypeParamIdQuery (purge)
      0b InternImplTraitIdQuery
      0b InternImplTraitIdQuery (deps)
      0b InternImplTraitIdQuery (purge)
      0b InternClosureQuery
      0b InternClosureQuery (deps)
      0b FileSymbolsQuery
      0b FileSymbolsQuery (deps)
      0b FileSymbolsQuery (purge)
      0b LibrarySymbolsQuery
      0b LibrarySymbolsQuery (deps)
      0b LibrarySymbolsQuery (purge)
      0b LocalRootsQuery
      0b LocalRootsQuery (deps)
      0b LibraryRootsQuery
      0b LibraryRootsQuery (deps)
      0b LineIndexQuery (deps)
      0b InternMacroQuery (deps)
      0b InternEagerExpansionQuery (deps)
      0b InternFunctionQuery (deps)
      0b InternStructQuery (deps)
      0b InternUnionQuery (deps)
      0b InternEnumQuery (deps)
      0b InternConstQuery (deps)
      0b InternStaticQuery (deps)
      0b InternTraitQuery (deps)
      0b InternTypeAliasQuery (deps)
      0b InternImplQuery (deps)
      0b InternTypeParamIdQuery
      0b InternTypeParamIdQuery (deps)
      0b InternTypeParamIdQuery (purge)
 -3368mb FileTextQuery (purge)

(note: database has been cleared)

[jyn514]

So it looks like almost 3.3 GB is being used for FileTextQuery alone (??) Is that expected?

[bjorn3]

Notice how there is a - at the start. It seems that something else was running while the memory usage was being reported. Memory usage is reported by clearing parts of the salsa database one by one and then determining the change in memory usage of the whole process. If another thread caused the salsa database to be filled again, that will cause a negative memory usage to be reported for certain items.

[jyn514]

How can I get rust-analyzer to run in a single thread? Is there some way to get a more accurate report?

[bjorn3]

There is no way to run it in a single thread. The main lsp server loop runs on a different thread than the thread(s?) processing lsp requests.

[jyn514]

Ok, is there some way to tell the RA process not to accept new requests while it's dropping memory?

[jonas-schievink]

If you wait until r-a has finished indexing the project and stops using CPU, and you don't use the editor while the memory usage command is running, it should work

[jonas-schievink]

rust-analyzer uses 1.3 GB on the docs.rs code base for me (while looking at src/lib.rs). Is there anything else I have to do to reproduce this?

[jonas-schievink]

Profile on my machine:

Per-query memory usage:
   461mb ItemTreeQuery
   110mb CrateDefMapQueryQuery
    81mb FileTextQuery (purge)
    55mb MacroArgTextQuery
    48mb InternEagerExpansionQuery
    37mb ImplDataQuery
    20mb GenericParamsQuery
    16mb MacroDefQuery
    13mb GenericParamsQuery (purge)
    12mb ParseQuery
    10mb ImplSelfTyQuery (purge)
     9mb AstIdMapQuery
...

[jyn514]

@jonas-schievink I didn't do anything special, this is with src/web/mod.rs open. I used top for the measurements.

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND       
1994657 joshua    20   0 2905536   2.4g  14596 S   0.0  32.2   0:43.63 rust-analyze+ 

I don't know why I have different (and negative) numbers :/ I waited until rust-analyzer was using 0% CPU and disabled all other extensions, and still got the same result with -3367mb for FileTextQuery.

[0x8f701]

on my PC it's around 1.2GB but I saw some panics:

[lnicola]

I too get 1.3 GB after scrolling around and triggering go to definition on various names in src/web/mod.rs. Out of those, about 200 MB are unaccounted for in the Memory Usage output.