Garbage collection is not aggressive enough as memory usage gets higher.

[dnfield]

Flutter allocates images in such a way that the native memory is held on to as long as there are any references to the underlying SkImage. These images are typically held as GPU memory, and Skia has no functionality to clean up the memory until all references are dropped.

It has been observed in flutter/flutter#56482 that a forcing a garbage collection (e.g. via the observatory or by calling Dart_NotifyLowMemory) frees up the resources after the image has passed into the old gen correctly. However, the image can sit in the old gen for many many frames (seconds), and while allocating another large image will trigger a GC, there can be overlap between the allocation of the new image memory and the GC such that time passes where both images are still resident in GPU memory. This can be a problem on devices with lower memory when sufficiently large images are allocated.

A Flutter program can dispose an image when it doesn't want to use it anymore, but other garbage-collectable objects may still hold references to that image (e.g. SkPicture/Picture objects). While one option is to track down all the native references and try to clear them, that can become brittle and is basically re-implementing a subset of work that the GC would be doing for a GC. Instead, we'd like to have some API to tell the GC that doing a full collection now is believed to result in significant native memory reduction, and have the GC run when we dispose the image.

One suggestion offline was to implement some method to tell the VM memory size has changed - that's probably a good thing to have as well, but unfortunately wouldn't help us in this case, since to change the native memory allocation we would have to find all references and unref them as long as they're garbage.

/cc @rmacnak-google @mraleph @bkonyi
/cc @jason-simmons @chinmaygarde

[dnfield]

I'm actually seeing that if I report enough of the collectable objects as using enough memory, they get collected.

Unfortunately, "enough" is some absurdly high number - gigabytes of memory.

This seems very dangerous to me. Flutter does regularly call NotifyIdle when there's frame budget left, which is certainly the case in my test application. It's been suggested by @rmacnak-google (if I understood correctly) that NotifyIdle doesn't necessarily do a GC if the heap size is similar from the last time a GC was run. But that seems wrong if the heap is sufficiently large - the very next allocation made could be enough to OOM kill the program, and it might allocate before the GC thread succeeds in freeing all of the old gen memory.

[a-siva]

NotifyIdle is specified a deadline which tells the VM how much time is available, we apply some heuristics currently based on the deadline specified and factors like idle scavenge/mark sweep thresholds. If the heap is sufficiently large then I am pretty sure the idle notification will trigger a GC if it can finish within the deadline provided.

If you are observing a case where the heap is close to being full and idle notification is not triggering a GC when it could have done one within the deadline that could indicate some kind of bug in the thresholds computation. if you have a test that demonstrates it we could take a look.

Adding an API to allow the embedder to trigger GC is easy but this API would not guarantee a GC would happen as the VM has a number of heuristics which determine when a GC happens and having the embedder trigger a GC would potentially shift some of that to the embedder and I am not sure if that would solve the problem in all cases.

[dnfield]

In trying to create a smaller image based example with Flutter, I came up with this one - I believe it's a very similar problem, if not quite the same one, but I'm note 100% sure of that. At any rate, I would expect this program to not allocate memory the way it does. Running this program will consume as much as 6gb of memory at times before a GC finally collects all the bad stuff. I would have expected more aggressive garbage collection to happen during the delay - but it doesn't, even if I increase the delay, or further space out the iterations between allocating new lists from the old ones.

import 'dart:typed_data';

Uint8List pixels;
const int entries = 1000000;

void main() {
  printListLoop(0);
}

void printListLoop(int frame) {
  if (frame % 10 == 0) {
    pixels = Uint8List.fromList(List<int>.generate(
      entries,
      (int index) {
        return index % 255;
      },
    ));
  }
  print('Frame #$frame ${pixels[frame]}');
  Future<void>.delayed(const Duration(milliseconds: 10)).then((_) {
    printListLoop(frame + 1);
  });
}

The memory increase can be observed usint devtools or using OS native memory profiling/monitoring tools.

For devtools, run this program using dart --observe=1024 --disable-service-auth-codes simple.dart and then tell devtools where to find the port.

If you manually trigger a GC in devtools, the memory usage goes way down.

[dnfield]

Here's a similarly small Flutter based example, and here I'm certain NotifyIdle is getting called frequently based on the timeline:

import 'dart:typed_data';
import 'dart:ui';

Uint8List pixels;
int frame = 0;

void main() {
  window
    ..onBeginFrame = beginFrame
    ..scheduleFrame();
}

void beginFrame(Duration duration) {
  if (frame % 10 == 0) {
    pixels = Uint8List.fromList(List<int>.generate(
      1000000,
      (int index) {
        return index % 255;
      },
    ));
  }

  final PictureRecorder recorder = PictureRecorder();
  final Canvas canvas = Canvas(recorder);
  canvas.drawPaint(Paint());
  ParagraphBuilder paragraphBuilder = ParagraphBuilder(ParagraphStyle())
    ..pushStyle(TextStyle(fontSize: 30))
    ..addText('Frame #${frame++} ${pixels[frame]}');
  final Paragraph paragraph = paragraphBuilder.build();
  paragraph.layout(const ParagraphConstraints(width: double.infinity));
  canvas.drawParagraph(paragraph, const Offset(50, 50));
  final Picture picture = recorder.endRecording();
  SceneBuilder builder = SceneBuilder()..addPicture(Offset(10, 10), picture);
  window.render(builder.build());
  Future<void>.delayed(const Duration(milliseconds: 100)).then((_) {
    window.scheduleFrame();
  });
}

I still see memory grow much higher than I would have expected with this program.

[rmacnak-google]

Your example is shows a known issue with intergenerational garbage (cf https://dart-review.googlesource.com/c/sdk/+/133360). Because Dart traces generations independently, rather than tracing through new-space during old-space collections, a low rate of new-space allocations can retain old-space garbage. Notice that triggering a mark-sweep from Observatory does not significantly reduce heap size unless a scavenge is triggered first.

Also, the command line VM considers this example to have no idle time; it only considers an isolate idle if nothing happens for a full second.

[dnfield]

In the Flutter example in #42078 (comment), Dart_NotifyIdle gets called between most of the frames pumped. Is there some way we could at least improve that case?

[dnfield]

e.g. could we just trigger a scavenge when NotifyIdle is called?

[dnfield]

I'm assuming the boolean here exists for a reason, but commenting out

sdk/runtime/vm/heap/heap.cc

Line 405 in a2b5373

if (new_space_.ShouldPerformIdleScavenge(deadline)) {

makes the problem go away in Flutter applications.

Is there some way we could just make that check a bit more aggressive?

[dnfield]

There's still something different about the image issue in Flutter - my reduced test case performs much better memory-usage wise, but the bigger Flutter image example only reduces memory the first time the image goes away (subsequent times don't seem to work). I'll keep digging into that, but I suspect if anything some change in NotifyIdle would help us resolve the image issue in the engine side.