potential memory leak in aws sdk s3 code

[gorell]

Describe the bug

Hi Team. We run our code (that reads-from/writes-to s3) over time. We use v2.20.42. When taking heap dumps we observe consistent memory growth, like this:

-rw-r--r--  1 user  staff  394378864 Apr 18 13:54 /Users/user/heap1.bin
-rw-r--r--  1 user  staff  402753779 Apr 18 14:05 /Users/user/heap2.bin
-rw-r--r--  1 user  staff  409832891 Apr 18 14:11 /Users/user/heap3.bin
-rw-r--r--  1 user  staff  415905366 Apr 18 14:23 /Users/user/heap4.bin

When comparing heap dumps we can see that the majority of the new objects on the heap seem to be related to AWS SDK S3 library, almost solely reflecting the above memory growth. Screenshots attached. Code attached. Please your advice on the above. Thanks in advance.
aws-dal.zip

Expected Behavior

The memory dumps should remain consistent over time, no leaks should occur.

Current Behavior

The memory grows on a regular basis due to AWS SDK S3 library objects being created on the heap and not released.

Reproduction Steps

Run the attached code over time.

Possible Solution

No response

Additional Information/Context

No response

AWS Java SDK version used

2.20.42

JDK version used

openjdk 11.0.14.1

Operating System and version

Linux service-5b7c8c4788-c62fp 5.4.238-148.347.amzn2.x86_64 #1 SMP Thu Apr 6 19:42:57 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

[debora-ito]

@gorell memory leaks are usually related to some bad practices with resource management while using the SDK.

• Are you creating a new client for every request or are you reusing the same client? We recommend the latter.
• If you are calling data stream operations like getObject, are you closing the input stream after the operation is done? We recommend closing all streams as soon as possible.

[lamis31]

@debora-ito
Yes, we create a new client for every request, is this can cause a memory leak?
Can you provide the api or code example we should use to close the input stream?
Thanks

[debora-ito]

Yes, we create a new client for every request, is this can cause a memory leak?

Yes, this is most likely the cause of the memory leak. Try changing to a shared instance of the SDK client, and remember to close the client when it is not needed anymore.

Can you provide the api or code example we should use to close the input stream?

I meant the input stream (or any data type) used to read the data from a streaming operation. For example, if you use ResponseInputStream to retrieve data from S3Client#getObject, call ResponseInputStream#close as soon as you're done reading the stream.

We have a list of best practices in the Developer Guide -
https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/best-practices.html

[lamis31]

ok thanks, will try the suggested changes