Unable to use Java AWS SDK v2 to get object using SHA256

[gorell]

Describe the bug

hi team,
after putting the object using the checksum algorithm SHA256 as following,

        PutObjectRequest putObjectRequest = PutObjectRequest.builder()
            .bucket(bucketName)
            .key(key)
            .contentType(contentType)
            .checksumAlgorithm(ChecksumAlgorithm.SHA256)
            .build();
        awsClient.putObject(putObjectRequest, RequestBody.fromString(content));

we're trying to get this object:

          GetObjectResponse getObjectResponse = awsClient.getObject(
                  b -> b
                      .bucket(bucketName)
                      .key(key)
                      .checksumMode(ChecksumMode.ENABLED))
              .response();

but MD5 checksum (and not SHA256) is applied instead, via this hard-coded reference this
and we're getting the exception: java.lang.IllegalStateException: Unexpected error creating MD5 checksum
note: headObject works fine
what are we missing?
thanks in advance

Expected Behavior

GetObject should work

Current Behavior

Getting an exception (see above)

Reproduction Steps

Please see the code in the bug description

Possible Solution

No response

Additional Information/Context

No response

AWS Java SDK version used

2.20.42

JDK version used

openjdk 11.0.14.1

Operating System and version

Linux service-5b7c8c4788-c62fp 5.4.238-148.347.amzn2.x86_64 #1 SMP Thu Apr 6 19:42:57 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

[gorell]

[debora-ito]

@gorell can you share the full stacktrace of the error?

[gorell]

@debora-ito here it is:

Caused by: java.lang.IllegalStateException: Unexpected error creating MD5 checksum
	at software.amazon.awssdk.core.checksums.Md5Checksum.getDigest(Md5Checksum.java:63)
	at software.amazon.awssdk.core.checksums.Md5Checksum.<init>(Md5Checksum.java:32)
	at software.amazon.awssdk.services.s3.internal.handlers.SyncChecksumValidationInterceptor.modifyHttpResponseContent(SyncChecksumValidationInterceptor.java:74)
	at software.amazon.awssdk.core.interceptor.ExecutionInterceptorChain.modifyHttpResponse(ExecutionInterceptorChain.java:148)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AfterTransmissionExecutionInterceptorsStage.execute(AfterTransmissionExecutionInterceptorsStage.java:46)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AfterTransmissionExecutionInterceptorsStage.execute(AfterTransmissionExecutionInterceptorsStage.java:28)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:73)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:50)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:36)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:81)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)
	at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:48)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:31)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
	at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:193)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:103)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:171)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$0(BaseSyncClientHandler.java:68)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:179)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:62)
	at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:52)
	at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:63)
	at software.amazon.awssdk.services.s3.DefaultS3Client.getObject(DefaultS3Client.java:4535)
	at software.amazon.awssdk.services.s3.S3Client.getObjectAsBytes(S3Client.java:8465)
	at .......doGetObject(....java:322)
	... 65 more
Caused by: java.security.NoSuchAlgorithmException: MD5 MessageDigest not available
	at java.base/sun.security.jca.GetInstance.getInstance(GetInstance.java:159)
	at java.base/java.security.MessageDigest.getInstance(MessageDigest.java:185)
	at software.amazon.awssdk.core.checksums.Md5Checksum.getDigest(Md5Checksum.java:61)

[gorell]

@debora-ito just to clarify - we've customized our JDK to use SHA256 (and not MD5 which we don't support) for checksum

[debora-ito]

The Java SDK will validate the MD5 checksum by default on GetObject. This is a client-side configuration, separate from the GetObject checksumMode option, and this is what the code in SyncChecksumValidationInterceptor is doing.

You can disable this default MD5 checksum validation in the S3 Client by setting a custom S3Configuration:

S3Client s3Client = S3Client.builder()
        .serviceConfiguration(S3Configuration.builder().checksumValidationEnabled(Boolean.FALSE).build())
        .region(Region.US_WEST_2)
        .build();

[gorell]

Got it @debora-ito , thank you. So is there a way to configure the GetObject validation to be SHA256? Given how the PutObject is configured.

[debora-ito]

In this use-case, the SDK will automatically validate the SHA256 checksum on GetObject, after finishing reading the object. The SDK validates any of the 4 checksum types supported by the ChecksumAlgorithm in PutObject.