DDB Enhanced: Allow custom versioning #6019

RanVaknin · 2025-04-09T22:19:34Z

Motivation and Context

This PR builds on the work started in issue #3894 where the VersionedRecordExtension didn't support starting version numbers at 0, requiring clients to use Integer objects (with null initial values) rather than long primitives. As mentioned by @akiesler in the original issue, developers might expect versions to start at 0 and increment from there, rather than having a special case where the value must be initialized to null.

This implementation allows the extension to be more flexible by allowing:

Starting versions at 0 (or any non-negative long)
Configuring custom increment values (only positive numbers)
Supporting these configs both at the extension level and via annotations

Modifications

Refactored the VersionedRecordExtension to support explicit startAt and incrementBy values via builder methods (making it opt-in).
Expanded the DynamoDbVersionAttribute annotation to support startAt and incrementBy parameters
Added validation to prevent negative startAt values and non-positive incrementBy values

Testing

Custom startAt and incrementBy values through both builder and annotations
Validation of illegal values (negative startAt, zero/negative incrementBy)
Precedence rules between annotation and builder configurations
Version incrementation for both initial and existing records
Edge cases with different configuration combinations

...tom/dynamodb-enhanced/src/main/java/software/amazon/awssdk/enhanced/dynamodb/Expression.java

.../main/java/software/amazon/awssdk/enhanced/dynamodb/extensions/VersionedRecordExtension.java

zoewangg · 2025-04-15T17:06:50Z

services-custom/dynamodb-enhanced/pom.xml

+                <configuration>
+                    <parameter>
+                        <excludes>
+                            <!-- japicmp flagging these incorrectly as a breaking change -->


Hmm, this is odd, can you paste the error logs? If we have to add exclusion, we should add it to the top level pom https://github.com/aws/aws-sdk-java-v2/blob/master/pom.xml#L681

the error only was shown on CI, but I was able to raise with running mvn japicmp:cmp -pl :dynamodb-enhanced

[INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 4.557 s [INFO] Finished at: 2025-04-21T11:29:14-07:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal com.github.siom79.japicmp:japicmp-maven-plugin:0.15.6:cmp (default-cli) on project dynamodb-enhanced: There is at least one incompatibility: software.amazon.awssdk.enhanced.dynamodb.extensions.annotations.DynamoDbVersionAttribute.incrementBy():METHOD_ABSTRACT_ADDED_TO_CLASS,software.amazon.awssdk.enhanced.dynamodb.extensions.annotations.DynamoDbVersionAttribute.startAt():METHOD_ABSTRACT_ADDED_TO_CLASS -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

.../main/java/software/amazon/awssdk/enhanced/dynamodb/extensions/VersionedRecordExtension.java

akiesler · 2025-04-15T20:52:49Z

Thank you for picking up this change! It will greatly simplify our version handling. Please let me know if I can do anything to help.

…tly set

…odepaths

sonarqubecloud · 2025-04-22T00:42:51Z

Quality Gate passed

Issues
7 New issues
0 Accepted issues

Measures
0 Security Hotspots
91.3% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

dagnir · 2025-04-29T20:29:49Z

.changes/next-release/feature-DynamoDBEnhancedClient-2a501d8.json

+    "type": "feature",
+    "category": "DynamoDB Enhanced Client",
+    "contributor": "akiesler",
+    "description": "DynamoDB Enhanced Client: Support for Version Starting at 0 with Configurable Increment"


Nit: Don't need the DynamoDB Enhanced Client: in the description since it's already a category

dagnir · 2025-04-29T20:30:27Z

pom.xml

@@ -679,6 +679,8 @@
                            <includeModule>polly</includeModule>
                        </includeModules>
                        <excludes>
+                            <exclude>software.amazon.awssdk.enhanced.dynamodb.extensions.annotations.DynamoDbVersionAttribute#incrementBy()</exclude>


Let's add a TODO and create a tracking task to remove this after release

dagnir · 2025-04-29T20:35:39Z

...tom/dynamodb-enhanced/src/main/java/software/amazon/awssdk/enhanced/dynamodb/Expression.java

+        return "Expression{" +
+               "expression='" + expression + '\'' +
+               ", expressionValues=" + expressionValues +
+               ", expressionNames=" + expressionNames +
+               '}';
+    }
+


Let's use ToString.builder() so it's consistent with the other toString methods in DDB enhanced:

aws-sdk-java-v2/services-custom/dynamodb-enhanced/src/main/java/software/amazon/awssdk/enhanced/dynamodb/internal/converter/attribute/EnhancedAttributeValue.java

Lines 736 to 740 in f6adeaa

return ToString.builder("EnhancedAttributeValue")

.add("type", type)

.add("value", value)

.build();

}

dagnir · 2025-04-29T20:41:25Z

.../main/java/software/amazon/awssdk/enhanced/dynamodb/extensions/VersionedRecordExtension.java

+        Validate.isNotNegativeOrNull(startAt, "startAt");
+
+        if (incrementBy != null && incrementBy < 1) {
+            throw new IllegalArgumentException("IncrementBy must be greater than 0.");


nit: technically should match the param name, so should be lowercase "i"

dagnir · 2025-04-29T20:46:02Z

.../main/java/software/amazon/awssdk/enhanced/dynamodb/extensions/VersionedRecordExtension.java

+        if (!existingVersionValue.isPresent() || isNullAttributeValue(existingVersionValue.get()) ||
+            (existingVersionValue.get().n() != null &&
+             ((versionStartAtFromAnnotation.isPresent() &&
+               Long.parseLong(existingVersionValue.get().n()) == versionStartAtFromAnnotation.get()) ||
+              Long.parseLong(existingVersionValue.get().n()) == this.startAt))) {


This is a very complex expression. Let's move this to a separate function so it's easier to reason about and we can document it

+1. Why do we need to check existingVersionValue.get().n() != null && ((versionStartAtFromAnnotation.isPresent() && Long.parseLong(existingVersionValue.get().n()) == versionStartAtFromAnnotation.get()) || Long.parseLong(existingVersionValue.get().n()) == this.startAt)))?

Suggesting the following

if (existingValueNotAvailable(existingAttributeValue)) { long startVersion = versionStartAtFromAnnotation.orElse(this.startAt); long incrementVersion = versionIncrementByFromAnnotation.orElse(this.incrementBy); ... } else { .... }

dagnir · 2025-04-29T21:02:31Z

.../main/java/software/amazon/awssdk/enhanced/dynamodb/extensions/VersionedRecordExtension.java

-            newVersionValue = AttributeValue.builder().n(Integer.toString(existingVersion + 1)).build();
+
+            long increment = versionIncrementByFromAnnotation.orElse(this.incrementBy);
+            newVersionValue = AttributeValue.builder().n(Long.toString(existingVersion + increment)).build();


It doesn't look like the original version handled this, but how do we deal with integer overflow? Originally incrementBy was always 1 so it's unlikely to happen, but with incrementBy being configurable now, it's technically more likely now if the user sets a relatively large value.

Can we check how v1's mapper handles this (if at all)?

zoewangg · 2025-04-29T22:59:29Z

.../main/java/software/amazon/awssdk/enhanced/dynamodb/extensions/VersionedRecordExtension.java

+        if (!existingVersionValue.isPresent() || isNullAttributeValue(existingVersionValue.get()) ||
+            (existingVersionValue.get().n() != null &&
+             ((versionStartAtFromAnnotation.isPresent() &&
+               Long.parseLong(existingVersionValue.get().n()) == versionStartAtFromAnnotation.get()) ||
+              Long.parseLong(existingVersionValue.get().n()) == this.startAt))) {


+1. Why do we need to check existingVersionValue.get().n() != null && ((versionStartAtFromAnnotation.isPresent() && Long.parseLong(existingVersionValue.get().n()) == versionStartAtFromAnnotation.get()) || Long.parseLong(existingVersionValue.get().n()) == this.startAt)))?

Suggesting the following

if (existingValueNotAvailable(existingAttributeValue)) { long startVersion = versionStartAtFromAnnotation.orElse(this.startAt); long incrementVersion = versionIncrementByFromAnnotation.orElse(this.incrementBy); ... } else { .... }

zoewangg · 2025-04-29T23:00:13Z

.../main/java/software/amazon/awssdk/enhanced/dynamodb/extensions/VersionedRecordExtension.java

@@ -144,11 +202,38 @@ public WriteModification beforeWrite(DynamoDbExtensionContext.BeforeWrite contex

    @NotThreadSafe
    public static final class Builder {
+        private Long startAt = 0L;


Is there any reason we are setting the default values in Builder? Builder class should not have any logic ideally and it also seems we are already setting the default values in the top level classes

zoewangg · 2025-04-29T23:03:00Z

...t/java/software/amazon/awssdk/enhanced/dynamodb/extensions/VersionedRecordExtensionTest.java

                                           .operationContext(PRIMARY_CONTEXT)
                                           .tableMetadata(FakeItem.getTableMetadata())
                                           .build());
    }
+
+    @Test
+    public void beforeWrite_versionEqualsStartAt_treatedAsInitialVersion() {


Can we add tests for different table schemas such as static table schema, immutable table scheme and bean table schema? https://github.com/aws/aws-sdk-java-v2/pull/6067/files#diff-9826429ca566bd1325f9ffa5d4fe1141cd26df01ec67262bab6f4acb12c1aa28

RanVaknin requested a review from a team as a code owner April 9, 2025 22:19

RanVaknin force-pushed the rvaknin/ddb-enhanced-client-versioned-record-custom-startAt branch from abcd87a to 4e82d7e Compare April 10, 2025 22:54

davidh44 reviewed Apr 15, 2025

View reviewed changes

zoewangg requested changes Apr 15, 2025

View reviewed changes

RanVaknin added 8 commits April 21, 2025 16:42

Adding annotation support

88d1198

Making annotation values nullable to know if initial value is explici…

6d44364

…tly set

Added changelog with attribution

f0255ff

Fix checkstyle

db487aa

Adding japicmp excludes block

d87025f

Adding additional test coverage to account for all isInitialVersion c…

3036143

…odepaths

Removing logger config file

e8a6075

porting version from Integer to Long, removing refactoring

3d505a6

RanVaknin force-pushed the rvaknin/ddb-enhanced-client-versioned-record-custom-startAt branch from 9e39a83 to 3d505a6 Compare April 21, 2025 23:42

dagnir requested changes Apr 29, 2025

View reviewed changes

zoewangg requested changes Apr 29, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DDB Enhanced: Allow custom versioning #6019

DDB Enhanced: Allow custom versioning #6019

RanVaknin commented Apr 9, 2025 •

edited

Loading

zoewangg Apr 15, 2025

RanVaknin Apr 21, 2025

akiesler commented Apr 15, 2025

sonarqubecloud bot commented Apr 22, 2025

dagnir Apr 29, 2025

dagnir Apr 29, 2025

dagnir Apr 29, 2025

dagnir Apr 29, 2025

dagnir Apr 29, 2025

zoewangg Apr 29, 2025

dagnir Apr 29, 2025

zoewangg Apr 29, 2025

zoewangg Apr 29, 2025

zoewangg Apr 29, 2025

	return ToString.builder("EnhancedAttributeValue")
	.add("type", type)
	.add("value", value)
	.build();
	}

DDB Enhanced: Allow custom versioning #6019

Are you sure you want to change the base?

DDB Enhanced: Allow custom versioning #6019

Conversation

RanVaknin commented Apr 9, 2025 • edited Loading

Motivation and Context

Modifications

Testing

Choose a reason for hiding this comment

Choose a reason for hiding this comment

akiesler commented Apr 15, 2025

sonarqubecloud bot commented Apr 22, 2025

Quality Gate passed

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RanVaknin commented Apr 9, 2025 •

edited

Loading