Introduction
ElasticSearch has a concurrency control capability that enables writes to use a version value to determine whether to apply an update or discard it as being stale in comparison to some existing data.
As part of considering a migration away from ElasticSearch as a data store, I was interested in how other databases could be made to achieve the same type of version aware upsert capability.
In some earlier posts on this blog I have shared how the version aware upserting can be done with PostgreSQL, MariaDB and TitanDB.
This post is to share how the same capability can be achieved with AWS's DynamoDB document database, as an example of a non-relational database.
What does the DynamoDB API offer?
Insert, or update
DynamoDB has putItem for creating an item in a DynamoDB table, and updateItem for updating an existing item.
On first look, we might expect some combination of putItem and updateItem to need to be applied, as that would resemble how the relational databases had to detect conflict and fall back to attempt the second type of operation.
It turns out that we can just use putItem, as the API documentation states:
"If an item with the same key already exists in the table, it is replaced with the new item."
So, that takes care of the insert otherwise update aspect of the implementation, but what about version awareness?
Conditional updating
The putItem API offers us the option of specifying some conditional logic that includes the ability to compare existing data against the data being sent.
If we have a table called event, containing an id and a version then we can have a call like the following:
String idAsString = "event-123";
String version = "456";
Map<String, AttributeValue> eventDataUpdating = Map.of(
"id", AttributeValue.builder().s(idAsString).build(),
"version", AttributeValue.builder().n(version).build());
dbClient.putItem(
PutItemRequest.builder().tableName("event")
.item(eventDataUpdating)
.conditionExpression("attribute_not_exists(id) OR (version < :version)")
.expressionAttributeValues(Map.of(":version", AttributeValue.builder().n(version).build()))
.returnValuesOnConditionCheckFailure(ReturnValuesOnConditionCheckFailure.ALL_OLD)
.build());
In that code, conditionExpression and expressionAttributeValues combine to express the two situations that determine whether the content should be written into the event table:
- attribute_not_exists covers the insert case, as there is no existing record with the specified id;
- version < :version covers the situation where an existing item exists but has a lower version value than what is being provided now.
Try it out
I've been experimenting in Java, using the Localstack Docker container as a standalone environment for interacting with DynamoDB, so you can grab the code and try running it for yourself.
At the time of this post, it is just a single class that:
- creates the table
- writes an initial low version
- sets up 100 randomly ordered version values
- spins up virtual threads that each pick up one of the version values and concurrently attempt to apply the update using the condition check
- prints out when a conflict has prevented an attempted update (as expected)
- verifies that when the dust has settled we ultimately end up with the highest version being written
(You'll need Java 21 or later, Maven, and Docker).
Disclaimer
So far I have only scratched the surface of how to achieve the desired functionality.
I would not recommend applying this approach without also diving deep into the documentation for further layers of potential limitations and situations where eventual consistency may make this less appropriate than it appears.
Comments
Post a Comment