Skip to content

Changes on the document entities persistence operations at the Eclipse JNoSQL DynamoDB Database #330

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 6, 2025
Merged

Changes on the document entities persistence operations at the Eclipse JNoSQL DynamoDB Database #330

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1046602
chore: Changes on the document entities persistence operations at the…
dearrudam Jul 3, 2025
584682a
chore: implemented entitiesMetadata() method in the subclasses of Bas…
dearrudam Jul 3, 2025
d5c85a9
chore: removed https://oss.sonatype.org/content/repositories/snapshot…
dearrudam Jul 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,11 @@ and this project adheres to https://semver.org/spec/v2.0.0.html[Semantic Version

- Update query at Oracle NoSQL to support parameter with enum type

=== Changes

- Changes on the document entities persistence operations at the Eclipse JNoSQL DynamoDB Database


== [1.1.8] - 2025-05-21

=== Changed
Expand Down
63 changes: 38 additions & 25 deletions README.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -597,27 +597,6 @@ jnosql.keyvalue.database=heroes

=== Using the Document API

The DynamoDB's Document API implementation follows the *SINGLE TABLE* strategy, it means, the table will store multiple entity types. To satisfy this strategy, the implementation assumes that the target table will have a composed primary key:

- The `entityType` field as the partitioning key;
- The `id` field as the sort key;

To customize the partitioning key field name, you can define the following configuration

[source,properties]
----
jnosql.dynamodb.entity.pk=entityType
----

By default, the implementation doesn't create the table on-the-fly, letting this requirement for the users. If you prefer, the implementation is able to create the table on-the-fly as well. To activate this capability you should define explicitly the following configuration:

[source,properties]
----
jnosql.dynamodb.create.tables=true
----

The table will be created with the composed primary key mentioned previously.

Here's an example using DynamoDB's Document API with MicroProfile Config.

[source,properties]
Expand Down Expand Up @@ -648,7 +627,7 @@ public class ManagerSupplier implements Supplier<DynamoDBDocumentManager> {

=== Repository

The ```DynamoDBRepository``` interface is an extension of the ```Repository``` interface that allows execution of PartiQL via the ```@PartiQL``` annotation.
The `DynamoDBRepository` interface is an extension of the `Repository` interface that allows execution of PartiQL via the `@PartiQL` annotation.

WARNING: DynamoDB supports a limited subset of
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/ql-reference.html[PartiQL].
Expand All @@ -669,10 +648,9 @@ List<Person> findByName(@Param("") String name);
}
----


=== Template

The ```DynamoDBTemplate``` interface is a specialization of the ```DocumentTemplate``` interface that allows using PartiQL queries.
The `DynamoDBTemplate` interface is a specialization of the `DocumentTemplate` interface that allows using PartiQL queries.

WARNING: DynamoDB supports a limited subset of
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/ql-reference.html[PartiQL].
Expand All @@ -681,9 +659,44 @@ NOTE: This implementation doesn't provide pagination on the queries.

[source,java]
----
List<Person> people = template.partiQL("select * from Person where name = ? ", params);
List<Person> people = template.partiQL("select * from Person where name = ? ", Person.class, params);
----

==== Creating the tables on-the-fly

[IMPORTANT]
====
It's highly recommended to create the tables in a proper way, paying attention to the partition key and sort key, as well as the indexes.
====

The DynamoDB implementation allows you to create tables on-the-fly, which can be useful for development and testing purposes. However, this feature should be used with caution in production environments, as it may lead to unexpected behavior or performance issues if not properly configured.

To create tables on-the-fly, you need to define the following properties:

Please note that you can establish properties using the https://microprofile.io/microprofile-config/[MicroProfile Config] specification.

[cols="DynamoDB"]
|===
|Configuration property |Description | Default value

|`jnosql.dynamodb.create.tables`
| If set to true, the implementation will create the tables on-the-fly when the application starts. This is useful for development and testing purposes, but should be used with caution in production environments.
| false

|`jnosql.dynamodb.<table>.pk`
| The partition key field name for the table. This is used to define the primary key of the table. The `<table>` part should be replaced with the actual table name.
| _id

|`jnosql.dynamodb.<table>.read.capacity.units`
| The read capacity units for the table. This defines the number of strongly consistent reads per second that the table can support.The `<table>` part should be replaced with the actual table name. It's optional.
| none

|`jnosql.dynamodb.<table>.write.capacity.units`
| The write capacity units for the table. This defines the number of strongly consistent writes per second that the table can support.The `<table>` part should be replaced with the actual table name. It's optional.
| none

|===

== Elasticsearch

image::https://jnosql.github.io/img/logos/elastic.svg[Elasticsearch Project,align="center"width=25%,height=25%]
Expand Down
8 changes: 8 additions & 0 deletions ...n/java/org/eclipse/jnosql/databases/arangodb/mapping/ArangoDBDocumentRepositoryProxy.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,8 @@ class ArangoDBDocumentRepositoryProxy<T, K> extends AbstractSemiStructuredReposi

private final EntityMetadata entityMetadata;

private final EntitiesMetadata entitiesMetadata;

ArangoDBDocumentRepositoryProxy(ArangoDBTemplate template,
Class<?> type,
Converters converters,
Expand All @@ -57,6 +59,7 @@ class ArangoDBDocumentRepositoryProxy<T, K> extends AbstractSemiStructuredReposi
.getActualTypeArguments()[0]);
this.type = type;
this.converters = converters;
this.entitiesMetadata = entitiesMetadata;
this.entityMetadata = entitiesMetadata.get(typeClass);
this.repository = SemiStructuredRepositoryProxy.SemiStructuredRepository.of(template, entityMetadata);
}
Expand All @@ -82,6 +85,11 @@ protected EntityMetadata entityMetadata() {
return entityMetadata;
}

@Override
protected EntitiesMetadata entitiesMetadata() {
return this.entitiesMetadata;
}

@Override
protected DocumentTemplate template() {
return template;
Expand Down
8 changes: 8 additions & 0 deletions ...rc/main/java/org/eclipse/jnosql/databases/cassandra/mapping/CassandraRepositoryProxy.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@ class CassandraRepositoryProxy<T, K> extends AbstractSemiStructuredRepositoryPro

private final Converters converters;

private final EntitiesMetadata entitiesMetadata;

private final EntityMetadata entityMetadata;

private final Class<?> repositoryType;
Expand All @@ -55,6 +57,7 @@ class CassandraRepositoryProxy<T, K> extends AbstractSemiStructuredRepositoryPro
.getActualTypeArguments()[0]);

this.converters = converters;
this.entitiesMetadata = entitiesMetadata;
this.entityMetadata = entitiesMetadata.get(typeClass);
this.repositoryType = repositoryType;
this.repository = SemiStructuredRepositoryProxy.SemiStructuredRepository.of(template, entityMetadata);
Expand All @@ -80,6 +83,11 @@ protected EntityMetadata entityMetadata() {
return entityMetadata;
}

@Override
protected EntitiesMetadata entitiesMetadata() {
return this.entitiesMetadata;
}

@Override
protected ColumnTemplate template() {
return template;
Expand Down
8 changes: 8 additions & 0 deletions ...java/org/eclipse/jnosql/databases/couchbase/mapping/CouchbaseDocumentRepositoryProxy.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@ class CouchbaseDocumentRepositoryProxy<T, K> extends AbstractSemiStructuredRepos

private final Converters converters;

private final EntitiesMetadata entitiesMetadata;

private final EntityMetadata entityMetadata;

private final Class<?> repositoryType;
Expand All @@ -54,6 +56,7 @@ class CouchbaseDocumentRepositoryProxy<T, K> extends AbstractSemiStructuredRepos
this.typeClass = Class.class.cast(ParameterizedType.class.cast(repositoryType.getGenericInterfaces()[0])
.getActualTypeArguments()[0]);
this.converters = converters;
this.entitiesMetadata = entitiesMetadata;
this.entityMetadata = entitiesMetadata.get(typeClass);
this.repositoryType = repositoryType;
this.repository = SemiStructuredRepositoryProxy.SemiStructuredRepository.of(template, entityMetadata);
Expand All @@ -80,6 +83,11 @@ protected EntityMetadata entityMetadata() {
return entityMetadata;
}

@Override
protected EntitiesMetadata entitiesMetadata() {
return entitiesMetadata;
}

@Override
protected DocumentTemplate template() {
return template;
Expand Down
72 changes: 40 additions & 32 deletions ...a/org/eclipse/jnosql/databases/dynamodb/communication/DefaultDynamoDBDatabaseManager.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,6 @@
import java.util.stream.StreamSupport;

import static java.util.Objects.requireNonNull;
import static org.eclipse.jnosql.databases.dynamodb.communication.DynamoDBConverter.entityAttributeName;
import static org.eclipse.jnosql.databases.dynamodb.communication.DynamoDBConverter.toAttributeValue;
import static org.eclipse.jnosql.databases.dynamodb.communication.DynamoDBConverter.toCommunicationEntity;
import static org.eclipse.jnosql.databases.dynamodb.communication.DynamoDBConverter.toItem;
Expand All @@ -81,10 +80,6 @@ public DefaultDynamoDBDatabaseManager(String database, DynamoDbClient dynamoDbCl
this.dynamoDbClient = dynamoDbClient;
}

private String resolveEntityNameAttributeName(String entityName) {
return this.settings.get(DynamoDBConfigurations.ENTITY_PARTITION_KEY, String.class).orElse(entityName);
}

public DynamoDbClient dynamoDbClient() {
return dynamoDbClient;
}
Expand All @@ -99,7 +94,7 @@ public CommunicationEntity insert(CommunicationEntity documentEntity) {
requireNonNull(documentEntity, "documentEntity is required");
dynamoDbClient().putItem(PutItemRequest.builder()
.tableName(createTableIfNeeded(documentEntity.name()).table().tableName())
.item(toItem(this::resolveEntityNameAttributeName, documentEntity))
.item(toItem(documentEntity))
.build());
return documentEntity;
}
Expand Down Expand Up @@ -141,11 +136,12 @@ private DescribeTableResponse getDescribeTableResponse(String tableName) {

private DescribeTableResponse createTable(String tableName) {
try (var waiter = dynamoDbClient().waiter()) {

dynamoDbClient().createTable(CreateTableRequest.builder()
.tableName(tableName)
.keySchema(defaultKeySchemaFor())
.attributeDefinitions(defaultAttributeDefinitionsFor())
.provisionedThroughput(defaultProvisionedThroughputFor())
.keySchema(defaultKeySchemaFor(tableName))
.attributeDefinitions(defaultAttributeDefinitionsFor(tableName))
.provisionedThroughput(defaultProvisionedThroughputFor(tableName))
.build());

var tableRequest = DescribeTableRequest.builder().tableName(tableName).build();
Expand All @@ -154,34 +150,40 @@ private DescribeTableResponse createTable(String tableName) {
}
}

private ProvisionedThroughput defaultProvisionedThroughputFor() {
return DynamoTableUtils.createProvisionedThroughput(null, null);
private ProvisionedThroughput defaultProvisionedThroughputFor(String tableName) {
return DynamoTableUtils.createProvisionedThroughput(
this.settings.get(DynamoDBConfigurations.ENTITY_READ_CAPACITY_UNITS.get().formatted(tableName), Long.class)
.orElse(null),
this.settings.get(DynamoDBConfigurations.ENTITY_WRITE_CAPACITY_UNITS.get().formatted(tableName), Long.class)
.orElse(null));
}

private Collection<AttributeDefinition> defaultAttributeDefinitionsFor() {
private Collection<AttributeDefinition> defaultAttributeDefinitionsFor(String tableName) {
return List.of(
AttributeDefinition.builder().attributeName(getEntityAttributeName()).attributeType(ScalarAttributeType.S).build(),
AttributeDefinition.builder().attributeName(DynamoDBConverter.ID).attributeType(ScalarAttributeType.S).build()
AttributeDefinition.builder()
.attributeName(partitionKeyName(tableName, DynamoDBConverter.ID)).attributeType(ScalarAttributeType.S).build()
);
}

private Collection<KeySchemaElement> defaultKeySchemaFor() {
private Collection<KeySchemaElement> defaultKeySchemaFor(String tableName) {
return List.of(
KeySchemaElement.builder().attributeName(getEntityAttributeName()).keyType(KeyType.HASH).build(),
KeySchemaElement.builder().attributeName(DynamoDBConverter.ID).keyType(KeyType.RANGE).build()
KeySchemaElement.builder()
.attributeName(partitionKeyName(tableName, DynamoDBConverter.ID)).keyType(KeyType.HASH).build()
);
}

private String partitionKeyName(String table, String defaultName) {
return this.settings
.get(DynamoDBConfigurations.ENTITY_PARTITION_KEY.name().formatted(table), String.class)
.orElse(defaultName);
}

private boolean shouldCreateTables() {
return this.settings
.get(DynamoDBConfigurations.CREATE_TABLES, Boolean.class)
.orElse(false);
}

private String getEntityAttributeName() {
return entityAttributeName(this::resolveEntityNameAttributeName);
}

@Override
public CommunicationEntity insert(CommunicationEntity documentEntity, Duration ttl) {
requireNonNull(documentEntity, "documentEntity is required");
Expand Down Expand Up @@ -233,12 +235,11 @@ private Map<String, AttributeValue> getItemKey(CommunicationEntity documentEntit
a.putAll(b);
return a;
});
itemKey.put(getEntityAttributeName(), toAttributeValue(documentEntity.name()));
return itemKey;
}

private Map<String, AttributeValueUpdate> asItemToUpdate(CommunicationEntity documentEntity) {
return toItemUpdate(this::resolveEntityNameAttributeName, documentEntity);
return toItemUpdate(documentEntity);
}

@Override
Expand Down Expand Up @@ -278,22 +279,29 @@ public void delete(DeleteQuery deleteQuery) {
public Stream<CommunicationEntity> select(SelectQuery query) {
Objects.requireNonNull(query, "query is required");
DynamoDBQuery dynamoDBQuery = DynamoDBQuery
.builderOf(query.name(), getEntityAttributeName(), query)
.builderOf(query.name(), query)
.get();

ScanRequest.Builder selectRequest = ScanRequest.builder()
.consistentRead(true)
.tableName(dynamoDBQuery.table())
.tableName(createTableIfNeeded(dynamoDBQuery.table()).table().tableName())
.projectionExpression(dynamoDBQuery.projectionExpression())
.filterExpression(dynamoDBQuery.filterExpression())
.expressionAttributeNames(dynamoDBQuery.expressionAttributeNames())
.expressionAttributeValues(dynamoDBQuery.expressionAttributeValues())
.select(dynamoDBQuery.projectionExpression() != null ? Select.SPECIFIC_ATTRIBUTES : Select.ALL_ATTRIBUTES);

if (!dynamoDBQuery.filterExpression().isBlank()) {
selectRequest = selectRequest.filterExpression(dynamoDBQuery.filterExpression());
}

if (!dynamoDBQuery.expressionAttributeNames().isEmpty()) {
selectRequest = selectRequest
.expressionAttributeNames(dynamoDBQuery.expressionAttributeNames())
.expressionAttributeValues(dynamoDBQuery.expressionAttributeValues());
}

return StreamSupport
.stream(dynamoDbClient().scanPaginator(selectRequest.build()).spliterator(), false)
.flatMap(scanResponse -> scanResponse.items().stream()
.map(item -> toCommunicationEntity(this::resolveEntityNameAttributeName, item)));
.map(item -> toCommunicationEntity(dynamoDBQuery.table(), item)));
}

@Override
Expand All @@ -314,7 +322,7 @@ public void close() {
}

@Override
public Stream<CommunicationEntity> partiQL(String query, Object... params) {
public Stream<CommunicationEntity> partiQL(String query, String entityName, Object... params) {
Objects.requireNonNull(query, "query is required");
List<AttributeValue> parameters = Stream.of(params).map(DynamoDBConverter::toAttributeValue).toList();
ExecuteStatementResponse executeStatementResponse = dynamoDbClient()
Expand All @@ -323,15 +331,15 @@ public Stream<CommunicationEntity> partiQL(String query, Object... params) {
.parameters(parameters)
.build());
List<CommunicationEntity> result = new LinkedList<>();
executeStatementResponse.items().forEach(item -> result.add(toCommunicationEntity(this::resolveEntityNameAttributeName, item)));
executeStatementResponse.items().forEach(item -> result.add(toCommunicationEntity(entityName, item)));
while (executeStatementResponse.nextToken() != null) {
executeStatementResponse = dynamoDbClient()
.executeStatement(ExecuteStatementRequest.builder()
.statement(query)
.parameters(parameters)
.nextToken(executeStatementResponse.nextToken())
.build());
executeStatementResponse.items().forEach(item -> result.add(toCommunicationEntity(this::resolveEntityNameAttributeName, item)));
executeStatementResponse.items().forEach(item -> result.add(toCommunicationEntity(entityName, item)));
}
return result.stream();
}
Expand Down
7 changes: 5 additions & 2 deletions ...main/java/org/eclipse/jnosql/databases/dynamodb/communication/DynamoDBConfigurations.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,11 @@ public enum DynamoDBConfigurations implements Supplier<String> {
PROFILE("jnosql.dynamodb.profile"),
AWS_ACCESSKEY("jnosql.dynamodb.awsaccesskey"),
AWS_SECRET_ACCESS("jnosql.dynamodb.secretaccess"),
ENTITY_PARTITION_KEY("jnosql.dynamodb.entity.pk"),
CREATE_TABLES("jnosql.dynamodb.create.tables");
CREATE_TABLES("jnosql.dynamodb.create.tables"),
ENTITY_PARTITION_KEY("jnosql.dynamodb.%s.pk"),
ENTITY_READ_CAPACITY_UNITS("jnosql.dynamodb.%s.read.capacity.units"),
ENTITY_WRITE_CAPACITY_UNITS("jnosql.dynamodb.%s.write.capacity.units"),
;

private final String configuration;

Expand Down
Loading