Coding Moments

Execution order of SQL statements in JPA / Hibernate

Nilesh Wani's photo
Nilesh Wani
·

4 min read

In my project, I was having a database table with a unique constraint on a column. Using Spring Data JPA, in the same transaction, I was first deleting a record from that table and then adding back a record with the same value for that unique column. Surprisingly, the database threw a unique constraint violation exception.

In this blog, we will look at this problem and solutions to solve it.

Demo application

Imagine that we want to build REST APIs to manage shapes and their properties. We can have different shapes, and each shape can have different and multiple properties. For example, a circle can have color and radius.

We use below database tables to store shapes and properties.

CREATE TABLE shape (
  shape_id INT AUTO_INCREMENT PRIMARY KEY,
  name     VARCHAR2(50) NOT NULL
);

CREATE TABLE property (
  property_id     INT AUTO_INCREMENT PRIMARY KEY,
  shape_id        INT NOT NULL,
  property_key    VARCHAR2(50) NOT NULL,
  property_value  VARCHAR2(50) NOT NULL,
  FOREIGN KEY ( shape_id ) REFERENCES shape ( shape_id ),
  UNIQUE (shape_id, property_key)  
);

Note that the table PROPERTY has a unique constraint on the columns shape_id and property_key.

You can find the source code in a GitHub repository here. The application uses an in-memory H2 database. You can clone this repository and start the application.

Let's create a shape circle along with its properties color blue and radius 10, by sending an HTTP POST request to the endpoint /shapes with below JSON request body.

{
  "name": "circle",
  "properties": [
    {
      "propertyKey": "color",
      "propertyValue": "blue"
    },
    {
      "propertyKey": "radius",
      "propertyValue": 10
    }
  ]
}

Problem

Let's update the properties of this shape by sending an HTTP PUT request to /shapes/{shape_id}/properties endpoint with the below request body.

[
  {
    "propertyKey": "color",
    "propertyValue": "red"
  },
  {
    "propertyKey": "border",
    "propertyValue": 20
  }
]

The application throws the unique constraint violation exception.

org.h2.jdbc.JdbcSQLIntegrityConstraintViolationException: Unique index or primary key violation: "PUBLIC.CONSTRAINT_F37_INDEX_6 ON PUBLIC.PROPERTY(SHAPE_ID NULLS FIRST, PROPERTY_KEY NULLS FIRST) VALUES ( /* key:1 */ CAST(1 AS BIGINT), 'color')"; SQL statement:
insert into property (property_key,property_value,shape_id,property_id) values (?,?,?,default) [23505-214]
    at org.h2.message.DbException.getJdbcSQLException(DbException.java:508) ~[h2-2.1.214.jar:2.1.214]

Below is the code that updates the shape with its properties.

private List<Property> deleteAndAddProperties(Shape dbShape, List<Property> properties) {
  // Deleting all the existing properties for the shape
  dbShape.getProperties().forEach(prop -> propertyRepository.deleteById(prop.getPropertyId()));

  // Inserting new properties for the shape
  properties.forEach(prop -> {
    prop.setShapeId(dbShape.getShapeId());
    propertyRepository.save(prop);
    dbShape.getProperties().add(prop);
  });

  return properties;
}

We expect that the unique constraint should not be violated because we are first deleting the old records and then adding the new records.

Cause

When you perform SQL operations using JPA, those operations are not immediately executed on the database side. Instead, they are recorded within the persistence context as pending changes.

When the transaction is committed, all the pending changes recorded in the persistence context are sent (flushed) to the database and executed as a batch of SQL statements. This is known as the "write-behind" mechanism.

The SQL operations are queued in the ActionQueue class which also specifies the order of execution of those operations.

  • OrphanRemovalAction

  • AbstractEntityInsertAction

  • EntityUpdateAction

  • QueuedOperationCollectionAction

  • CollectionRemoveAction

  • CollectionUpdateAction

  • CollectionRecreateAction

  • EntityDeleteAction

You can notice that the DELETE statements are executed at the end of the transaction while the INSERT statements are executed in the beginning.

In our example, even though we first deleted the records and then inserted the records, Hibernate first executes the insert operations in the database. The database sees that there are already records with the same keys and so throws a unique constraint violation exception.

Solution

One way to work around this issue is to manually flush the persistence context after the DELETE operation. It synchronizes the persistence context to the underlying database, meaning it will execute the SQL queries for all the changes recorded in the persistence context on the database side.

private List<Property> deleteFlushAndAddProperties(Shape dbShape, List<Property> properties) {
  // Deleting all the existing properties for the shape
  dbShape.getProperties().forEach(prop -> propertyRepository.deleteById(prop.getPropertyId()));

  entityManager.flush();

  // Inserting new properties for the shape
  properties.forEach(prop -> {
    prop.setShapeId(dbShape.getShapeId());
    propertyRepository.save(prop);
    dbShape.getProperties().add(prop);
  });

  return properties;
}

The better solution is to update the records that already exist in the database instead of removing and adding them again.

private List<Property> mergeProperties(Shape dbShape, List<Property> properties) {
  Map<String, Property> newKeyToPropertyMap = properties.stream().collect(Collectors.toMap(Property::getPropertyKey, Function.identity()));
  Map<String, Property> existingKeyToPropertyMap = dbShape.getProperties().stream().collect(Collectors.toMap(Property::getPropertyKey, Function.identity()));

  List<Property> resultProperties = new ArrayList<>();

  // Deleting the properties that are not present in the request body
  dbShape.getProperties().stream().filter(prop -> !newKeyToPropertyMap.containsKey(prop.getPropertyKey())).forEach(prop -> propertyRepository.deleteById(prop.getPropertyId()));
  // Updating the existing properties present in the request body
  dbShape.getProperties().stream().filter(prop -> newKeyToPropertyMap.containsKey(prop.getPropertyKey())).forEach(prop -> {
    Property newProperty = newKeyToPropertyMap.get(prop.getPropertyKey());
    prop.setPropertyValue(newProperty.getPropertyValue());
    resultProperties.add(prop);
  });
  // Inserting the properties not present in database
  properties.stream().filter(prop -> !existingKeyToPropertyMap.containsKey(prop.getPropertyKey())).forEach(prop -> {
    prop.setShapeId(dbShape.getShapeId());
    propertyRepository.save(prop);
    resultProperties.add(prop);
  });

  return resultProperties;
}

This solution also avoids the overhead of reindexing the corresponding database records as we are updating the existing records with new values instead of deleting-inserting.

hibernatejpaorder of SQL execution