English | Site Directory

Transactions

The App Engine datastore supports transactions. A transaction is an operation or set of operations that either succeeds completely, or fails completely. An application can perform multiple operations in a single transaction using Python function objects and the db.run_in_transaction() function.

Using Transactions

A transaction is a datastore operation or a set of datastore operations that either succeed completely, or fail completely. If the transaction succeeds, then all of its intended effects are applied to the datastore. If the transaction fails, then none of the effects are applied.

Every datastore write operation is atomic. A put() or delete() either happens, or it doesn't. An operation may fail due to a high rate of contention, with too many users trying to modify an entity at the same time. Or an operation may fail due to the application reaching a quota limit. Or there may be an internal error with the datastore. In all cases, the operation's effects are not applied, and the datastore API raises an exception.

An application can execute a set of statements and datastore operations in a single transaction, such that if any statement or operation raises an exception, none of the datastore operations in the set are applied. The application defines the actions to perform in the transaction using a Python function, then calls db.run_in_transaction() with the function as an argument:

from google.appengine.ext import db

class Accumulator(db.Model):
  counter = db.IntegerProperty()

def increment_counter(key, amount):
  obj = db.get(key)
  obj.counter += amount
  obj.put()

q = db.GqlQuery("SELECT * FROM Accumulator")
acc = q.get()

db.run_in_transaction(increment_counter, acc.key(), 5)

db.run_in_transaction() takes the function object, and positional and keyword arguments to pass to the function. If the function returns a value, db.run_in_transaction() will return the value.

If the function returns, the transaction is committed, and all effects of datastore operations are applied. If the function raises an exception, the transaction is "rolled back," and the effects are not applied.

If the function raises the Rollback exception, db.run_in_transaction() returns None. For any other exception, db.run_in_transaction() re-raises the exception.

What Can Be Done In a Transaction

The datastore imposes several restrictions on what can be done inside a single transaction.

All datastore operations in a transaction must operate on entities in the same entity group. This includes db.get(), put() and delete(). Notice that each root entity belongs to a separate entity group, so a single transaction cannot create or operate on more than one root entity. For an explanation of entity groups, see Keys and Entity Groups.

A transaction cannot perform queries using Query or GqlQuery. However, a transaction can retrieve datastore entities using keys and db.get(). Keys can be passed to the transaction function, or built inside the function with key names or IDs and Key.from_path(), Model.get_by_key_name() or Model.get_by_id().

An application cannot create or update an entity more than once in a single transaction.

Note: As of this writing, there is a issue that prevents the creation of a new root entity and descendants of the entity in a single transaction. Until this issue is fixed, a root entity must be created in a separate transaction from its descendants.

All other Python code is allowed inside a transaction function. The transaction function should not have side effects other than the datastore operations. The transaction function may be called multiple times if a datastore operation fails due to another user updating entities in the entity group at the same time. When this happens, the datastore API retries the transaction a fixed number of times. If they all fail, db.run_in_transaction() raises a TransactionFailedError.

Similarly, the transaction function should not have side effects that depend on the success of the transaction, unless the code that calls the transaction function knows to undo those effects. For example, if the transaction stores a new datastore entity, saves the created entity's ID for later use, then the transaction fails, the saved ID does not refer to the intended entity because the entity's creation was rolled back. The calling code would have to be careful not to use the saved ID in this case.

Uses For Transactions

The example above demonstrates one use of transactions: updating an entity with a new property value relative to its current value.

def increment_counter(key, amount):
  obj = db.get(key)
  obj.counter += amount
  obj.put()

This requires a transaction because the value may be updated by another user after this user's request calls db.get(key) but before it calls obj.put(). Without a transaction, the user's request will use the value of obj.counter prior to the update, and obj.put() will overwrite the update. With a transaction, the entity is guaranteed not to change between the two calls. If the entity is updated during the transaction, then the transaction is retried until all steps are completed without interruption.

Another common use for transactions is to update an entity with a named key, or create it if it doesn't yet exist:

class SalesAccount(db.Model):
  address = db.PostalAddressProperty()
  phone_number = db.PhoneNumberProperty()

def create_or_update(parent_obj, account_id, address, phone_number):
  obj = db.get(Key.from_path("SalesAccount", account_id, parent=parent_obj))
  if not obj:
    obj = SalesAccount(parent=parent_obj,
                       address=address,
                       phone_number=phone_number)
  else:
    obj.address = address
    obj.phone_number = phone_number

  obj.put()

As before, a transaction is necessary to handle the case where another user is attempting to create or update an entity with the same account_id. Without a transaction, if the entity does not exist and two users attempt to create it, the second will fail. With a transaction, the second attempt will retry, notice that the entity now exists, and update the entity instead.

Create-or-update is so useful that there is a built-in method for it: Model.get_or_insert() takes a key name, an optional parent, and arguments to pass to the model constructor if an entity of that name and path does not exist. The get attempt and the create happen in one transaction, so (if the transaction is successful) the method always returns a model instance that represents an actual entity.

Tip: A transaction should happen as quickly as possible to reduce the likelihood that the entities used by the transaction will change, requiring the transaction be retried. As much as possible, prepare data outside of the transaction, then execute the transaction to perform datastore operations that depend on a consistent state. The application should prepare Keys for objects used inside the transaction, then use db.get() to fetch the entities inside the transaction.