Creating, Getting and Deleting Data

The datastore API represents entities as instances of model classes. Methods of a model instance create, update and delete the entity. Entities can be fetched from the datastore as model instances using queries or keys.

• Creating and Updating an Entity
• Getting Entities Using a Query
• Getting an Entity Using a Key
• Deleting an Entity

Creating and Updating an Entity

Instances of Model (and Expando) classes represent datastore entities. An application creates a new entity of a given kind by calling the constructor of the model class that corresponds with the kind.

pet = Pet(name="Fluffy",
          type="cat",
          owner=users.get_current_user())

The new entity is not created in the datastore until the instance is "put" for the first time, either by calling the put() method on the instance, or by passing the instance to the db.put() function.

pet.put()

db.put(pet)

If an instance has been stored before, the put() method updates the existing entity.

Queries return results as model instances. These instances can be modified and put back into the datastore.

if users.get_current_user():
  user_pets = db.GqlQuery("SELECT * FROM Pet WHERE pet.owner = :1",
                          users.get_current_user())
  for pet in user_pets:
    pet.spayed_or_neutered = True

  db.put(user_pets)

Getting Entities Using a Query

The datastore can execute queries across entities of a given kind. A query can filter results using conditions that entity property values must meet, and can return the results sorted by property values. A query can also limit the scope to entities with a given ancestor; see Keys and Entity Groups.

For a complete description of how queries work, including several things queries cannot do, see Queries and Indexes.

The datastore API provides two interfaces for performing queries on entity properties: Query, an interface that prepares queries using methods on a query object, and GqlQuery, an interface that uses a SQL-like query language called GQL.

The Query Interface

The all() method on a Model (or Expando) class returns a Query object that represents a query for all entities of the corresponding kind. The application prepares the query by calling the filter(), order(), and ancestor() methods on the object.

class Story(db.Model):
  title = db.StringProperty()
  date = db.DateTimeProperty()

query = Story.all()

query.filter('title =', 'Foo')
query.order('-date')
query.ancestor(key)

# These methods can be chained together on one line.
query.filter('title =', 'Foo').order('-date').ancestor(key)

The GqlQuery Interface

The GqlQuery class constructor takes a GQL query string and optional parameter bindings. The query string specifies the kind, and the filters, sort orders and ancestor conditions. The query string can also include a result limit and offset.

# Parameters can be bound with positional arguments.
query = db.GqlQuery("SELECT * FROM Story WHERE title = :1 "
                    "AND ANCESTOR IS :2 "
                    "ORDER BY date DESC",
                    'Foo', key)

# Or, parameters can be bound with keyword arguments.
query = db.GqlQuery("SELECT * FROM Story WHERE title = :title "
                    "AND ANCESTOR IS :parent "
                    "ORDER BY date DESC",
                    title='Foo', parent=key)

# String, number and Boolean values can be literal values in the string.
query = db.GqlQuery("SELECT * FROM Story WHERE title = 'Foo' "
                    "AND ANCESTOR IS :parent "
                    "ORDER BY date DESC",
                    parent=key)

The gql() class method of a Model class also prepares a GqlQuery object from a string. The string is the GQL query string with the SELECT * FROM Model omitted, because this part is implied.

query = Story.gql("WHERE title = :title "
                  "AND ANCESTOR IS :parent "
                  "ORDER BY date DESC",
                  title='Foo', parent=key)

Parameter bindings can be re-bound to new values using the bind() method. An application can re-use a GqlQuery object by re-binding parameters and re-executing the query.

Executing the Query and Accessing Results

Query and GqlQuery objects do not execute the query until the application tries to access the results. When the application accesses results, the query is executed, and results are loaded into memory as instances of the model class for the query. Both query classes provide two ways to execute the query and access results: the fetch() method, and the iterator interface.

The fetch() method takes a maximum number of results to fetch (the limit), and an optional number of results to skip (the offset). The method executes the query, then fetches results until it has fetched the limit or there are no more results. Once the results are loaded into memory, it skips to the offset if one was specified, then returns the requested results as a list of model instances. The full query is executed for each call to fetch().

Note: The offset does not affect the number of results fetched from the datastore. All results up to the limit are fetched and stored in memory. The offset only affects what is returned by the fetch() method.

results = query.fetch(10)
for result in results:
  print "Title: " + result.title

The limit and offset given to the fetch() method override any limit and offset specified in a GQL query string.

If the query object is used as an iterator, the query is executed with no limit or offset, the results are loaded into memory, and the value returned is an iterator over the results. The iterator yields instances of the model class.

for result in query:
  print "Title: " + result.title

Note: The datastore returns a maximum of 1000 results in response to a query, regardless of the limit and offset used to fetch the results. The 1000 results includes any that are skipped using an offset, so a query with more than 1000 results using an offset of 100 will return 900 results.

Getting an Entity Using a Key

After an entity is stored in the datastore, the entity has a unique key. Key values are represented in the API represents as instances of the Key class. The put() method of a model instance and db.put() function return the Key of the stored entity. After a model instance has been stored for the first time, the key() method of the model instance returns the Key for the instance.

entity.put()
key = entity.key()

# ...

entity = db.get(key)

A common use of a Key value is to store it as the value of a property on another entity. The ReferenceProperty model property class provides automatic referencing and dereferencing of model instances as keys: A model instance can be assigned to a ReferenceProperty directly, and its key will be used as the value.

class Pet(db.Model):
  name = StringProperty()
  owner = ReferenceProperty(PetOwner)

class PetOwner(db.Model):
  name = StringProperty()

owner = PetOwner(name="Albert")
pet = Pet(name="Fluffy", owner=owner)

# This is equivalent:
pet = Pet(name="Fluffy", owner=owner.key())

Similarly, a ReferenceProperty value accessed via a property acts like its instance. The data entity is fetched automatically, and is not fetched until it is used.

pets = GqlQuery("SELECT * FROM Pet WHERE name = :1", "Fluffy")
pet = pets.get()

owner_name = pet.owner.name

Key values stored without the ReferenceProperty model, such as with an Expando dynamic property or a ListProperty element, do not have the automatic dereferencing behavior.

The db.get() function fetches an entity from the datastore for a Key (or list of Keys).

Keys can be encoded as strings for passing around outside of the application. To convert a string-encoded key back to a Key object, the application passes the string to the Key constructor.

obj = MyModel(name="Foo")
self.response.write('<a href="/view?key=%s">%s</a>' % (str(obj.key()), 
                                                      obj.name()))

# ...

key_name = self.request.get('key')
obj = db.get(db.Key(key_name))

Note: The string encoding of a Key is opaque, but not encrypted. If your application needs keys to not be guessable, you should further encrypt the string-encoded Key before sending it to the user.

Deleting an Entity

An application can delete an entity from the datastore using a model instance or a Key. The model instance's delete() method deletes the corresponding entity from the datastore. The delete() function takes a Key or list of Keys and deletes the entity (or entities) from the datastore.

q = db.GqlQuery("SELECT * FROM Message WHERE create_date < :1", earliest_date)
results = q.fetch(10)
for result in results:
  result.delete()

# or...

q = db.GqlQuery("SELECT * FROM Message WHERE create_date < :1", earliest_date)
results = q.fetch(10)
db.delete(results)

Deleting an entity does not change any Key values in the datastore that may have referred to the entity. If your application may attempt to de-reference a Key value for a deleted entity, the application should do so using db.get(), then test the return value before accessing properties.

Deleting an entity that is an ancestor for other entities does not affect the other entities. As long as the application does not depend on the existence of the ancestor to build keys for the descendant entities, the application can still access the descendants.