English | Site Directory

Using the Datastore

Storing data in a scalable web application can be tricky. A user could be interacting with any of dozens of web servers at a given time, and the user's next request could go to a different web server than the one that handled the previous request. All web servers need to be interacting with data that is also spread out across dozens of machines, possibly in different locations around the world.

Thanks to Google App Engine, you don't have to worry about any of that. App Engine's infrastructure takes care of all of the distribution, replication and load balancing of data behind a simple API—and you get a powerful query engine and transactions as well.

Storing the Submitted Greetings

App Engine includes a data modelling API for Python. It's similar to Django's data modelling API, but uses App Engine's scalable datastore behind the scenes.

For the guestbook application, we want to store greetings posted by users. Each greeting includes the author's name, the message content, and the date and time the message was posted so we can display messages in chronological order.

Edit helloworld/helloworld.py, and add the following import statements at the top:

from google.appengine.ext import db

Add the following class just above the MainPage class:

class Greeting(db.Model):
  author = db.UserProperty()
  content = db.StringProperty(multiline=True)
  date = db.DateTimeProperty(auto_now_add=True)

This defines a Greeting model with three properties: author whose value is a User object, content whose value is a string, and date whose value is a datetime.datetime.

Some property constructors take parameters to further configure their behavior. Giving the db.StringProperty constructor the multiline=True parameter says that values for this property can contain newline characters. Giving the db.DateTimeProperty constructor a auto_now_add=True parameter configures the model to automatically give new objects a date of the time the object is created, if the application doesn't otherwise provide a value. For a complete list of property types and their options, see the Datastore reference.

Now that we have a data model for greetings, the application can use the model to create new Greeting objects and put them into the datastore. Edit the Guestbook handler to appear similar to the following:

class Guestbook(webapp.RequestHandler):
  def post(self):
    greeting = Greeting()

    if users.get_current_user():
      greeting.author = users.get_current_user()

    greeting.content = self.request.get('content')
    greeting.put()
    self.redirect('/')

This new Guestbook handler creates a new Greeting object, then sets its author and content properties with the data posted by the user. It does not set the date property, so date is automatically set to "now," as we configured the model to do.

Finally, greeting.put() saves our new object to the datastore. If we acquired this object from a query, put() would update the existing object. Since we created this object with the model constructor, put() adds the new object to the datastore.

Retrieving the Stored Greetings With GQL

The App Engine datastore has a sophisticated query engine for data models. Because the App Engine datastore is not a traditional relational database, queries are not specified using SQL. Instead, you can prepare queries using a SQL-like query language we call GQL. GQL provides access to the App Engine datastore query engine's features using a familiar syntax.

Edit the MainPage handler to appear similar to the following:

class MainPage(webapp.RequestHandler):
  def get(self):
    self.response.out.write('<html><body>')

    greetings = db.GqlQuery("SELECT * FROM Greeting ORDER BY date DESC LIMIT 10")

    for greeting in greetings:
      if greeting.author:
        self.response.out.write('<b>%s</b> wrote:' % greeting.author.nickname())
      else:
        self.response.out.write('An anonymous person wrote:')
      self.response.out.write('<blockquote>%s</blockquote>' %
                              cgi.escape(greeting.content))

    # Write the submission form and the footer of the page
    self.response.out.write("""
          <form action="/sign" method="post">
            <div><textarea name="content" rows="3" cols="60"></textarea></div>
            <div><input type="submit" value="Sign Guestbook"></div>
          </form>
        </body>
      </html>""")

Reload http://localhost:8080/ in your browser, and post a few messages to verify that messages get posted correctly.

The query happens on this line:

    greetings = db.GqlQuery("SELECT * FROM Greeting ORDER BY date DESC LIMIT 10")

Alternatively, you can call the gql(...) method on the Greeting class, and omit the SELECT * FROM Greeting from the query:

    greetings = Greeting.gql("ORDER BY date DESC LIMIT 10")

As with SQL, keywords (such as SELECT) are case insensitive. Names, however, are case sensitive.

Because the query returns full data objects, it does not make sense to select specific properties from the model. All GQL queries start with SELECT * FROM model (or are so implied by the model's gql(...) method) so as to resemble their SQL equivalents.

A GQL query can have a WHERE clause that filters the result set by one or more conditions based on property values. Unlike SQL, GQL queries may not contain value constants: Instead, GQL uses parameter binding for all values in queries. For example, to get only the greetings posted by the current user:

    if users.get_current_user():
      greetings = Greeting.gql("WHERE author = :1 ORDER BY date DESC",
                               users.get_current_user())

You can also use named parameters instead of positional parameters:

      greetings = Greeting.gql("WHERE author = :author ORDER BY date DESC",
                               author=users.get_current_user())

In addition to GQL, the datastore API provides another mechanism for building query objects using methods. The query above could also be prepared as follows:

      greetings = Greeting.all()
      greetings.filter("author =", users.get_current_user())
      greetings.order("-date")

For a complete description of GQL and the query APIs, see the Datastore reference.

Clearing the Development Server Datastore

The development web server uses a local version of the datastore for testing your application, using temporary files. The data persists as long as the temporary files exist, and the web server does not reset these files unless you ask it to do so.

If you want the development server to erase its datastore prior to starting up, use the --clear_datastore option when starting the server:

dev_appserver.py --clear_datastore helloworld/

Next...

We now have a working guest book application that authenticates users using Google accounts, lets them submit messages, and displays messages other users have left. Because App Engine handles scaling automatically, we will not need to revisit this code as our application gets popular.

This latest version mixes HTML content with the code for the MainPage handler. This will make it difficult to change the appearance of the application, especially as our application gets bigger and more complex. Let's use templates to manage the appearance, and introduce static files for a CSS stylesheet.

Continue to Using Templates.