What Is Google App Engine?Google App Engine lets you run your web applications on Google's infrastructure. App Engine applications are easy to build, easy to maintain, and easy to scale as your traffic and data storage needs grow. With App Engine, there are no servers to maintain: You just upload your application, and it's ready to serve your users.You can serve your app from your own domain name (such as http://www.example.com/) using Google Apps. Or, you can serve your app using a free name on the appspot.com domain. You can share your application with the world, or limit access to members of your organization.Google App Engine supports apps written in several programming languages. With App Engine's Java runtime environment, you can build your app using standard Java technologies, including the JVM, Java servlets, and the Java programming language—or any other language using a JVM-based interpreter or compiler, such as JavaScript or Ruby. App Engine also features a dedicated Python runtime environment, which includes a fast Python interpreter and the Python standard library. The Java and Python runtime environments are built to ensure that your application runs quickly, securely, and without interference from other apps on the system.With App Engine, you only pay for what you use. There are no set-up costs and no recurring fees. The resources your application uses, such as storage and bandwidth, are measured by the gigabyte, and billed at competitive rates. You control the maximum amounts of resources your app can consume, so it always stays within your budget.App Engine costs nothing to get started. All applications can use up to 500 MB of storage and enough CPU and bandwidth to support an efficient app serving around 5 million page views a month, absolutely free. When you enable billing for your application, your free limits are raised, and you only pay for resources you use above the free levels.

The Application EnvironmentGoogle App Engine makes it easy to build an application that runs reliably, even under heavy load and with large amounts of data. App Engine includes the following features:

dynamic web serving, with full support for common web technologies

persistent storage with queries, sorting and transactions

automatic scaling and load balancing

APIs for authenticating users and sending email using Google Accounts

a fully featured local development environment that simulates Google App Engine on your computer

task queues for performing work outside of the scope of a web request

scheduled tasks for triggering events at specified times and regular intervalsYour application can run in one of two runtime environments: the Java environment, and the Python environment. Each environment provides standard protocols and common technologies for web application development.

The Sandbox

Applications run in a secure environment that provides limited access to the underlying operating system. These limitations allow App Engine to distribute web requests for the application across multiple servers, and start and stop servers to meet traffic demands. The sandbox isolates your application in its own secure, reliable environment that is independent of the hardware, operating system and physical location of the web server.Examples of the limitations of the secure sandbox environment include:

An application can only access other computers on the Internet through the provided URL fetch and email services. Other computers can only connect to the application by making HTTP (or HTTPS) requests on the standard ports.

An application cannot write to the file system. An app can read files, but only files uploaded with the application code. The app must use the App Engine datastore, memcache or other services for all data that persists between requests.

Application code only runs in response to a web request, a queued task, or a scheduled task, and must return response data within 30 seconds in any case. A request handler cannot spawn a sub-process or execute code after the response has been sent.

The Java Runtime EnvironmentYou can develop your application for the Java runtime environment using common Java web development tools and API standards. Your app interacts with the environment using the Java Servlet standard, and can use common web application technologies such as JavaServer Pages (JSPs).The Java runtime environment uses Java 6. The App Engine Java SDK supports developing apps using either Java 5 or 6.The environment includes the Java SE Runtime Environment (JRE) 6 platform and libraries. The restrictions of the sandbox environment are implemented in the JVM. An app can use any JVM bytecode or library feature, as long as it does not exceed the sandbox restrictions. For instance, bytecode that attempts to open a socket or write to a file will throw a runtime exception.Your app accesses most App Engine services using Java standard APIs. For the App Engine datastore, the Java SDK includes implementations of the Java Data Objects (JDO) and Java Persistence API (JPA) interfaces. Your app can use the JavaMail API to send email messages with the App Engine Mail service. Thejava.net HTTP APIs access the App Engine URL fetch service. App Engine also includes low-level APIs for its services to implement additional adapters, or to use directly from the application. See the documentation for the datastore, memcache, URL fetch, mail, images and Google Accounts APIs.Typically, Java developers use the Java programming language and APIs to implement web applications for the JVM. With the use of JVM-compatible compilers or interpreters, you can also use other languages to develop web applications, such as JavaScript, Ruby, or Scala.For more information about the Java runtime environment, see The Java Runtime Environment.

The Python Runtime Environment

With App Engine's Python runtime environment, you can implement your app using the Python programming language, and run it on an optimized Python interpreter. App Engine includes rich APIs and tools for Python web application development, including a feature rich data modeling API, an easy-to-use web application framework, and tools for managing and accessing your app's data. You can also take advantage of a wide variety of mature libraries and frameworks for Python web application development, such as Django.The Python runtime environment uses Python version 2.5.2. Additional support for Python 3 is being considered for a future release.The Python environment includes the Python standard library. Of course, not all of the library's features can run in the sandbox environment. For instance, a call to a method that attempts to open a socket or write to a file will raise an exception. For convenience, several modules in the standard library whose core features are not supported by the runtime environment have been disabled, and code that imports them will raise an error.Application code written for the Python environment must be written exclusively in Python. Extensions written in the C language are not supported.The Python environment provides rich Python APIs for the datastore, Google Accounts, URL fetch, and email services. App Engine also provides a simple Python web application framework called webapp to make it easy to start building applications.You can upload other third-party libraries with your application, as long as they are implemented in pure Python and do not require any unsupported standard library modules.For more information about the Python runtime environment, see The Python Runtime Environment.

The DatastoreApp Engine provides a distributed data storage service that features a query engine and transactions. Just as the distributed web server grows with your traffic, the distributed datastore grows with your data. You have the choice between two different data storage options differentiated by their availability and consistency guarantees.The App Engine datastore is not like a traditional relational database. Data objects, or "entities," have a kind and a set of properties. Queries can retrieve entities of a given kind filtered and sorted by the values of the properties. Property values can be of any of the supported property value types.Datastore entities are "schemaless." The structure of data entities is provided by and enforced by your application code. The Java JDO/JPA interfaces and the Python datastore interface include features for applying and enforcing structure within your app. Your app can also access the datastore directly to apply as much or as little structure as it needs.The datastore is strongly consistent and uses optimistic concurrency control. An update of a entity occurs in a transaction that is retried a fixed number of times if other processes are trying to update the same entity simultaneously. Your application can execute multiple datastore operations in a single transaction which either all succeed or all fail, ensuring the integrity of your data.The datastore implements transactions across its distributed network using "entity groups." A transaction manipulates entities within a single group. Entities of the same

group are stored together for efficient execution of transactions. Your application can assign entities to groups when the entities are created.

Google AccountsApp Engine supports integrating an app with Google Accounts for user authentication. Your application can allow a user to sign in with a Google account, and access the email address and displayable name associated with the account. Using Google Accounts lets the user start using your application faster, because the user may not need to create a new account. It also saves you the effort of implementing a user account system just for your application.If your application is running under Google Apps, it can use the same features with members of your organization and Google Apps accounts.The Users API can also tell the application whether the current user is a registered administrator for the application. This makes it easy to implement admin-only areas of your site.For more information about integrating with Google Accounts, see the Users API reference.

App Engine ServicesApp Engine provides a variety of services that enable you to perform common operations when managing your application. The following APIs are provided to access these services:

URL FetchApplications can access resources on the Internet, such as web services or other data, using App Engine's URL fetch service. The URL fetch service retrieves web resources using the same high-speed Google infrastructure that retrieves web pages for many other Google products.

MailApplications can send email messages using App Engine's mail service. The mail service uses Google infrastructure to send email messages.

MemcacheThe Memcache service provides your application with a high performance in-memory key-value cache that is accessible by multiple instances of your application. Memcache is useful for data that does not need the persistence and transactional features of the datastore, such as temporary data or data copied from the datastore to the cache for high speed access.

Image ManipulationThe Image service lets your application manipulate images. With this API, you can resize, crop, rotate and flip images in JPEG and PNG formats.

Scheduled Tasks and Task QueuesAn application can perform tasks outside of responding to web requests. Your application can perform these tasks on a schedule that you configure, such as on a

daily or hourly basis. Or, the application can perform tasks added to a queue by the application itself, such as a background task created while handling a request.Scheduled tasks are also known as "cron jobs," handled by the Cron service. For more information on using the Cron service, see the Python or Java cron documentation.Task queues are currently released as an experimental feature. At this time, only the Python runtime environment can use task queues. A task queue interface for Java applications will be released in the near future. For information about the task queue service and the Python API, see the Python documentation.

Development WorkflowThe App Engine software development kits (SDKs) for Java and Python each include a web server application that emulates all of the App Engine services on your local computer. Each SDK includes all of the APIs and libraries available on App Engine. The web server also simulates the secure sandbox environment, including checks for attempts to access system resources disallowed in the App Engine runtime environment.Each SDK also includes a tool to upload your application to App Engine. Once you have created your application's code, static files and configuration files, you run the tool to upload the data. The tool prompts you for your Google account email address and password.When you build a new major release of an application that is already running on App Engine, you can upload the new release as a new version. The old version will continue to serve users until you switch to the new version. You can test the new version on App Engine while the old version is still running.The Java SDK runs on any platform with Java 5 or Java 6. The SDK is available as a Zip file. If you use the Eclipse development environment, you can use the Google Plugin for Eclipse to create, test and upload App Engine applications. The SDK also includes command-line tools for running the development server and uploading your app.The Python SDK is implemented in pure Python, and runs on any platform with Python 2.5, including Windows, Mac OS X and Linux. The SDK is available as a Zip file, and installers are available for Windows and Mac OS X.The Administration Console is the web-based interface for managing your applications running on App Engine. You can use it to create new applications, configure domain names, change which version of your application is live, examine access and error logs, and browse an application's datastore.

Quotas and LimitsNot only is creating an App Engine application easy, it's free! You can create an account and publish an application that people can use right away at no charge, and with no obligation. An application on a free account can use up to 500MB of storage and up to 5 million page views a month. When you are ready for more, you can enable billing, set a maximum daily budget, and allocate your budget for each resource according to your needs.You can register up to 10 applications per developer account.

Each app is allocated resources within limits, or "quotas." A quota determines how much of a given resource an app can use during a calendar day. In the near future, you will be able to adjust some of these quotas by purchasing additional resources.Some features impose limits unrelated to quotas to protect the stability of the system. For example, when an application is called to serve a web request, it must issue a response within 30 seconds. If the application takes too long, the process is terminated and the server returns an error code to the user. The request timeout is dynamic, and may be shortened if a request handler reaches its timeout frequently to conserve resources.Attempts to subvert or abuse quotas, such as by operating applications on multiple accounts that work in tandem, are a violation of the Terms of Service, and could result in apps being disabled or accounts being closed.For a list of quotas and an explanation of the quota system, including which quotas can be increased by enabling billing, see Quotas.

Google App Engine is a platform for developing and hosting web applications in Google-managed data

centers. It was first released as a beta version in April 2008.

Google App Engine is cloud computing technology. It virtualizes applications across multiple servers and

data centers. Other "cloud"-based platforms include offerings such as Amazon Web Services and

Microsoft's Azure Services Platform. App Engine differs from services like Amazon Web Services, though,

in that AWS is Infrastructure as a Service while App Engine is Platform as a Service.

Google App Engine is free up to a certain level of used resources. Fees are charged for additional

storage, bandwidth, or CPU cycles required by the application

Supported features/restrictions

Currently, the supported programming languages are Python and Java (and, by extension, other JVM

languages such as Groovy, JRuby, Scala, Clojure, Jython, and special version of Quercus) . Python web

frameworks that run on Google App Engine include GAE framework, CherryPy, Pylons, and web2py, as

well as a custom Google-written webapp framework and several others designed specifically for the

platform that emerged since the release.[3] Google has said that it plans to support more languages in the

future, and that the Google App Engine has been written to be language independent. Any Python

framework that supports the WSGI using the CGI adapter can be used to create an application; the

framework can be uploaded with the developed application. Third-party libraries written in pure Python

may also be uploaded

Bulk downloading

SDK version 1.2.2 adds support for bulk downloads of data using Python.[6] The open source Python

projects gaebar,[7] approcket,[8] and gawsh[9] also allow users to download and backup App Engine data.

No method for bulk downloading data from GAE using Java currently exists.

Restrictions

Developers have read-only access to the filesystem on App Engine. Your application can use

only virtual filesystems, like gae-filestore.

App Engine can only execute code called from an HTTP request (scheduled background tasks

allow for self calling HTTP requests).

Users may upload arbitrary Python modules, but only if they are pure-

Python; C and Pyrex modules are not supported.

Java applications may only use a subset (The JRE Class White List) of the classes from the JRE

standard edition.[10]

Java applications cannot create new threads.

Does not support 'naked' domains (without www) like http://example.com, due to use of

DNS CNAME records for load balancing.[11] (Suggested workaround is to use DNS provider's HTTP

redirection to a subdomain, e.g. "www.example.com".)

SSL/HTTPS is only available via *.appspot.com domains and not via Google Apps Domains.

Datastore cannot use inequality filters on more than one entity property per query.

Not possible to change some default Jetty configurations such as accepting a form POST/GET

over 200KB in size.

A process started on the server to answer a request can't last more than 30 seconds. (with the

1.4.0 release, this restriction does not apply to background jobs anymore)

Major differences

Differences from other application hosting

Compared to other scalable hosting services such as Amazon EC2, App Engine provides more

infrastructure to make it easy to write scalable applications, but can only run a limited range of

applications designed for that infrastructure.

App Engine's infrastructure removes many of the system administration and development challenges of

building applications to scale to hundreds of requests per second and beyond. Google handles deploying

code to a cluster, monitoring, failover, and launching application instances as necessary.

While other services let users install and configure nearly any *NIX compatible software, App Engine

requires developers to use only its supported languages, APIs, and frameworks. Current APIs allow

storing and retrieving data from a BigTable non-relational database; making HTTP requests; sending e-

mail; manipulating images; and caching. Most existing Web applications can't run on App Engine without

modification, because they require a relational database.

Per-day and per-minute quotas restrict bandwidth and CPU use, number of requests served, number of

concurrent requests, and calls to the various APIs, and individual requests are terminated if they take

more than 30 seconds or return more than 10MB of data.

Differences between SQL and GQL

Google App Engine's datastore has a SQL-like syntax called "GQL". Select statements in GQL can be

performed on one table only. GQL intentionally does not support the Join statement, because it seems to

be inefficient when queries span more than one machine.[15] Instead, one-to-many and many-to-many

relationships can be accomplished using ReferenceProperty().[16] This shared-nothing approach allows

disks to fail without the system failing.[17] Switching from a relational database to the Datastore requires a

paradigm shift for developers when modelling their data.

Unlike a relational database the Datastore API is not relational in the SQL sense.

The Java version supports asynchronous non-blocking queries using the Twig Object Datastore interface.

This offers an alternative to using threads for parallel data processing.

Portability concerns

Developers worry that the applications will not be portable from App Engine and fear being locked into the

technology.[18] In response, there are a number of projects to create open-source back-ends for the

various proprietary/closed APIs of app engine, especially the datastore. Although these projects are at

various levels of maturity, none of them is at the point where installing and running an App Engine app is

as simple as it is on Google's service.[19]

Web2py web framework offers migration between SQL Databases and Google App Engine, however it

doesn't support several App Engine-specific features such as transactions and namespaces.[20]

The Django web framework and applications running on it can be used on App Engine with

modification. Django-nonrel aims to allow Django to work with non-relation databases and the project

includes support for App Engine.

Applications developed for the Grails web application framework may be modified and deployed to

Google App Engine with very little effort using the App Engine Plugin.

The GAE Java servlet container uses the Jetty Web Server, an open source project that implements the

2.5 servlet specification.

Usage quotas

App Engine defines usage quotas for free applications. Extensions to these quotas can be requested, and

application authors can pay for additional resources.

Hard limits

Quota Limit

Apps per developer 10

Time per request 30 sec

Blobstore size (total file size per app)

2 GB

HTTP response size 10 MB

Datastore item size 1 MB

Application code size 150 MB

Free quotas

Application creators who enable billing pay only for CPU, bandwidth, storage, and e-mails used in excess

of the free quotas. Limits marked with * are increased for application authors who enable billing, even if

their application never uses enough resources to incur charges. Free quotas were reduced on May 25,

2009[24] and were reduced again on June 22, 2009.[25]

Quota Limit

Emails per day 2,000

Bandwidth in per day 1,000 MB

Bandwidth out per day 1,000 MB

CPU time per day 6.5 hours per day

HTTP Requests per Day 1,300,000*

Datastore API calls per day 10,000,000*

Data stored 1 GB

URLFetch API calls per day.. 657,084*

Competition

The service competes with Amazon Web Services, a set of application services that enable web sites to

host files and execute code on Amazon's servers. Many tech analysts have been predicting Google's

entry into this field for years. "Google finally realizes it needs to be the web

platform," Techdirt publisher Mike Masnick wrote. "The easier it is to develop and deploy highly scalable

web applications, the more innovative and creative solutions we're going to start to see."[26]

Other competitors include Microsoft's Azure Services Platform, Salesforce.com's Force.com Platform,

Sina.com's Sina App Engine (Chinese) and Heroku.

AppScale is an open-source framework for running Google App Engine applications.

For tutorials: http://code.google.com/appengine/docs/java/gettingstarted/

App Development

Name What it is, and why use it? More Information

AppStats Language: Python and Java

Description: Measure the performance of your app with Appstats. Appstats is a suite of App and API call profiling tools that record events and provide a web-based administrative interface for browsing statistics.

More Info:

Appstats for Python

Appstats for Java

Easy Performance Profiling with Appstats (Google App Engine Blog)

Datastore SQLite Stub

Language: Python Only

Description: If you store a lot of data in your datastore, you can improve performance and startup times by switching to SQLite as the backend for your local datastore stub.

More Info:

Switching to SQLite for your local Datstore

Announcing the SQLite datastore stub... (Nick's Blog)

Local Unit Testing

Language: Java Only

Description: Write tests that use local service implimentations that are bundled with the SDK.

More Info:

Local Unit Testing

Samples for Google App Engine

Language: Python Only

Description: Check out sample applications that run on App Engine. You may check out samples from the trunk of the project.

More Info:

Samples for Google App Engine Project Page

Samples for Google App Engine Wiki

App Management

Name What it is, and why use it? More Information

Capabilities API

Language: Python and Java

Description: Use the Capabilities API to identify API outages and scheduled downtime.

More Info:

Capabilities API package summary (Java)

Capabilities API source (Python)

DoS Protection Service Configuration

Language: Python and Java

Description: Protect your application from running out of quota when subjected to DoS attacks or similar events with App Engine Denial of Service Protection Service. You can update the DoS configuration when you upload your application to App Engine, or you can update just the DoS configuration without uploading the rest of your application.

More Info:

Uploading DoS Configuration (Java)

Uploading DoS Configuration (Python)

Quota API Language: Python and Java

Description: Ensure your application doesn't exceed your resource quota(s) by using the Quota API.

More Info:

Using Quota API to Monitore CPU Usage in a Request

Quotas

Zipserve Language: Python Only

Description: Use Zipserve to allow your app to serve static files from a zipfile. This is useful for apps that need to serve a 1000+ small static files while staying within the 1000-file limit.

More Info:

Zipserve source

App Performance

Name What it is, and why use it? More Information

AppStats Language: Python and Java

Description: Measure the performance of your app with Appstats. Appstats is a suite of App and API call profiling tools that record events and provide a web-based administrative interface for browsing statistics.

More Info:

Appstats for Python

Appstats for Java

Easy Performance Profiling with Appstats (Google App Engine Blog)

Precompilation Language: Python and Java

Description: Enhance the performance of your app using precompilation. Precompilation is already enabled by default, but you have the option to disable it.

More Info:

Python Precompilation

Java Precompilation

App Testing

Name What it is, and why use it? More Information

Local Unit Testing

Language: Java Only

Description: Write tests that use local service implimentations that are bundled with the SDK.

More Info:

Local Unit Testing

Blobstore API

Name What it is, and why use it? More Information

Blobreader and BlobstoreInputStream

Language: Python and Java

Description: Read Blobs via a file-like interface using BlobReader in Python and/or BlobstoreInputStream in Java.

More Info:

BlobReader class (Python)

BlobstoreInputStream class summary (Java)

Datastore

Name What it is, and why use it? More Information

Bulk Loader Language: Python and Java

Description: Upload and download data to and from your application's datastore using the bulk loader tool. Python developers can use it directly from the appcfg.py command. Java developers can use appcfg.py from a Java app by installing the Java version of the remote_api handler.

More Info:

Uploading and Downloading Data

Datastore Statistics

Language: Python and Java

Description: View statistics about the data stored for an application using Datastore Statistics. You can view these statistics from the Administration Console under Datastore > Statistics or access these values programmatically within your application.

More Info:

Datastore Statistics in Python

Datastore Statistics in Java

Datastore Language: Python Only More Info:

SQLite StubDescription: If you store a lot of data in your datastore, you can improve performance and startup times by switching to SQLite as the backend for your local datastore stub.

Switching to SQLite for your local Datstore

Announcing the SQLite datastore stub... (Nick's Blog)

Development Console

Language: Python and Java

Description: Use the development console included in the development web server to browse the local datastore.

More Info:

The Development Console (Python)

The Development Console (Java)

Expando Class

Language: Python Only

Description: Use dynamic properties with datastore entities.

More Info:

The Expando Class

Mapper API Language: Python and Java

Description: Manipulate data on a large scale using the Mapper API, which is one component of the MapReduce toolkit.

More Info:

Introducing the Mapper API

Exploring the new mapper API in Python (Nick's blog)

Using the Java mapper framework (Ikai Lan's blog)

AppEngine Mapreduce Project Page.

Remote API Language: Python Only

Description: Access the datastore remotely while still using the standard familiar APIs with the remote_api module. This is useful when you need to manipulate the datastore in ways that are not well suited to the request/response model used for serving web applications.

More Info:

Accessing the datastore remotely with remote_api

Transactional Tasks

Language: Python and Java

Description: Enqueue tasks as part of a datastore transaction using Transactional Tasks such that the task is only enqueued—and guaranteed to be enqueued—if the transaction is committed successfully.

More Info:

Tasks within Transactions (Python)

Tasks within Transactions (Java)

Mail API

Name What it is, and why use it? More Information

Ereporter Language: Python Only

Description: Receive exception reports from your application via email by using the google.appengine.ext.ereporterpackage.

More Info:

Recording Exceptions with ereporter

Memcache

Name What it is, and why use it? More Information

Development Console

Language: Python and Java

Description: Use the development console included in the development web server to browse the local datastore.

More Info:

The Development Console (Python)

The Development Console (Java)

Task Queue

Name What it is, and why use it? More Information

Deferred Library

Language: Python Only

Description: Execute arbitrary functions using the Task Queue. This can be useful for one-off or ad-hoc tasks. The Deferred library lets you bypass all the work of setting up dedicated task handlers and serializing/deserializing parameters by exposing a

More Info:

Background work with the deferred library

simple function,deferred.defer().

Task Queue Storage Limits

Language: Python and Java

Description: Configure your task queue to specify or limit the total amount of storage task data that is allowed to be consumed over all queues.

More Info:

Python Task Queue Configuration

Java Task Queue Configuration

Transactional Tasks

Language: Python and Java

Description: Enqueue tasks as part of a datastore transaction using Transactional Tasks such that the task is only enqueued—and guaranteed to be enqueued—if the transaction is committed successfully.

More Info:

Tasks within Transactions (Python)

Tasks within Transactions (Java)

Users API

Name What it is, and why use it? More Information

OAuth Language: Python and Java

Description: Allow users to grant a third party or application limited permission to access your app on his or her behalf without sharing credentials using the OAuth protocol.

More Info:

OAuth for Python Overview

OAuth for Java Overview

OpenID Language: Python and Java

Description: Allow users to authenticate using their OpenID credentials by selecting Federated Login" instead of "Google Accounts" at creation time.

More Info:

Using Federated Authentication via OpenID

Using OpenID authentication (Nick's Blog)