Business Process Automation with Google App Engine and Google Data APIs
-
Upload
david-hodge -
Category
Documents
-
view
1.039 -
download
0
description
Transcript of Business Process Automation with Google App Engine and Google Data APIs
![Page 1: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/1.jpg)
Automating Business with Google APIs
David Hodge
Thursday, March 7, 13
![Page 2: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/2.jpg)
Thursday, March 7, 13
![Page 3: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/3.jpg)
Thursday, March 7, 13
![Page 4: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/4.jpg)
Thursday, March 7, 13
![Page 5: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/5.jpg)
Thursday, March 7, 13
![Page 6: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/6.jpg)
We automate business processes using cloud computing and
machine learning.
Dave & Bediako
Thursday, March 7, 13
![Page 7: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/7.jpg)
IAAS and PAAS
- Amazon, Google, Microsoft
- DevOps, NoOps, OOPs
- Beware single point of failure - read SLAs
Thursday, March 7, 13
![Page 8: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/8.jpg)
Google Docs Drive
• Google Docs API deprecated
• Google Drive Polyglot API - .NET, Java, Dart, Go, GWT, Java, Javascript, Objective-C, PHP, Python, Ruby
• ReSTful API - create and manage documents
Thursday, March 7, 13
![Page 9: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/9.jpg)
Google Spreadsheets API
• Spreadsheets, Worksheets, List Row, Cell
• Create, Delete, Share
• List Based Feeds vs Cell Based Feeds
• Updating multiple cells with Batch
Thursday, March 7, 13
![Page 10: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/10.jpg)
Google Spreadsheets API
• Gotchas
• Queries - age > 25 and height < 175
Thursday, March 7, 13
![Page 11: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/11.jpg)
Google Apps Scripts
• Javascript cloud scripting language
• Automate workflows
• Customize spreadsheets
• Schedule Tasks
• Save data using Spreadsheets, JDBC, ScriptDB
Thursday, March 7, 13
![Page 12: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/12.jpg)
Google App Engine
Thursday, March 7, 13
![Page 13: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/13.jpg)
Google App Engine
Backends
Thursday, March 7, 13
![Page 14: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/14.jpg)
Google App Engine
Backends
Storing Data
Thursday, March 7, 13
![Page 15: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/15.jpg)
Google App Engine
Backends
Storing Data
Blobstore
Thursday, March 7, 13
![Page 16: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/16.jpg)
Google App Engine
Backends
Storing Data
BlobstoreApp
Identity
Thursday, March 7, 13
![Page 17: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/17.jpg)
Google App Engine
Backends
Storing Data
BlobstoreApp
Identity
Google Cloud
Storage
Thursday, March 7, 13
![Page 18: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/18.jpg)
Google App Engine
Backends
Storing Data
BlobstoreApp
Identity
Google Cloud
Storage
Capabilities
Thursday, March 7, 13
![Page 19: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/19.jpg)
Google App Engine
Backends
Storing Data
BlobstoreApp
Identity
Google Cloud
Storage
Capabilities Channel
Thursday, March 7, 13
![Page 20: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/20.jpg)
Google App Engine
Backends
Storing Data
BlobstoreApp
Identity
Google Cloud
Storage
Capabilities Channel Endpoints
Thursday, March 7, 13
![Page 21: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/21.jpg)
Google App Engine
Backends
Storing Data
BlobstoreApp
Identity
Google Cloud
Storage
Capabilities Channel Endpoints
Images
Thursday, March 7, 13
![Page 22: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/22.jpg)
Google App Engine
Backends
Storing Data
BlobstoreApp
Identity
Google Cloud
Storage
Capabilities Channel Endpoints
Images Logs
Thursday, March 7, 13
![Page 23: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/23.jpg)
Google App Engine
Backends
Storing Data
BlobstoreApp
Identity
Google Cloud
Storage
Capabilities Channel Endpoints
Images Logs Mail
Thursday, March 7, 13
![Page 24: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/24.jpg)
Google App Engine
Backends
Storing Data
BlobstoreApp
Identity
Google Cloud
Storage
Capabilities Channel Endpoints
Images Logs Mail
Memcache
Thursday, March 7, 13
![Page 25: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/25.jpg)
Google App Engine
Backends
Storing Data
BlobstoreApp
Identity
Google Cloud
Storage
Capabilities Channel Endpoints
Images Logs Mail
Memcache Multitenancy
Thursday, March 7, 13
![Page 26: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/26.jpg)
Google App Engine
Backends
Storing Data
BlobstoreApp
Identity
Google Cloud
Storage
Capabilities Channel Endpoints
Images Logs Mail
Memcache Multitenancy OAuth
Thursday, March 7, 13
![Page 27: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/27.jpg)
Google App Engine
Backends
Storing Data
BlobstoreApp
Identity
Google Cloud
Storage
Capabilities Channel Endpoints
Images Logs Mail
Memcache Multitenancy OAuth
Prospective Search
Thursday, March 7, 13
![Page 28: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/28.jpg)
Google App Engine
Backends
Storing Data
BlobstoreApp
Identity
Google Cloud
Storage
Capabilities Channel Endpoints
Images Logs Mail
Memcache Multitenancy OAuth
Prospective Search
Search
Thursday, March 7, 13
![Page 29: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/29.jpg)
Google App Engine
Backends
Storing Data
BlobstoreApp
Identity
Google Cloud
Storage
Capabilities Channel Endpoints
Images Logs Mail
Memcache Multitenancy OAuth
Prospective Search
Search
Task Queues
Thursday, March 7, 13
![Page 30: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/30.jpg)
Google App Engine
Backends
Storing Data
BlobstoreApp
Identity
Google Cloud
Storage
Capabilities Channel Endpoints
Images Logs Mail
Memcache Multitenancy OAuth
Prospective Search
Search
Task Queues
Users
Thursday, March 7, 13
![Page 31: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/31.jpg)
Google App Engine
Backends
Storing Data
BlobstoreApp
Identity
Google Cloud
Storage
Capabilities Channel Endpoints
Images Logs Mail
Memcache Multitenancy OAuth
Prospective Search
Search
Task Queues
Users
XMPP
Thursday, March 7, 13
![Page 32: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/32.jpg)
One does not simply
Automate using Google APIs
Thursday, March 7, 13
![Page 33: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/33.jpg)
How we build softwareA Customerhas a nick namehas a full namehas a date of birthhas a social security number which is sensitivecan be viewed or created or changed by a customer service representative
An Orderbelongs to a Customerhas a descriptioncan be viewed or created or changed by a customer service representative
An Order Line Item is auditedbelongs to an Orderhas a descriptionhas a price which is a numbercan be viewed or created or changed by a customer service representative
Dictation + Airlift on Google App Engine
Thursday, March 7, 13
![Page 34: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/34.jpg)
Current processes manual and paper based for recording line items for accounts receivable and accounts payable.
Thursday, March 7, 13
![Page 35: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/35.jpg)
Customer Widgets Spreadsheets
Thursday, March 7, 13
![Page 36: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/36.jpg)
Widget Maker Spreadsheets
Thursday, March 7, 13
![Page 37: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/37.jpg)
Evolution One - Cron<?xml version="1.0" encoding="UTF-8"?><cronentries> <cron> <url>/accounting/receivable</url> <description>Process widgets sold for accounting line items</description> <schedule>every day 12:30</schedule> <timezone>America/New_York</timezone> </cron> <cron> <url>/accounting/payable</url> <description>Process line items for widget making workers</description> <schedule>every day 03:30</schedule> <timezone>America/New_York</timezone> </cron></cronentries>
Thursday, March 7, 13
![Page 38: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/38.jpg)
Evolution Two - Backends
• Use Backends to process long lived requests - No Request Deadlines
• Higher CPU up to 4.8GHz and Memory limits - up to 1GB
• Also using Spreadsheet Batch
Thursday, March 7, 13
![Page 39: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/39.jpg)
Evolution Two - Backends
• Dynamic vs Resident - Dynamic Backends become alive only on request and are turned off when idle
Thursday, March 7, 13
![Page 40: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/40.jpg)
Evolution Two - Backends
<backends> <backend name="accounting"> <options> <dynamic>true</dynamic> <fail-fast>true</fail-fast> <class>B8</class> </options> </backend></backends>
backends.xml
<?xml version="1.0" encoding="UTF-8"?><cronentries> <cron> <url>/accounting/receivable</url> <description>Process widgets sold for accounting line items</description> <schedule>every 20 minutes</schedule> <timezone>America/New_York</timezone>
<target>accounting</target> </cron> <cron> <url>/accounting/payable</url> <description>Process line items for widget making workers</description> <schedule>every 40 minutes</schedule> <timezone>America/New_York</timezone>
<target>accounting</target> </cron></cronentries>
cron.xml
Thursday, March 7, 13
![Page 41: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/41.jpg)
Task Queues work discretely on a unit of work
Push - processes tasks based on rate configuration, auto scales
Pull - allows for specific time and timeframe to run
Thursday, March 7, 13
![Page 42: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/42.jpg)
Task Push Config<queue-entries> <queue> <name>fooqueue</name> <rate>1/s</rate>
<bucket-size>40</bucket-size> <max-concurrent-requests>1</max-concurrent-requests> <retry-parameters> <task-retry-limit>7</task-retry-limit> <task-age-limit>2</task-age-limit> </retry-parameters> </queue> <queue> <name>barqueue</name> <rate>1/s</rate> <retry-parameters> <min-backoff-seconds>10</min-backoff-seconds> <max-backoff-seconds>200</max-backoff-seconds> <max-doublings>0</max-doublings> </retry-parameters> </queue> </queue-entries>
queue.xml
Thursday, March 7, 13
![Page 43: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/43.jpg)
Task Queue Headers
• X-AppEngine-QueueName, the name of the queue (possibly default)• X-AppEngine-TaskName, the name of the task, or a system-generated
unique ID if no name was specified• X-AppEngine-TaskRetryCount, the number of times this task has
been retried; for the first attempt, this value is 0. This number includes attempts where the task failed due to a lack of available instances and never reached the execution phase.
• X-AppEngine-TaskExecutionCount, the number of times this task has previously failed during the execution phase. This number does not include failures due to a lack of available instances.
• X-AppEngine-TaskETA, the target execution time of the task, specified in microseconds since January 1st 1970.
Thursday, March 7, 13
![Page 44: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/44.jpg)
Task Push Queues
import com.google.appengine.api.taskqueue.Queue;import com.google.appengine.api.taskqueue.QueueFactory;import static com.google.appengine.api.taskqueue.TaskOptions.Builder.*;
//Discrete job for processing single widget to write customer line item Queue queue = QueueFactory.getDefaultQueue(); queue.add(withUrl("/widgetorderforcustomerjob").param("widgetKey", key));
//Discrete job for processing single widget to write vendor line item queue.add(withUrl("/widgetorderforvendorjob").param("widgetKey", key));
Thursday, March 7, 13
![Page 45: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/45.jpg)
Google DocsGoogle App Engine
Automation Achieved
Receivable Line Items Job
Payable Line Items Job
Google Spreadsheet
Line Items Per Client
Google Spreadsheet
Line Items Per Vendor
Invoices
Payments
Thursday, March 7, 13
![Page 46: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/46.jpg)
Experiment
EvolveArchitecture
Thursday, March 7, 13
![Page 47: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/47.jpg)
Support
• Google Plus Hangouts
• Google Forums
• Fellow Developers
Thursday, March 7, 13
![Page 48: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/48.jpg)
Some More Tips
• Python: Bulk Loader Tool to download/upload data - good for any flavor of GAE
• Get involved - NoOps doesn’t mean you don’t have to know your technology stack
• https://github.com/LucidTechnics/google-apps-util
• https://code.google.com/p/googleappengine/wiki/WillItPlayInJava
Thursday, March 7, 13
![Page 49: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/49.jpg)
“Using no way as a way, having no limitation as limitation.”
- Bruce Lee
Thursday, March 7, 13
![Page 50: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/50.jpg)
Official White House Photo by Pete SouzaThursday, March 7, 13
![Page 51: Business Process Automation with Google App Engine and Google Data APIs](https://reader030.fdocuments.us/reader030/viewer/2022020105/55568fd0d8b42acc5e8b4950/html5/thumbnails/51.jpg)
David Hodge@davezen1
Thursday, March 7, 13