Web Planning an Effective Integration with a 3rd party API to Scale
23
Using Web Services for Third party integrations to scale (mandatory cliché business clipart)
Transcript of Web Planning an Effective Integration with a 3rd party API to Scale
Using Web Services for Third party integrations to scale
(mandatory cliché business clipart)
Building Rapport with Audience
• 5 KIDS!• Moved from Small
Washington town, 4 months ago
• Many Web service integrations
Mandatory Animated GIF
What is a web service
• Pipeline for communication• A means to integrate to servers • Often synonymous with an API• Gateway to Saas
Where can I use it
• Maps• Social• Mobile• User Tracking• Data exchange• Automation/Workflow
How do I get started
Google “<BRAND> API Getting Started”
Get all available documentation
• Be greedy about documentation– Get Everything (Yahoo! Maps)– Read it– Learn it
• Identify available metaphors/verbs
REST vs SOAP
REST• Follows philosophy of
open web• Easy to test• Newer and standard• Multiple return formats
– JSON– CSV– XML
• Uses port 80 (Standard)• Easy to Debug
SOAP• XML based• Requires SOAP
scaffolding• XML return format• Older standard• Debugging is a harder• Can handle additional
security/ports
Web Services 101
Favor REST
REST is an acronym for the Representational State Transfer architectural style for distributed hypermedia systems.
You will get started faster
Don’t trust the API
• Verify method names– Input parameters– Return Values• Identify all error codes ahead
of time
– Confirm return structure
Handle all error codes
• Many error codes aren’t documented• Error codes are not priority of API provider• Missing error codes can break your integration
Write POC application
• Prove the API will work– Assume that it will not work– Better to fail implementation early
• POC App should be an inch wide and a mile deep
Identify Technical Resource
• The Webservice will have secret handshakes, Assume that everything is not documented.
• WebServices are often the last piece of the application that is implemented, they are an after thought.– Even when part of architecture
FIND A GUIDE
Multi-threaded parallelization
• Web Services are a single call, even batched, it is call/response.– GET User– GET ListOfUsers
• Single call is fast– Many “single calls” S-L-O-W
• Paging ( still single threaded )• Get all IDs -> Threaded Gets
Use Batch operations
• Time determines winner– Multithreaded API calls ( 22 hours )– Batch operations (20 minutes )
• Flat file– CSV/TSV
Implement retry logic
• Assume API calls will fail• Should have retry
threshold– retry max ‘n’ times
• Retries should have exponential backoff
• Fail BIG
Create strong feedback loop
• How does implementation break• Who hears about it and how?– Email– GitHub– User feedback– SMS
TDD – Testing Driven Development
• APIs tend to change silently• Write unit tests• Run them often
Use technical resource
• Suck up to these people• Get questions answered, don’t feel bad– It is often their job to evangelize
• They are your gateway to the real help– Product managers– Platform programmers– $$$
• Get code review from technical resources
Tips for Success
• Create mutual value for third party and self• Regular and ongoing communication• Send thank you notes/gifts• Use back channel communication• Push, but not too hard
