Keynote: Paving the Way and Making a Difference: AWS in the Public Sector

AWS Government, Education, and Nonprofits SymposiumLondon | 21 October 2014

AWS Government, Education, and Nonprofits SymposiumLondon I 21 October 2014

AWS in the Public Sector

Teresa CarlsonVice President

World Wide Public Sector


Strong growth and adoption

800+

government

agencies

3000+

educational

institutions

10000+

nonprofit

organizations


Used by Government Agencies & Educational Institutions

Worldwide


AWS Partners Focused on Public Sector

http://www.ppc.com/

http://www.ppc.com/

http://www.aws-partner-directory.com/PartnerDirectory/PartnerDetail?Name=Pegasystems

http://www.aws-partner-directory.com/PartnerDirectory/PartnerDetail?Name=Pegasystems

http://www.aws-partner-directory.com/PartnerDirectory/PartnerDetail?Name=Sonian

http://www.aws-partner-directory.com/PartnerDirectory/PartnerDetail?Name=Sonian

http://sourcedgroup.com.au/

http://sourcedgroup.com.au/


AWS Rapid Pace of Innovation

2009

Amazon RDS

Amazon VPC

Auto Scaling

Elastic Load

Balancing

+48

2010

Amazon SNS

AWS Identity

& Access

Management

Amazon Route 53

+61

2011

Amazon

ElastiCache

Amazon SES

AWS

CloudFormation

AWS Direct

Connect

AWS Elastic

Beanstalk

GovCloud

+82

Amazon

CloudTrail

Amazon

CloudHSM

Amazon

WorkSpaces

Amazon Kinesis

Amazon Elastic

Transcoder

Amazon

AppStream

AWS OpsWorks

+280

2013

Amazon SWF

Amazon Redshift

Amazon Glacier

Amazon

Dynamo DB

Amazon

CloudSearch

AWS Storage

Gateway

AWS Data

Pipeline

+159

2012

Since inception AWS has:

• Released 942 new services and features

• Introduced over 35 major new services

• Announced 45 price reductions

2008

+24Amazon EBS

Amazon

CloudFront

+285

2014

Amazon Cognito

Amazon Zocalo

Amazon Mobile

Analytics

*as of Aug 18, 2014


Amazon

ElasticCache

reduces prices for

cache nodes by an

average of 34%

March 26, 2014

34%Amazon S3 reduces

prices for Standard and

Reduced Redundancy

Storage, by an average

of 51%

March 26, 2014

51%

We’ve announced price reductions 45* times since

our inception in 2006. Recent price drops included…

*as of July 31, 2014

Amazon Route 53

lowers prices for both

Standard Queries and

Latency Based

Routing Queries by

20%

July 31, 2014

20%


Architected for Enterprise Security RequirementsCertifications and accreditations for

workloads that matter

AWS CloudTrail - AWS API call

logging for governance & compliance

Stores data in

S3, or archive

to Glacier

Log and review

user activity


AWS Helps Government Cut Costs

“We are in the process of putting most of our public-facing

data in an Amazon cloud service,” said Terry Halvorsen,

the Chief Information Officer of the Department of the

Navy, in a keynote at Meritalk’s Data Center Brainstorm

event Thursday. Halvorsen said the move could save

the Navy as much as 60 percent versus the cost of

managing that data in its own data centers.-Data Center Knowledge,

March 14, 2014


AWS Global Infrastructure

10 Regions

including GovCloud US

26 Availability

Zones

51 Edge Locations


Why Does the Cloud Matter to Public Sector?

Pave the Way for

Disruptive Innovation

Make the World

a Better Place


Paving the way

What do public sector customers need?

• Disruptive innovation

• Agility

• Capability

• Cost savings


Evolution of cloud adoption in government

Policy


Phase 0 – Stated Policy

“Cloud First”



DefinitionPolicy


Phase 1 – Standard Definition



Security and Compliance

DefinitionPolicy


Phase 2 – Security & Compliance




Procurement

DefinitionPolicy


Phase 3 – Cloud-Friendly Procurement

Department of Interior

Foundation Cloud

United Kingdom G-Cloud

Dept. of Treasury Public Cloud Web Hosting Services

US Communities

Texas Dept. of

Information Resources contract

Navy SPAWAR

US Intelligence Community



DefinitionPolicy


Procurement

Culture




Procurement

Culture

Broad Adoption

DefinitionPolicy


Paving the way for…

Real change


Make The World a Better Place

• Support world-changing projects

• Enable economic development

• Improve citizen services and engagement

• Improve research and education


When You’re Changing the World, You Can’t Afford to be Slow…

Add New Dev Environment

Add New Prod Environment

Go Global in Seconds

Add 1,000 Servers

Remove 1,000 Servers

Deploy 1 PB Data Warehouse

Shut down 1 PB Data Warehouse

AWS:

Resources in MinutesOld World:

Resources in Weeks

Everything changes with this

kind of agility and speed


Increased agility has

become the #1 reason

organizations use the AWS

cloud



1. Support world-changing projects

2. Enable economic development

3. Improve citizen services and engagement

4. Improve research and education


What better way to change the world…

Code.org ran a worldwide

“Hour of Code”

More than 20 million youth

coded on their website in a

single week, with a peak load of

330,000 concurrent users

“Running on the AWS Cloud

gave us the elasticity to keep

the website running when

traffic spiked from zero to 20

million coders during

campaign week, and then

scale back efficiently. AWS

was fantastic.”-Geoffrey Elliott, Code.org


Ensuring democracy…

Migration of core business applications for secure global

access, reduced costs, focused resources and improved

availability

"The driver really was the ability to

be responsive. The way we did that

was to move to the cloud.“

-Chris Spence, CIO

National Democratic

Institute


Helping Nonprofits Achieve Scale and Reach

"Working hand in glove with AWS,

LeanIn.Org's development team was

able to re-architect the site over a 64-

hour period to ensure it would stand

up to 200 click-throughs per second."

-Rachel Thomas

President, Lean In

“Ban Bossy” campaign

Featured on Google home page and

stayed up!


Using Analytics to Understand the World

“Do the simplest

thing,” but do it at a

stupidly large scale.”

-David Milne, a postdoctoral

research fellow at CSIRO



European Bioinformatics Institute

SureChEMBL: Patient Data in the Cloud

Mark Davies, Technical Lead

[email protected]


What is EMBL-EBI?• Part of the European

Molecular Biology Laboratory

• International, non-profit research institute

• Europe’s hub for biological data services and research

• 500 members of staff from 53 nations.


ChEMBL – Data for Drug Discovery

Bioactivity data

Compound

Assa

y/T

arg

et

>Thrombin

MAHVRGLQLPGCLALAALCSLVHSQHVFLAPQQARSLLQRVRRANT

FLEEVRKGNLERECVEETCSYEEAFEALESSTATDVFWAKYTACET

ARTPRDKLAACLEGNCAEGLGTNYRGHVNITRSGIECQLWRSRYPH

KPEINSTTHPGADLQENFCRNPDSSTTGPWCYTTDPTVRRQECSIP

VCGQDQVTVAMTPRSEGSSVNLSPPLEQCVPDRGQQYQGRLAVTTH

GLPCLAWASAQAKALSKHQDFNSAVQLVENFCRNPDGDEEGVWCYV

AGKPGDFGYCDLNYCEEAVE

3. Insight, tools and resources for translational drug discovery

2. Organization, integration, curation and standardization of pharmacology data

1. Scientific facts


Who works with ChEMBL?

https://www.ebi.ac.uk/chembl/

https://www.ebi.ac.uk/chembl/


Patent Data and Drug Discovery

• Historically a closed and costly data source

– Out of reach to many academics and SMEs

• Patent literature 2-3 years ahead of published literature

• Prior art and freedom to operate

• Competitor intelligence

• Provides access to lots more data

– High cost to extract and lots of noise


SureChEMBL – Open Patent Data

• December 2013 EMBL-EBI acquired SureChem

• A leading chemistry patent mining product from

Digital Science, Macmillan Group

• Commercial to open transition ‘perfect fit’ for both

EMBL-EBI and Digital Science

• System rebranded SureChEMBL and migrated to

EMBL-EBI resources


SureChEMBL Migration Challenge

• SureChem was an AWS based system, should we stay or

should we go?

– Live system with many dependencies

– Supporting existing customers during migration

– Short timescale

– Limited resources

– Limited budget

– Existing AWS experience


We stayed with AWS


SureChEMBL System

Keyword

searches

Chemistry

searches

Access to

chemistry

https://www.surechembl.org

Complex

filters

Access to

patents



SureChEMBL AWS Resources

Elastic Load

Balancing

Amazon

VPC

Amazon

EC2

Elastic

IP

Amazon

S3

Amazon

EBS

Snapshot CloudWatch IAM

Amazon SQS


AWS Benefits

• AWS provides a comprehensive suite tools and services

– As a developer, everything required to build system is just there

• Fast design, build, deploy and teardown cycle

• Web based AWS console

• Use programmatic access to automate tasks and monitor system

– CLI and multiple SDKs available

• CloudWatch alarms to monitor system

• Cost calculator

• New services being added all the time

• Excellent support

– Account manager, solution architects, online forums


AWS Recommendations

• Make use of excellent online documentation

– Many whitepapers and best practice guidelines

• Use VPC to architect system

– Refresh network knowledge

– Tricky to migrate from ‘Classic’ EC2

• Review costs

– Use the cost calculator

– Use reserved instances

– Monitor storage: RAIDs, backups, snapshots, storage medium, logs

– Shutdown resources when not being used

• Prepare for unexpected events

– Check the ‘Events’ section EC2 dashboard


SureChEMBL Migration Success19th September 2014 SureChEMBL system released




AcknowledgementsChEMBL team• John Overington

• Jon Chambers

• George Papadatos

• Mark Davies

• Nathan Dedman

• Anna Gaulton

Digital Science• Nicko Goncharoff

• James Siddle

• Richard Koks

Funding:• Wellcome Trust Strategic Award for

ChEMBL database (WT086151/Z/08/Z &

WT104104/Z/14/Z)

• Open PHACTS - Innovative Medicines

Initiative Joint Undertaking (grant no.

115191)

• European Molecular Biology Laboratory

• BioMedBridges - European Commission

FP7 Capacities Specific Programme (grant

no. 284209)

Technology Partners:



Thank YouMark Davies, Technical Lead

[email protected]


Public/private engagement for

economic development

• Geospatial collaborative

environment

• 80+ participating agencies

• 30+ new services

developed

• 100M+ users of the service


Public/private engagement for

economic development

“AWS has helped my organization to

provide better service availability

and handle higher traffic load at a

lower cost”

-Chan Chin Wai, CIO, SLA


June 2014 Competition

Finalists and Winners

Partners in Innovation

Best Practices Award


Best Practices Award Winners

https://onedrive.live.com/?cid=B2ABCDB3F5BCF837&id=B2ABCDB3F5BCF837!5667&v=3

https://onedrive.live.com/?cid=B2ABCDB3F5BCF837&id=B2ABCDB3F5BCF837!5667&v=3

http://propertymap.sfplanning.org/pimvideo.html

http://propertymap.sfplanning.org/pimvideo.html


Turning Government Data into Real Insight

Scalable web application Big Data Analytics

and Collaboration

Rapid deployment of analytics

engine

Redesigned portions of

Healthcare.gov

Post “flash crash”

forensics on EC2

Collaboration

platform for SEC

Mining social media for

early warnings of food and

drug safety issues on

accelerated timeline

Healthcare.gov



Taking Transport for

London into the cloud

Phil Young

Head of TfL Online

[email protected]

• Background TfL and our Online services

• Choosing the cloud what drove our selection

• Making it happen build and commissioning

• Outcomes our experiences so far

Overview

54

About TfL• London’s integrated transport authority

• Responsible for all forms of transport –Tube, buses, roads and

congestion charging, trams, river, DLR, Overground, cycling,

walking, coaches, freight, taxis and private hire and more

• Around 24 million journeys daily on our network

• Planning and developing for the future, new services to meet the

demands of an increasing population

• Iconic brand, embedded into the fabric of our city


What our customers want

56

Understand what we stand for

Value for MoneyProgress &

Innovation

Trust

Excellent

reliability and

customer

experience

TfL web visits 2006-2014

0

5

10

15

20

25

30

Mill

ion

s

Total Visits


Legacy issues

• Previous site not refreshed since 2007

• Met core functional needs, but required user effort for more complex

tasks (eg finding maps, logins)

• No longer met customer expectations for

localisation, personalisation, mapping

and journey planning

• Poor experience on mobiles and tablets

Requirements

Customer

TechnicalBusiness

The new site needed to meet

customer needs first, while delivering

on business objectives and being

technically deliverable

1. Personalised information

2. Fast realtime info on-the-move

3. Mobile first, great on all devices

4. One stop shop for planning and info

5. Improved visual appearance

6. Humanised interaction

7. Single log-in

8. Integrated journey planning tools

9. Trusted information

10. Simplified experience

11. Consistent experience of all of TfL

12. Easy access to customer service

13. Reassurance around travel decisions

Key customer requirements

Our vision of a new TfL site

CHOOSING THE CLOUD

61


Platform requirementsFunctional:

• Personalisation and localisation

• Separate presentation layer

• Disparate data sources

• Integrated and consistent data

needed

• CMS and web application

integration

• Responsive design

Performance:

• Scalable - 30x spikes

• 24 hour 100% availability

• Fast response times

• Accommodate limited back-end

systems

• Highly secure

• Deal in realtime services

• Cost effective


Choosing the cloud

• Elastic capacity

• Speed and flexibility, automation

• Cost

• Resilience and referencability

AWS selected because:

• Best match at the time for our multi-OS stack

• Cost-effective, usage-based approach

• Rapidly developing platform with regular new features and falling costs


Challenges of delivery

• Product evolving through an agile programme as platform being built

• In house team and multi-supplier environment

• Challenging timescales to move from experimental to production

• Lack of cloud experience

• Platform maturity – still developing

• Internal scepticism

OUTCOMES

65


What we delivered

• A new API with a canonical data model

• Complex web application through MVC

• Multiple environments, ‘push button’ creation and product deployment

• Autoscaling

• Blue/green release approach

• Varnish caching layer

• Web application firewall

http://youtu.be/RTwR558tzTw

http://youtu.be/RTwR558tzTw


New from the ground-up

67

Responsive design

Mobile first

New information architecture

New search

New Journey Planner

Google maps throughout

Google Streetview

New Roads service board and mapping

New River service board

Emirates Airline service board

Trams service board

New content throughout

Localisation

New cloud hosting – more resilient and auto-

scaled for capacityLower overall hosting costs

Varnish for intelligent caching

Normalised API for efficient development and open data

Faster live info – 5m to 30s

Integrated fares in Journey Planner

New ‘Nearby’ tool

Integration of all forms of travel information for the first time

Platform for further development, single sign-on,

TDM, Contactless etc

MVC approach

HTML5 schematic mapping

Personalisation


Key outcomes - infrastructure

• Solid availability and performance

• Sustained massive load through April Tube strikes (1 million pages

an hour, plus all syndicated load)

• Ease of standing up environments proved

• Autoscaling and blue/green deployment working well

• Substantial savings on operating costs

• Now extending to other services – Journey Planner and wider as

appropriate

AWS Government, Education, and Nonprofits SymposiumLondon | 21 October 2014 69

• Journey Planner usage up 10% with highest ever number of visits (10

million) in April

• Mobile usage has overtaken desktop usage since launch

• Customer satisfaction from April survey stood at 87%

• 76% of Londoners are using the site, 20 million visits a month

• 12 industry awards so far (and counting)

Key outcomes - customers


What we learned

• Benefits of scripting and autoscaling are very significant, even though

it’s complex

• Specific cloud experience makes things much faster

• An integrated, co-located team of product development and

infrastructure really helps

• Need a mindset which allows for agility and can accommodate

change

• This type of transformation is not easy – but worthwhile, requires a

bold approach



Thank YouPHIL YOUNG

Head of TfL Online

[email protected]


Accelerating Research

University of Chicago needed a cost-effective way to provide

big-data analysis to labs around the world while providing

always-on service. The university now hosts its Globus

Transfer service on AWS, helping more than 12,000 users to

move data with 99% availability.

“AWS has helped us scale up and

lower the cost of doing analysis.”- Ravi Madduri,

Research Fellow and Project Manager


Improve Education and Break the Mold

Migrated public facing web properties with large bursts of traffic from 38,000 visitors to

150,000 a day

Improved disaster recovery, handled major event spikes in usage and 40%

less expensive

Amazon Jisc Web Portal

21/10/14 Cloud Computing

Tim Marshall

CEO, Janet

Universities are migrating off campus

Benefits

»Sweating the network asset

»Options from colocation to cloud

»Larger storage facilities

»Agility

»Economies of scale

»National agreements

»Better use of campus estate

Jisc Amazon Cloud Computing 76

And have been for some time ...

But the cloud...

What about?

»Data Preservation

»Security

»Legal Issues

»Costs

»Logistics

»This is where Jisc adds even more…


Tier 1 Cloud Vendors

Janet Network

Microsoft

O365 &

Azure

Google

Apps &…

Amazon

AWS &

Glacier

• Approach is to gain sector sponsorship and involvement• Largest global providers with large sector market share • Broad applicability of service including for research, teaching & learning and enterprise• Long term investment and long term partnerships (including on technology)• Looking for standardisation• Working with Géant and Global Services Group

Data

location &

processing

Pricing &

Costs

Data Egress

/ bandwidth

Federated

Identity

NREN

Aggregation

What sector issue did we seek to solve?Individual users with:

• No protection

• No aggregation benefits

• No institution awareness or control

Organisational Issues

1. Where is your data?

2. Are you compliant?

3. Are you legal?


Jisc & Amazon

Sector leadership:

»University of York

»University of Newcastle

»University of Cambridge

Commercial Partnering:

»Arcus

»Amazon


Building a national sector level portal

The Portal is now Live

The Portal offers:

»Federated login for quick easy access

»Monthly invoicing IN STERLING, no need to use a credit card

» Itemised billing to consolidate costs across users and departments

»The ability to set budget limits against accounts

»Volume discounts through aggregation across users


Academic institutions are beginning to benefit


“Frictionless access to Cloud Services that’s our job”Dan Perry Director of Product and Marketing - Jisc

Jisc

Google Apps

Data Archiving

IT Financial

X-Ray

Shared Data

CentresAmazon

Web Services

File Sync and

Share

Microsoft

Data Centre Frame-

work

Find out more ...

http://www.jisc.ac.uk/amazon-web-services

http://www.jisc.ac.uk/network/cloud

http://www.jisc.ac.uk/

http://community.ja.net/cloud

Title of presentation 00/00/2013 84

http://www.jisc.ac.uk/amazon-web-services

http://www.jisc.ac.uk/network/cloud

http://www.jisc.ac.uk/


Customer

obsession

1

Invention

2

Long-term

thinking

3

Some final thoughts about Amazon…

A Faster, Cheaper and Better

Path to a Better World



Thank YouTeresa Carlson

Keynote: Paving the Way and Making a Difference: AWS in the Public Sector

Technology

Transcript of Keynote: Paving the Way and Making a Difference: AWS in the Public Sector