Keynote: Paving the Way and Making a Difference: AWS in the Public Sector

AWS Government, Education, and Nonprofits SymposiumLondon | 21 October 2014

AWS Government, Education, and Nonprofits SymposiumLondon I 21 October 2014

AWS in the Public Sector

Teresa CarlsonVice President

World Wide Public Sector

Strong growth and adoption

government

agencies

educational

institutions

10000+

nonprofit

organizations

Used by Government Agencies & Educational Institutions

Worldwide

AWS Partners Focused on Public Sector

AWS Rapid Pace of Innovation

Amazon RDS

Amazon VPC

Auto Scaling

Elastic Load

Balancing

Amazon SNS

AWS Identity

& Access

Management

Amazon Route 53

Amazon

ElastiCache

Amazon SES

CloudFormation

AWS Direct

Connect

AWS Elastic

Beanstalk

GovCloud

Amazon

CloudTrail

Amazon

CloudHSM

Amazon

WorkSpaces

Amazon Kinesis

Amazon Elastic

Transcoder

Amazon

AppStream

AWS OpsWorks

Amazon SWF

Amazon Redshift

Amazon Glacier

Amazon

Dynamo DB

Amazon

CloudSearch

AWS Storage

Gateway

AWS Data

Pipeline

Since inception AWS has:

• Released 942 new services and features

• Introduced over 35 major new services

• Announced 45 price reductions

+24Amazon EBS

Amazon

CloudFront

Amazon Cognito

Amazon Zocalo

Amazon Mobile

Analytics

*as of Aug 18, 2014

Amazon

ElasticCache

reduces prices for

cache nodes by an

average of 34%

March 26, 2014

34%Amazon S3 reduces

prices for Standard and

Reduced Redundancy

Storage, by an average

of 51%

March 26, 2014

We’ve announced price reductions 45* times since

our inception in 2006. Recent price drops included…

*as of July 31, 2014

Amazon Route 53

lowers prices for both

Standard Queries and

Latency Based

Routing Queries by

July 31, 2014

Architected for Enterprise Security RequirementsCertifications and accreditations for

workloads that matter

AWS CloudTrail - AWS API call

logging for governance & compliance

Stores data in

S3, or archive

to Glacier

Log and review

user activity

AWS Helps Government Cut Costs

“We are in the process of putting most of our public-facing

data in an Amazon cloud service,” said Terry Halvorsen,

the Chief Information Officer of the Department of the

Navy, in a keynote at Meritalk’s Data Center Brainstorm

event Thursday. Halvorsen said the move could save

the Navy as much as 60 percent versus the cost of

managing that data in its own data centers.-Data Center Knowledge,

March 14, 2014

AWS Global Infrastructure

10 Regions

including GovCloud US

26 Availability

51 Edge Locations

Why Does the Cloud Matter to Public Sector?

Pave the Way for

Disruptive Innovation

Make the World

a Better Place

Paving the way

What do public sector customers need?

• Disruptive innovation

• Agility

• Capability

• Cost savings

Evolution of cloud adoption in government

Policy

Phase 0 – Stated Policy

“Cloud First”

DefinitionPolicy

Phase 1 – Standard Definition

Security and Compliance

DefinitionPolicy

Phase 2 – Security & Compliance

Procurement

DefinitionPolicy

Phase 3 – Cloud-Friendly Procurement

Department of Interior

Foundation Cloud

United Kingdom G-Cloud

Dept. of Treasury Public Cloud Web Hosting Services

US Communities

Texas Dept. of

Information Resources contract

Navy SPAWAR

US Intelligence Community

DefinitionPolicy

Procurement

Culture

Procurement

Culture

Broad Adoption

DefinitionPolicy

Paving the way for…

Real change

Make The World a Better Place

• Support world-changing projects

• Enable economic development

• Improve citizen services and engagement

• Improve research and education

When You’re Changing the World, You Can’t Afford to be Slow…

Add New Dev Environment

Add New Prod Environment

Go Global in Seconds

Add 1,000 Servers

Remove 1,000 Servers

Deploy 1 PB Data Warehouse

Shut down 1 PB Data Warehouse

Resources in MinutesOld World:

Resources in Weeks

Everything changes with this

kind of agility and speed

Increased agility has

become the #1 reason

organizations use the AWS

1. Support world-changing projects

2. Enable economic development

3. Improve citizen services and engagement

4. Improve research and education

What better way to change the world…

Code.org ran a worldwide

“Hour of Code”

More than 20 million youth

coded on their website in a

single week, with a peak load of

330,000 concurrent users

“Running on the AWS Cloud

gave us the elasticity to keep

the website running when

traffic spiked from zero to 20

million coders during

campaign week, and then

scale back efficiently. AWS

was fantastic.”-Geoffrey Elliott, Code.org

Ensuring democracy…

Migration of core business applications for secure global

access, reduced costs, focused resources and improved

availability

"The driver really was the ability to

be responsive. The way we did that

was to move to the cloud.“

-Chris Spence, CIO

National Democratic

Institute

Helping Nonprofits Achieve Scale and Reach

"Working hand in glove with AWS,

LeanIn.Org's development team was

able to re-architect the site over a 64-

hour period to ensure it would stand

up to 200 click-throughs per second."

-Rachel Thomas

President, Lean In

“Ban Bossy” campaign

Featured on Google home page and

stayed up!

Using Analytics to Understand the World

“Do the simplest

thing,” but do it at a

stupidly large scale.”

-David Milne, a postdoctoral

research fellow at CSIRO

European Bioinformatics Institute

SureChEMBL: Patient Data in the Cloud

Mark Davies, Technical Lead

mdavies@ebi.ac.uk

What is EMBL-EBI?• Part of the European

Molecular Biology Laboratory

• International, non-profit research institute

• Europe’s hub for biological data services and research

• 500 members of staff from 53 nations.

ChEMBL – Data for Drug Discovery

Bioactivity data

Compound

>Thrombin

MAHVRGLQLPGCLALAALCSLVHSQHVFLAPQQARSLLQRVRRANT

FLEEVRKGNLERECVEETCSYEEAFEALESSTATDVFWAKYTACET

ARTPRDKLAACLEGNCAEGLGTNYRGHVNITRSGIECQLWRSRYPH

KPEINSTTHPGADLQENFCRNPDSSTTGPWCYTTDPTVRRQECSIP

VCGQDQVTVAMTPRSEGSSVNLSPPLEQCVPDRGQQYQGRLAVTTH

GLPCLAWASAQAKALSKHQDFNSAVQLVENFCRNPDGDEEGVWCYV

AGKPGDFGYCDLNYCEEAVE

3. Insight, tools and resources for translational drug discovery

2. Organization, integration, curation and standardization of pharmacology data

1. Scientific facts

Who works with ChEMBL?

https://www.ebi.ac.uk/chembl/

Patent Data and Drug Discovery

• Historically a closed and costly data source

– Out of reach to many academics and SMEs

• Patent literature 2-3 years ahead of published literature

• Prior art and freedom to operate

• Competitor intelligence

• Provides access to lots more data

– High cost to extract and lots of noise

SureChEMBL – Open Patent Data

• December 2013 EMBL-EBI acquired SureChem

• A leading chemistry patent mining product from

Digital Science, Macmillan Group

• Commercial to open transition ‘perfect fit’ for both

EMBL-EBI and Digital Science

• System rebranded SureChEMBL and migrated to

EMBL-EBI resources

SureChEMBL Migration Challenge

• SureChem was an AWS based system, should we stay or

should we go?

– Live system with many dependencies

– Supporting existing customers during migration

– Short timescale

– Limited resources

– Limited budget

– Existing AWS experience

We stayed with AWS

SureChEMBL System

Keyword

searches

Chemistry

searches

Access to

chemistry

https://www.surechembl.org

Complex

filters

Access to

patents

SureChEMBL AWS Resources

Elastic Load

Balancing

Amazon

Elastic

Amazon

Snapshot CloudWatch IAM

Amazon SQS

AWS Benefits

• AWS provides a comprehensive suite tools and services

– As a developer, everything required to build system is just there

• Fast design, build, deploy and teardown cycle

• Web based AWS console

• Use programmatic access to automate tasks and monitor system

– CLI and multiple SDKs available

• CloudWatch alarms to monitor system

• Cost calculator

• New services being added all the time

• Excellent support

– Account manager, solution architects, online forums

AWS Recommendations

• Make use of excellent online documentation

– Many whitepapers and best practice guidelines

• Use VPC to architect system

– Refresh network knowledge

– Tricky to migrate from ‘Classic’ EC2

• Review costs

– Use the cost calculator

– Use reserved instances

– Monitor storage: RAIDs, backups, snapshots, storage medium, logs

– Shutdown resources when not being used

• Prepare for unexpected events

– Check the ‘Events’ section EC2 dashboard

SureChEMBL Migration Success19th September 2014 SureChEMBL system released

https://www.surechembl.org

AcknowledgementsChEMBL team• John Overington

• Jon Chambers

• George Papadatos

• Mark Davies

• Nathan Dedman

• Anna Gaulton

Digital Science• Nicko Goncharoff

• James Siddle

• Richard Koks

Funding:• Wellcome Trust Strategic Award for

ChEMBL database (WT086151/Z/08/Z &

WT104104/Z/14/Z)

• Open PHACTS - Innovative Medicines

Initiative Joint Undertaking (grant no.

115191)

• European Molecular Biology Laboratory

• BioMedBridges - European Commission

FP7 Capacities Specific Programme (grant

no. 284209)

Technology Partners:

Thank YouMark Davies, Technical Lead

mdavies@ebi.ac.uk

Public/private engagement for

economic development

• Geospatial collaborative

environment

• 80+ participating agencies

• 30+ new services

developed

• 100M+ users of the service

Public/private engagement for

economic development

“AWS has helped my organization to

provide better service availability

and handle higher traffic load at a

lower cost”

-Chan Chin Wai, CIO, SLA

June 2014 Competition

Finalists and Winners

Partners in Innovation

Best Practices Award

Best Practices Award Winners

Turning Government Data into Real Insight

Scalable web application Big Data Analytics

and Collaboration

Rapid deployment of analytics

engine

Redesigned portions of

Healthcare.gov

Post “flash crash”

forensics on EC2

Collaboration

platform for SEC

Mining social media for

early warnings of food and

drug safety issues on

accelerated timeline

Healthcare.gov

Taking Transport for

London into the cloud

Phil Young

Head of TfL Online

philyoung@tfl.gov.uk

• Background TfL and our Online services

• Choosing the cloud what drove our selection

• Making it happen build and commissioning

• Outcomes our experiences so far

Overview

About TfL• London’s integrated transport authority

• Responsible for all forms of transport –Tube, buses, roads and

congestion charging, trams, river, DLR, Overground, cycling,

walking, coaches, freight, taxis and private hire and more

• Around 24 million journeys daily on our network

• Planning and developing for the future, new services to meet the

demands of an increasing population

• Iconic brand, embedded into the fabric of our city

What our customers want

Understand what we stand for

Value for MoneyProgress &

Innovation

Excellent

reliability and

customer

experience

TfL web visits 2006-2014

Total Visits

Legacy issues

• Previous site not refreshed since 2007

• Met core functional needs, but required user effort for more complex

tasks (eg finding maps, logins)

• No longer met customer expectations for

localisation, personalisation, mapping

and journey planning

• Poor experience on mobiles and tablets

Requirements

Customer

TechnicalBusiness

The new site needed to meet

customer needs first, while delivering

on business objectives and being

technically deliverable

1. Personalised information

2. Fast realtime info on-the-move

3. Mobile first, great on all devices

4. One stop shop for planning and info

5. Improved visual appearance

6. Humanised interaction

7. Single log-in

8. Integrated journey planning tools

9. Trusted information

10. Simplified experience

11. Consistent experience of all of TfL

12. Easy access to customer service

13. Reassurance around travel decisions

Key customer requirements

Our vision of a new TfL site

CHOOSING THE CLOUD

Platform requirementsFunctional:

• Personalisation and localisation

• Separate presentation layer

• Disparate data sources

• Integrated and consistent data

needed

• CMS and web application

integration

• Responsive design

Performance:

• Scalable - 30x spikes

• 24 hour 100% availability

• Fast response times

• Accommodate limited back-end

systems

• Highly secure

• Deal in realtime services

• Cost effective

Choosing the cloud

• Elastic capacity

• Speed and flexibility, automation

• Cost

• Resilience and referencability

AWS selected because:

• Best match at the time for our multi-OS stack

• Cost-effective, usage-based approach

• Rapidly developing platform with regular new features and falling costs

Challenges of delivery

• Product evolving through an agile programme as platform being built

• In house team and multi-supplier environment

• Challenging timescales to move from experimental to production

• Lack of cloud experience

• Platform maturity – still developing

• Internal scepticism

OUTCOMES

What we delivered

• A new API with a canonical data model

• Complex web application through MVC

• Multiple environments, ‘push button’ creation and product deployment

• Autoscaling

• Blue/green release approach

• Varnish caching layer

• Web application firewall

New from the ground-up

Responsive design

Mobile first

New information architecture

New search

New Journey Planner

Google maps throughout

Google Streetview

New Roads service board and mapping

New River service board

Emirates Airline service board

Trams service board

New content throughout

Localisation

New cloud hosting – more resilient and auto-

scaled for capacityLower overall hosting costs

Varnish for intelligent caching

Normalised API for efficient development and open data

Faster live info – 5m to 30s

Integrated fares in Journey Planner

New ‘Nearby’ tool

Integration of all forms of travel information for the first time

Platform for further development, single sign-on,

TDM, Contactless etc

MVC approach

HTML5 schematic mapping

Personalisation

Key outcomes - infrastructure

• Solid availability and performance

• Sustained massive load through April Tube strikes (1 million pages

an hour, plus all syndicated load)

• Ease of standing up environments proved

• Autoscaling and blue/green deployment working well

• Substantial savings on operating costs

• Now extending to other services – Journey Planner and wider as

appropriate

AWS Government, Education, and Nonprofits SymposiumLondon | 21 October 2014 69

• Journey Planner usage up 10% with highest ever number of visits (10

million) in April

• Mobile usage has overtaken desktop usage since launch

• Customer satisfaction from April survey stood at 87%

• 76% of Londoners are using the site, 20 million visits a month

• 12 industry awards so far (and counting)

Key outcomes - customers

What we learned

• Benefits of scripting and autoscaling are very significant, even though

it’s complex

• Specific cloud experience makes things much faster

• An integrated, co-located team of product development and

infrastructure really helps

• Need a mindset which allows for agility and can accommodate

change

• This type of transformation is not easy – but worthwhile, requires a

bold approach

Thank YouPHIL YOUNG

Head of TfL Online

philyoung@tfl.gov.uk

Accelerating Research

University of Chicago needed a cost-effective way to provide

big-data analysis to labs around the world while providing

always-on service. The university now hosts its Globus

Transfer service on AWS, helping more than 12,000 users to

move data with 99% availability.

“AWS has helped us scale up and

lower the cost of doing analysis.”- Ravi Madduri,

Research Fellow and Project Manager

Improve Education and Break the Mold

Migrated public facing web properties with large bursts of traffic from 38,000 visitors to

150,000 a day

Improved disaster recovery, handled major event spikes in usage and 40%

less expensive

Amazon Jisc Web Portal

21/10/14 Cloud Computing

Tim Marshall

CEO, Janet

Universities are migrating off campus

Benefits

»Sweating the network asset

»Options from colocation to cloud

»Larger storage facilities

»Agility

»Economies of scale

»National agreements

»Better use of campus estate

Jisc Amazon Cloud Computing 76

And have been for some time ...

But the cloud...

What about?

»Data Preservation

»Security

»Legal Issues

»Costs

»Logistics

»This is where Jisc adds even more…

Tier 1 Cloud Vendors

Janet Network

Microsoft

O365 &

Google

Apps &…

Amazon

Glacier

• Approach is to gain sector sponsorship and involvement• Largest global providers with large sector market share • Broad applicability of service including for research, teaching & learning and enterprise• Long term investment and long term partnerships (including on technology)• Looking for standardisation• Working with Géant and Global Services Group

location &

processing

Pricing &

Data Egress

/ bandwidth

Federated

Identity

Aggregation

What sector issue did we seek to solve?Individual users with:

• No protection

• No aggregation benefits

• No institution awareness or control

Organisational Issues

1. Where is your data?

2. Are you compliant?

3. Are you legal?

Jisc & Amazon

Sector leadership:

»University of York

»University of Newcastle

»University of Cambridge

Commercial Partnering:

»Arcus

»Amazon

Building a national sector level portal

The Portal is now Live

The Portal offers:

»Federated login for quick easy access

»Monthly invoicing IN STERLING, no need to use a credit card

» Itemised billing to consolidate costs across users and departments

»The ability to set budget limits against accounts

»Volume discounts through aggregation across users

Academic institutions are beginning to benefit

“Frictionless access to Cloud Services that’s our job”Dan Perry Director of Product and Marketing - Jisc

Google Apps

Data Archiving

IT Financial

Shared Data

CentresAmazon

Web Services

File Sync and

Microsoft

Data Centre Frame-

Find out more ...

http://www.jisc.ac.uk/amazon-web-services

http://www.jisc.ac.uk/network/cloud

http://www.jisc.ac.uk/

http://community.ja.net/cloud

Title of presentation 00/00/2013 84

Customer

obsession

Invention

Long-term

thinking

Some final thoughts about Amazon…

A Faster, Cheaper and Better

Path to a Better World

Thank YouTeresa Carlson

Keynote: Paving the Way and Making a Difference: AWS in the Public Sector

Technology

Transcript of Keynote: Paving the Way and Making a Difference: AWS in the Public Sector

AWS Summit Nordics - Opening Keynote

KGC 2013 AWS Keynote

AWS Innovate 2016 : Opening Keynote - Glenn Gore

ABOF keynote session @ AWS Summit 2016

Aws microservice keynote

IPM | Paving the path to a modern AIESEC | Keynote

AWS Summit London - Keynote - Stephen Schmidt

Closing keynote migrating amazon.com to aws

AWS Security – Keynote Address (SEC101) | AWS re:Invent 2013

Keynote AWS Summit 2013 Sao Paulo

AWS Summit Berlin 2013 - Keynote Steve Schmidt

AWS Summit Berlin 2013 - Keynote - 6wunderkinder

AWS Summit Nordics - Security Keynote

AWS Summit Milan - Capire la Sicurezza Keynote

AWS Summit Milan - Opening Keynote

AWS Summit Paris - Keynote Slides

AWS Summit London 2016 Keynote

AWS IoT Workshop Keynote

AWS DC Symposium Keynote: Teresa Carlson

Keynote AWS RoadShow Belo Horizonte 2013