Resource Sharing Across Users Resource Sharing Across Users in Server Clustersin Server Clusters

Krithi Ramamritham

IIT Bombay

krithi@iitb.ac.in

Optimizing and ScalingOptimizing and ScalingEnterprise ApplicationsEnterprise Applications

Enterprise (Information) SystemsEnterprise (Information) Systems

• Any kind of computing system that is of "enterprise class"

– offering high quality of service– dealing with large volumes of data– capable of supporting a large organization -- “an enterprise“

• Enterprise Information Systems – provide a technology platform that enables organisations to

integrate and coordinate their business processes.

– provide a single system that is central to the organisation.

– ensure that information can be shared across all functional levels and management hierarchies.

– help eliminate the problem of information fragmentation caused by multiple information systems in an organisation.

• Enterprise applications are constructed using a multi-tier architecture for simplified development and maintenance

• Considerable time and money is invested in the server infrastructure

• A significant amount of developer time is being spent to optimize Web applications

WebServerLayer

ApplicationServerLayer

Database

Request

Content

Optimizing and Scaling Enterprise Applications…Optimizing and Scaling Enterprise Applications…

WS vs. ASWS vs. AS

• Web servers– Do well defined and quantifiable local work

• e.g., processing HTTP headers, serving static content

• Application servers– Run multi-layer programs

• e.g., scripts involving calls to backends

… …

WebSwitch

WebServerCluster

ApplicationServerCluster

… …

WebSwitch

WebServerCluster

ApplicationServerCluster

Inside the Application LayerInside the Application Layer3-tier model3-tier model

PRESENTATION

BUSINESS LOGIC

DATA CONNECTOR

HTML

Objects

Row Set

• JDBC• ODBC

• Servlets• COM+• EJB

• JSP• ASP

LegacySystems

Databases

ADDT’LSERVICES

• Commerce• Content Mgt.• Personalization

Inside the Application Layer…Inside the Application Layer…

PRESENTATION

BUSINESS LOGIC

DATA CONNECTOR

• JDBC• ODBC

...Code

Block(s)

...Code

Block(s)

LegacySystems

Databases

ADDT’LSERVICES

• Commerce• Content Mgt.• Personalization

1. JSP invokes a Servlet2. Servlet contacts CMS

3. CMS requests data

4. DBMS calls storage system

Performance and Performance and Scalability IssuesScalability Issues

• Computationally-intensive logic executed atmultiple tiers

• Cross-tier communication

• Object instantiation and cleanup processing

• External I/O calls

• Database connection pool latencies

• Content conversion and formatting

Optimizing the Application LayerOptimizing the Application LayerTraditional MeansTraditional Means

• Optimize each tier independently:– Presentation-level caches built inside application server

processes– Main memory database employed over persistent DBMS– Persistent object storage techniques employed inside

content management systems … and so on

PRESENTATION

BUSINESS LOGIC

DATA CONNECTOR

• JDBC• ODBC

• Servlets• COM+• EJB

• JSP• ASP

ADDT’LSERVICES

Local cacheand optimization

code

Query result cachingQuery result caching

• Many application server products offer this feature

-- mitigates only local database access latency-- only a subset of query results may be reused in

page generation-- page fragments may not all be from databases

Middle tier database cachingMiddle tier database caching

• Caching database tables in main memory

Oracle 9i CacheMain-memory databases, e.g., TimesTen

-- mitigates only database access latency

-- caching at table granularity results in poor cache utilization

-- main-memory databases are difficult to integrate and maintain and can be expensive

Page Level CachingPage Level Caching

• Dynamically generated HTML pages are cached

+ Can completely offload work from web/app server– Low reusability for highly personalized web pages– URL may not uniquely identify a page -- increasing the risk of delivering incorrect pages– Often introduces excessive invalidations -- e.g., even if a single element on the page changes

Optimizing the Application LayerOptimizing the Application LayerIssuesIssues

• Traditional techniques impact specific components within the application, but not the entire application

– No mitigation of component-to-component interaction latencies

– Different synchronization and invalidation policies risk data integrity

– Each optimization scheme consumes programmer timefor development and maintenance

Key ideasKey ideas

• Re-use program results to eliminate redundant work

• Facilitate single-point, architecture-wide optimization

Apply to both programmatic objects and result fragments

Optimizing the Application LayerOptimizing the Application Layer

PRESENTATION

BUSINESS LOGIC

DATA CONNECTOR

• JDBC• ODBC

• Servlets• COM+• EJB

• JSP• ASP

LegacySystems

Databases

ADDT’LSERVICES

• Commerce• Content Mgt.• Personalization

cache

Enables the resultsof programs to bere-used.

Usually….Usually….

LegacySystems

1. JSP invokes a Servlet

PRESENTATION

BUSINESS LOGIC

DATA CONNECTOR

• JDBC• ODBC

...Code

Block(s)

...Code

Block(s)

Databases

ADDT’LSERVICES

• Commerce• Content Mgt.• Personalization

2. Servlet contacts CMS

3. CMS requests data

4. DBMS calls storage system

Plus, at each step there are communication delays and logic processing delays

With Our Solution…With Our Solution…

PRESENTATION

BUSINESS LOGIC

DATA CONNECTOR

• JDBC• ODBC

...Code

Block(s)

...Code

Block(s)

Function Parameter(s) Result

Real-time storage engine

Tags trigger calls to the storage engine.

Can store any program output, but is most commonly an HTML fragment or a Programmatic Object.Chutney

tags

When the Result of a Function with a specific Parameter set is already known (and up-to-date), the work normally necessary to produce that Result is bypassed.

Appl. Programming Interface

Cache ManagementCache Management

• A critical aspect of any caching solution

• Support novel cache management strategies:

– Prediction-based cache replacement– Observation-based cache invalidation

Cache ReplacementCache Replacement

• Prediction-based replacement⁻ fragments having lowest

probability of access replaced⁻ Least-Likely-to-be-Used (LLU)

– Access probabilities based on:• Current user navigational

patterns over site graph (in the form of clickstreams)• Historical user navigational

patterns over site graph (in the form of association rules)

News

Sports

Hockey

Schedules ScoresPlayers Teams

Site Graph

(News, Sports, Hockey) Schedules = 20%

(News, Sports, Hockey) Players = 15%

(News, Sports, Hockey) Teams = 10%

(News, Sports, Hockey) Scores = 55%

LLU

Cache InvalidationCache Invalidation

Need to support common cache invalidation techniques:

– Time-based: Each cache element assigned a TTL– Event-based: Updates to the database send an

invalidation message to the cache– On demand: Manual invalidation of selected

elements– ….

Cache Invalidation…Cache Invalidation…

• Other invalidation techniques supported:– Observation-based

• User-initiated updates are observed in scripts; each such update sends an invalidation message to the cache

• Most appropriate for auction sites, online trading sites

• Invalidation does not require communication with the databases

– Keyword-based: • Elements can be associated with keywords; e.g., a

retailer may wish to invalidate all “seasonal” items– Regular expression-based:

• Elements can be invalidated based on regular expression matching

Other Fragment Level Caching…Other Fragment Level Caching…

+ can offload presentation layer tasks

– runs in the application server process space

=> competes for server resources

– application server cluster

=> multiple cache instances,

duplication of content,

additional synchronization overhead

app servers (e.g., BEA’s WebLogic, IBM’s WebSphere) cache fragments produced by JSP scripts

ApplicationServerCluster

Performance Study…Performance Study…

Test Site

– Fictitious online retail site, allows browsing of product catalog

– Pages generated using JSP scripts– Site content stored in Oracle database– Database schema based on Dublin Core Metadata

Open Standard– Contains 200,000 products and 44,000 categories– Each page consists of 3 components, each involving a

database call

Performance Study…Performance Study…

Test Setup

– Content Database Server: Oracle 8.1.6

– Web/Application Server: WebLogic 6.0 running on cluster of 2 machines

– Server machines:have 1 GB RAM, dual P III-933 Mhz processorsrun Windows 2K Advanced Server

Testing Methodology...Testing Methodology...

• Baseline Parameters:

– Cache Size, i.e., percentage of fragments that fit into cache: 75%

– Cache replacement policy: LLU

• User load is varied by sending requests from client machines running Radview’s WebLoad

• Simulated users navigate site according to Zipf 80-20 distribution (i.e., 80% of users follow 20% of navigation links)

Performance ImpactPerformance Impact

80% faster response times through existing application infrastructure

Source: Fortune 100 client results

0

10

20

30

40

50

60

0 100 200 300 400 500

Number of Users

Ave

rag

e R

esp

on

se T

ime

(sec

on

ds)

non-Chutney

Chutney

Chutney Throughput ImpactChutney Throughput Impact

250% increase in transaction rates

Source: Fortune 100 client results

0

100

200

300

400

500

600

700

0 100 200 300 400 500

Number of Users

Tran

sact

ion

s P

er S

eco

nd

non-Chutney

Chutney

Broad InteroperabilityBroad Interoperability

Chutneycache

Java-basedJSP, Servlets, EJB, BEA WebLogic, IBM WebSphere,

iPlanet, Broadvision, etc.

Microsoft-basedASP, COM,

IIS, MS TransactionServer, etc.

OtherColdFusion,

Perl, etc.Multi-server, heterogeneous environments can interface with a single storage engine.

Presentation

BusinessLogic

Data

Presentation

BusinessLogic

Data

Presentation

BusinessLogic

Data

Alternative: CDNsAlternative: CDNs

Sources

Repositories

Clients

ContentDistributionNetworks

e.g., Akamai

Push BasedPush BasedCore InfrastructureCore Infrastructure

Request Distribution Request Distribution within Clusterswithin Clusters

… …

WebSwitch

WebServerCluster

ApplicationServerCluster

… …

WebSwitch

WebServerCluster

ApplicationServerCluster

Maximizing affinity

Exploit application characteristics

SummarySummary

• Bottlenecks persist throughout multi-tier architectures

• Traditional optimization approaches focus on individual components, not the entire application

• Need a solution which optimizes every tier of a web application, globally