The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decision. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Safe Harbor Statement

<Insert Picture Here><Insert Picture Here>

PeopleTools Integration Broker Performance TuningVictor LiPrinciple Support Engineer, PeopleTools support, Oracle GSS

Agenda

• Overview PeopleSoft integration broker Architecture

• Performance tuning for asynchronous messaging

• Performance tuning for synchronous messaging

Some issues to think

• Production system running fine for years and Integration Broker usage is low. Suddenly Application Server box’s CPU reaches 90%. Further troubleshooting reveals PSSUBDSP service causes it.

• We only use PSRF* messages, ie. Reporting tools messages before. This week, we have started to implement dynamic role sync messages. Now messages stuck in working status.

• We have a third party web interface sending sync messages to PeopleSoft web services. We have load balanced integration gateway. Web Server box and App server box are powerful. No queues in Tuxedo monitor. How can we improve the benchmark performance of message response time?

Architecture

“PeopleSoft 9.1 combines all the latest features customers expect in a web-based application with the tools needed to extend it into the future.”

Larry Acushnet Company

Architecture

Third PartySystemWeb Server

ApplicationServerRDBMS PeopleSoft Application

Outbound Operation

ApplicationServerRDBMS PeopleSoft Application

Inbound Operation

Integration Engine

Integration Gateway

ArchitectureIntegration Broker - detailed

ArchitectureAsynchronous Operation

ArchitectureSynchronous Operation

Local to local transaction

Asynchronous & Synchronous Service Operation“PeopleSoft 9.1 combines all the latest features

customers expect in a web-based application with the tools needed to extend it into the future.”

Larry Acushnet Company

Message segments

• Large amounts of data(gigabytes)• Divide rowset-based and nonrowset-based messages

into segments

Message segments

• Configuration and coding efforts are small and straightforward.

Total Time for 40 MB Transfer

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

1 2 3 4 5 6

Iterations

Se

co

nd

s Messages

Ordered Segments

Unordered Segments

Thread Pool Size--Asynchronous

• The number of concurrent threads the handler can spawn for HTTP requests

• Only for PSPUBHND handlerSeconds/Message

0.000

0.200

0.400

0.600

0.800

1.000

1.200

1.400

1

1 thread

5 threads

10 threads

15 threads

20 threads

[PSPUBHND_dflt];===============================================; Settings for publication contract handler;===============================================Max Instances=1Service Timeout=1200Recycle Count=20000Allowed Consec Service Failures=0; Thread Pool Size. The Minumum value is 1 and maximum allowable is 20.Thread Pool Size=1

Thread Pool Size--Synchronous

• The number of concurrent threads the handler can spawn for HTTP requests

• Default is 5.Avg SyncRequest Time

0.0

2.0

4.0

6.0

8.0

10.0

12.0

1

Thread Pool SizeE

lap

sed

Tim

e (

sec)

5 Threads

10 Threads

15 Threads

20 Threads

Integration Gateway-Load Balancing

• Load Balancer in front of Gateways.• JVM--MinimumHeapSize and MaximumHeapSize to

the same value, eg. 512MB.

Domain 5

Pub/SubSlave

Domain 4

Pub/SubSlave

Domain 6

Pub/SubSlave

Pub/SubMaster

Domain 2

Pub/SubMaster

Domain 1

Failover

Pub/SubMaster

Domain 3

Failover

ApplicationServerApplication

Server

ApplicationServerApplication

Server

ApplicationServerApplication

Server

ApplicationServerApplication

Server

ApplicationServer

Load Balancer

Gateway

Gateway

Gateway

Gateway

GatewayLoad Balancer

Asynchronous Service Operation

“PeopleSoft 9.1 combines all the latest features customers expect in a web-based application with the tools needed to extend it into the future.”

Larry Acushnet Company

Dedicated messaging server

• Dedicated server for high message volume • Dedicated server domains should have PSAPPSRV

configured in addition to pub/ sub servers. • Dedicated messaging servers are assigned a certain

number of queue names

Multiple domains

• PUB/SUB domain separates from PIA domain• Scan Interval parameter to a value of 1 for each

dispatcher in pub/sub server.• Only one active pub/sub domain is supported.

Domain Failover

• Configure Failover

• Dynamic Slave Option

Master/Slave Load Balance

• Compensate for processing capabilities of various machines.

• Two hyperlinks Master/Slave Load Balance and Slave Template Setup.

Master/Slave Load Balance

• Assign weighting based on the capability of the machine

Master/Slave Load Balance--Deferred Master Domain Processing

• “Enabled” –Default. Master will process its appropriate share of requests

• “Deferred – All Queues” --One active slave• “Deferred – Unordered Queues” -Unordered queue

+one active slave

Slave template

Slave template

Multi-queue

• Unordered queues assigned to same service operation• Use with master/ slave domains, dedicated messaging

server.

Multi-queue

Application Engine Handler

• Problem: • A long running message takes up all handler resources

• Solution:• Offer a Service Operation Handler that is a direct initiation of

an AppEngine program• Offload the intense processing of the message to the Process

Scheduler• Guarantee ordering of message

• Message must be asynchronous

Bulk Load Handler

• Principle of Data Mover script as notification handler• Asynchronous messages only• Great for loading data to staging table

Guidelines

• Partition queue • Notification PeopleCode • Component Interfaces• Message Compression

Basic performance tips---server instance

• How many messaging server: • At least 3 in production

[PSSUBHND_dflt]

;==============================================

; Settings for subscription contract handler

;==============================================

Min Instances=3

Max Instances=3

Basic performance tips---messaging tables

• For asynch messages, they are stored in PSAPMSGPUBHDR, PSAPMSGPUBCON, PSAPMGSSUBCON and etc.

• Archive them regularly

Batch processing or full sync EIP

• Database Tuning--Temp table indexing• Archiving—N in queue definition.• Database Layout –tablespaces• Chunk messages -Around 3 MB a message.• Message segment

Synchronous Service Operation

“PeopleSoft 9.1 combines all the latest features customers expect in a web-based application with the tools needed to extend it into the future.”

Larry Acushnet Company

Multi-Threading Synchronous Requests

• (Array of Response Messages = %IntBroker.SyncRequest(Array of Request Messages))

Tunning Application Server

• Provider of Synchronous Service Operations—Dedicated Application server domain

• Jolt Handlers• [JOLT Listener]• ;=========================================================================• ; Settings for JOLT Listener• ;=========================================================================• Address=%PS_MACH%• Port=9000• Encryption=0• Min Handlers=60• Max Handlers=80• Max Clients per Handler=20

Basic performance tips---server instance

• How many psappsrv instances : • At least 3 in production[PSAPPSRV]

;==============================

; Settings for PSAPPSRV

;================================

;-------------------------------------------------------------------------

; UBBGEN settings

Min Instances=3

Max Instances=3

Guideline

• Response Time---Light transform, light onrequest peoplecode and light CI.

• Number of Synchronous Requests—Minimum.• Small Messages—compression.• Exception

Tools to determine and Reduce bottlenecks“PeopleSoft 9.1 combines all the latest features

customers expect in a web-based application with the tools needed to extend it into the future.”

Larry Acushnet Company

Application Server layer

• Tuxedo Commands• Pq--shows the current Tuxedo queuing on the domain.• Psr--current service request for each server on the domain

and also the load (number of requests) that the server processed.

• Tuxedo Settings• Message Segment Size---Increase size from the default 24 to

8K or 16K• Message Queue Size---Increase from 64K to 1M or 2M• Max Message Size---Increase from 64K to 1M or 2M

Profile and Statistics

Some issues to think

• Production system running fine for years and Integration Broker usage is low. Suddenly Application Server box’s CPU reaches 90%. Further troubleshooting reveals PSSUBDSP service causes it.

• We only use PSRF* messages, ie. Reporting tools messages before. This week, we have started to implement dynamic role sync messages. Now messages stuck in working status.

• We have a third party web interface sending sync messages to PeopleSoft web services. We have load balanced integration gateway. Web Server box and App server box are powerful. No queues in Tuxedo monitor. How can we improve the benchmark performance of message response time?

Q & A

THANK YOU