7343.RedPrairie Parcel and Microsoft SQL Server Benchmark White Paper (1)

7/27/2019 7343.RedPrairie Parcel and Microsoft SQL Server Benchmark White Paper (1)

1/21

Benchmark Testing Results: RedPrairie Parcel and Microsoft SQL Server i

Benchmark Testing Results:RedPrairie Parcel Running on SQLServerBenchmark testing demonstrates that SQL Server is an ideal database platform for RedPrairie Parcel

customers looking for performance, scalability, and value

Technical White Paper

Published: April 2012

Applies to: Microsoft SQLServer 2008 R2 Enterprise and SQLServer code-named Denali

Community Technology Preview 3 (CTP3)

Abstract

RedPrairie and Microsoft worked together to test the performance and scalability of RedPrairie Parcel

on the Microsoft SQL Server data management software and the Windows Server operating system. The

tests used industry-standard hardware to confirm a realistic total cost of ownership (TCO).

Parcel is a functionally rich software tool that helps customers shop for package-carrier rates and

manifest parcels at both the carton and shipment levels. In the performance and scalability tests, Parcel

was deployed on a variety of hardware and software configurations in both physical and virtual server

clusters. To gauge performance, the tests measured both the application throughput and the execution

time for a single package. The goal was to achieve throughput of at least 100,000 packages per hour

with a response time of less than one second, and to confirm scalability with an increasing number of

application servers.

The test results were impressive. The data shows that on an entry-level server with six cores and hyper-

threading enabled, it is possible to achieve more than 150,000 packages per hour. If the number of

cores is doubled, the throughput can be between 230,000 and 245,000 packages per hour. A more

powerful server will likely perform even better. The response times for all benchmark tests were

significantly below one second. Tests repeated on SQL Server code-named Denali Community

Technology Previous 3 (CTP3) showed nearly identical performance. The benchmark tests also

confirmed that adding physical or virtual servers up to the number tested can increase performance.

These results clearly demonstrate that RedPrairie Parcel customers can get excellent performance,

scalability, and value with Microsoft SQL Server.


2/21

Benchmark Testing Results: RedPrairie Parcel and Microsoft SQL Server ii

2012 Microsoft Corporation. All rights reserved. This document is provided as-is. Information and views

expressed in this document, including URL and other Internet Web site references, may change without notice.

You bear the risk of using it.

This document does not provide you with any legal rights to any intellectual property in any Microsoft product.

You may copy and use this document for your internal, reference purposes.


3/21

Benchmark Testing Results: RedPrairie Parcel and Microsoft SQL Server iii

Table of ContentsIntroduction .................................................................................................................................................. 1

Introducing RedPrairie Parcel ........................................................................................................................... 1The Benefits of SQL Server ................................................................................................................................ 1

Performance and Scalability Test Overview ................................................................................................... 3

Test Metrics ...................................................................................................................................................... 3

Performance and Scalability Goals ................................................................................................................... 3

Test Deployment Scenario ................................................................................................................................ 4

Performance and Scalability Test Configuration ............................................................................................ 4

Hardware and Software Used in Tests ............................................................................................................. 4

Database Servers .......................................................................................................................................... 5Application Servers ....................................................................................................................................... 6Load Balancer ............................................................................................................................................... 7Storage ......................................................................................................................................................... 7

Test Data .......................................................................................................................................................... 7

Performance and Optimization Tuning .......................................................................................................... 2

Database Server Optimizations .................................................................................................................... 2Application Optimizations ............................................................................................................................ 2

Performance and Scalability Test Methodology ............................................................................................. 3

Test Results and Conclusions ......................................................................................................................... 4

Multiple Application Server Scalability Analysis ............................................................................................... 4Physical Server Scalability Tests ................................................................................................................... 4Virtual Server Scalability Tests ..................................................................................................................... 5

Single Application Server Performance Analysis .............................................................................................. 7

SQL Server Version Analysis ............................................................................................................................ 10

Results of Additional Tests ............................................................................................................................. 11

Summary ..................................................................................................................................................... 11

Additional Information ................................................................................................................................ 12

About RedPrairie ............................................................................................................................................ 12

About Microsoft.............................................................................................................................................. 12


4/21

Benchmark Testing Results: RedPrairie Parcel and Microsoft SQL Server 1

IntroductionThe recent explosion in online ordering, coupled with inventory strategies that favor smaller, more

frequent orders, has greatly increased the importance of efficiently managing parcel shipments.

Companies find that consolidating shipments, selecting the optimal carriers, and securing the best

shipping rates are all critical to controlling costs.

Introducing RedPrairie ParcelRedPrairie Parcel is designed to support the complex shipment consolidation, rate shopping,

manifesting, and label printing requirements of todays high-volume shippers. Parcel centralizes

carrier rating, submission, and shipment. It is fully integrated with RedPrairie's Transportation

Management and Warehouse Management systems, making order fulfillment and shipping a single,

efficient process.

Parcel provides carrier rates, zone maps, carrier-compliant labels, and shipment documentation, in

addition to support for carrier business rules and interfaces to carrier systems for transmission ofshipping data. Parcel supports UPS, FedEx, DHL, and USPS carriers, in addition to less-than-

truckload (LTL), single, and mixed-class shipments and consolidation. Parcel provides true

enterprise-wide applicability for warehouse shipping requirements with the following capabilities:

Automation support

Parcel operates both as a workstation and within an in-line print-and-apply process (does

not require an operator).

Download/update domestic carrier rates

Parcel acquires advance published rates from UPS, FedEx, DHL, and USPS, from which it

creates and distributes the update files that are loaded into the system.

Carrier delivery status process

The integrated tracking functionality provided by the web-based RedPrairie Visibility

solution automatically queries for proof of delivery data.

The Benefits of SQLServerBecause shipping systems involve massive amounts of data, Parcel relies on the underlying support

of a high-performance database. SQLServer provides a reliable, cost-effective, low-maintenance

database framework for Parcel that can support the largest and most process-intensive

deployments. Hundreds of enterprises are currently running multi-terabyte transactional databaseson SQLServer.


5/21


Running Parcel on SQLServer 2008 R2 Enterprise provides customers with many benefits:

Delivers six nines (99.9999 percent) uptime availability.

RedPrairie Parcelcustomers can protect their mission-critical databases from downtimeand data loss with six nines uptime availability.

1,2

Faster to deploy.On average, SQLServer database administrators (DBAs) can install and configure new

database servers in 1.5 hours, while the largest competitors DBAs can take six hours.3

Lower hardware costs.

SQLServer can run on standard commodity server hardware, which can dramatically lower

the TCO for customers.

Lower software costs.

The list price of SQLServer is a third of the largest competitors;4

in addition, SQLServer

includes major database-related features, such as high availability; remote disaster

recovery; partitioning; data compression; transparent data encryption; spatial; master datamanagement; complex event processing; extract, transform, and load (ETL); online

analytical processing (OLAP); data mining; reporting services; and self-service business

intelligence (BI) tools. Competitors licensing models add costs for options and add-ins.5

Simpler systems management and lower staffing costs.

SQLServer database administrators (DBAs) can typically manage four times as many

physical databases as competitors' DBAs.6

Lower administration costs.

SQLServer delivers a 460 percent savings in annual costs of administration per database

over that of the largest competitor.7

Fewer security vulnerabilities.SQLServer has consistently had fewer security vulnerabilities than the largest competitor

solutions.8

With SQLServer, RedPrairie Parcel customers can save with reduced licensing, hardware,

administration, and support fees, which translate into substantially lower costs over the life of the

system.

1

http://www.microsoft.com/sqlserver/en/us/product-info/why-sql-server.aspx2http://www.microsoft.com/casestudies/Microsoft-SQL-Server-2008-R2-Enterprise/Stratus-Technologies/Protect-your-

mission-critical-databases-from-downtime-and-data-loss-with-six-nines-uptime-availability/40000071363

http://www.microsoft.com/sqlserver/en/us/product-info/why-sql-server.aspx4

http://www.microsoft.com/sqlserver/en/us/tools/cost-savings-calclator.aspx5

http://www.microsoft.com/sqlserver/en/us/product-info/competitor-compare.aspx6

http://www.microsoft.com/sqlserver/en/us/product-info/why-sql-server.aspx7http://www.microsoft.com/sqlserver/en/us/product-info/why-sql-server.aspx

8http://www.microsoft.com/sqlserver/en/us/product-info/why-sql-server.aspx
http://go.microsoft.com/fwlink/?LinkId=248585%20http://go.microsoft.com/fwlink/?LinkId=248585%20http://go.microsoft.com/fwlink/?LinkId=248585%20http://go.microsoft.com/fwlink/?LinkId=248586http://go.microsoft.com/fwlink/?LinkId=248585%20http://go.microsoft.com/fwlink/?LinkId=248585%20http://go.microsoft.com/fwlink/?LinkId=248585%20http://go.microsoft.com/fwlink/?LinkId=248585%20http://go.microsoft.com/fwlink/?LinkId=248585%20http://go.microsoft.com/fwlink/?LinkId=248585%20http://go.microsoft.com/fwlink/?LinkId=248585%20http://go.microsoft.com/fwlink/?LinkId=248585%20http://go.microsoft.com/fwlink/?LinkId=248585%20http://go.microsoft.com/fwlink/?LinkId=248585%20http://go.microsoft.com/fwlink/?LinkId=248586http://go.microsoft.com/fwlink/?LinkId=248585%20http://go.microsoft.com/fwlink/?LinkId=248585%20


6/21


Performance and Scalability Test OverviewScalability and stability are important to users of RedPrairie Parcel. Customers want to be confident

that their database platform can maintain the performance and scalability they require to optimize

their business processes, even as their organizations grow and their parcel volumes increase.

For this reason, engineers from RedPrairie and Microsoft conducted scalability and performance

tests for Parcel with WindowsServer 2008 R2 Enterprise and SQLServer 2008 R2 Enterprise, in

addition to a pre-release version of SQLServer 2012, SQLServer code-named Denali Community

Technology Preview 3 (CTP3). The tests were run in October and November 2011, and were

designed to simulate real-world deployments of large RedPrairie Parcel customers by using

industry-standard hardware to confirm a realistic total cost of ownership (TCO). A variety of tests

were performed on both physical and virtual servers to analyze the system performance on

different hardware and software configurations, and a final benchmark run was documented.

Test MetricsTo gauge application performance, two per-package metrics were recorded in the tests:

Execution time

The execution time, or response time, is a measure of how long a single package takes to

manifest. In the tests, the response time was the average response time for a set of

packages in a test run.

Throughout

The throughput is a measure of how many packages were manifested in an hour. In the

tests, the throughput was calculated by using the average response times and the number

of threads that were used for the benchmark task.

Performance and Scalability GoalsThe performance and scalability goals were defined based on the requirements of a large, tier-1

customer deployment.

System throughput

o 100,000 packages per hour per eight-core application server when running non-

virtualized

o 66,000 packages per hour per eight-core application server when running virtualized

Response time

o Response time at or below 1 second (1,000 milliseconds) for 90 percent of the packages


7/21


Test Deployment ScenarioIn the performance and scalability tests, the Parcel application was deployed on several servers

configured in a cluster with a scale-out configuration.

When a task was initiated, the benchmark framework developed by RedPrairie to gather the per-

package metrics loaded package data from a comma-separated values (CSV) file into a queue. The

load balancer pulled each individual package from the queue and pushed the package to an

application server in the cluster to be manifested. The server receiving each incoming package was

chosen by using a "round robin" algorithm that was controlled by a configuration file, which

controls settings such as where the benchmark task finds the CSV file, the number of concurrent

threads to use, and the maximum number of packages to be processed. Note that the number of

concurrent threads is very important for maximizing CPU utilization and for controlling the balance

between throughput and response time.

Each Parcel instance handled the manifesting and printing labels and documentation. In a scale-out

scenario such as that used in the performance and scalability tests, the synchronization between

instances of the application was accomplished by using shared services that were based on Java

remote method invocation (RMI).

Performance and Scalability Test ConfigurationThe performance and scalability tests used a variety of hardware and software configurations with a

consistent set of test data.

Hardware and Software Used in Tests

The performance and scalability tests used three main types of servers:

Database servers

The database used for Parcel application information was located on one of the database

servers.

Application servers

The application servers were used for manifesting packages.

Load balancer/generator

The benchmark tasks were executed from the load balancer/load generator. The execution

results were also stored on this server.

Figure 1 shows the full test configuration. Note that different combinations of hardware were used

for different tests.


8/21


Brocade Silkworm

3800 fabric switch

2 Gb/s bandwidth

Back-End Tier

(Database)

SunFire X4100

Su

n

Sun Fire x4150

8 x 2.0 GHz32 GB RAM

SunFire X4100

Su

n

Sun Fire x4150


ISE

HP DL380 G58 x 3.16 GHz32 GB RAM

UID12

DIMMS

PCIRISERCAGE

POWERSUPPLY

POWERSUPPLY

I-PPM

OVERTEMP

INTER

ProLiant

DL385G5

HPProLiant

DL380 G6

FANS

PROC

1

PROC

2

POWERSUPPLY

2

POWERSUPPLY

1OVERTEMP

POWERCAP

1 2 3 4

9

8

7

6

5

4

3

2

1 1

2

3

4

5

6

7

8

9

ONLINESPARE

MIRROR

UID

2

1

4

3

6

5

8

76 5 4 3 2 1

HP DL380G7


SunFire X4100

Su

n

Sun Fire x4150

8 x 2.0 GHz

32 GB RAM

SunFire X4100

Su

n

Sun Fire x4150


SunFire X4100

Su

n

Sun Fire x4150


Front-End Tier

(Application/Load Generation)

8765432187654321

UID21

G5

HPProLiant

DL580

HP DL580 G5


Microsoft SQL Server

HPProLiant

DL380 G6

FANS

PROC

1

PROC

2

POW ERSUPPLY

2

POWERSUPPLY

1OVERTEMP

POWERCAP

1 2 3 4

9

8

7

6

5

4

3

2

1 1

2

3

4

5

6

7

8

9

ONLINESPARE

MIRROR

UID

2

1

4

3

6

5

8

76 5 4 3 2 1

HP DL380G7


PowerEdge1950

0

1

2

3

Dell PE1850

2 x 2.40 Ghz4 GB RAM

AD/DNS

Windows Server 2008 R2 Enterprise

ESD

0

3

1

4

2

5

Systemx3850

IBM System x3850 X5

32 x 2.26 GHz512 GB RAM

4 CPU-8 GB RAM

4 CPU-8 GB RAM

4 CPU-8 GB RAM4 CPU-8 GB RAM4 CPU-8 GB RAM

4 CPU-8 GB RAM4 CPU-8 GB RAM

4 CPU-8 GB RAM

Microsoft SQL Server

X-IO Storage

ISE-1

Pool 2

20 x 10k SAS

drives R10

Transaction

log files

Pool 1

20 x 10k SAS

drives R10

Database and

tempdb files

Load

Gener

ator

Application

Servers

VirtualServers

Figure 1. Lab schematic

Database Servers

The database used for the lab was intended to simulate a real-world customer's database

containing an estimated two months of processed data. The database was located on a server

separate from the application server clusters in order to more accurately record metrics for

application-related and database-related activities.

Two database servers were used in the performance and scalability tests. The tests were started

with a 24-core HP ProLiant DL580 G5 server. It was observed that this server was underutilized, so a

smaller 12-core HP ProLiant DL380 G7 server was used for the remaining tests. SQL Server 2008 R2


9/21


was used in all tests except those test runs specifically comparing SQL Server 2008 R2 and

SQL Server Denali CTP3.

The specifications for the two database servers are shown in Table 1.

Table 1. Database server specifications

Make and modelNumber of

CPUsCPU

Number

of cores

RAM

(GB)

Cache

(MB)

HP ProLiant DL580 G5 4 Intel Xeon processor E7450 (2.40 GHz) 24 64 12

HP ProLiant DL380 G7 2 Intel Xeon processor X5660 (2.80 GHz) 12 64 12

Application Servers

The application server cluster was tested with both physical application servers and virtual


The physical application server cluster used between one and five application servers.

These servers were used in the tests that evaluated the scalability of Parcel running on the

physical servers.

The virtual application server cluster used between one and eight application servers. Each

virtual machine was configured to use four cores and 8 GB of memory from the hosts

hardware. These servers were used in the tests that evaluated the scalability of Parcel

running on virtual application servers.

The virtual application server tests were performed on a single domain-joined host

computer running Windows Server 2008 R2 with Service Pack 1 (SP1) with the Hyper-V roleenabled. The appropriate critical hotfixes were applied from the TechNetsiteHyper-V:

Update List for Windows Server 2008 R2

(http://social.technet.microsoft.com/wiki/contents/articles/1349.aspx).

The server used to host the virtual machines (IBM System x3850) was also used as a

physical server for some of the tests to compare virtual and physical application

performance.

The single application server final performance benchmark tests were performed on a

separate physical application server with more powerful hardware (HP ProLiant DL380 G7).

The server specifications for the servers used for application servers are shown in Table 2.
http://social.technet.microsoft.com/wiki/contents/articles/1349.hyper-v-update-list-for-windows-server-2008-r2.aspxhttp://social.technet.microsoft.com/wiki/contents/articles/1349.hyper-v-update-list-for-windows-server-2008-r2.aspxhttp://social.technet.microsoft.com/wiki/contents/articles/1349.hyper-v-update-list-for-windows-server-2008-r2.aspxhttp://social.technet.microsoft.com/wiki/contents/articles/1349.hyper-v-update-list-for-windows-server-2008-r2.aspxhttp://social.technet.microsoft.com/wiki/contents/articles/1349.aspxhttp://social.technet.microsoft.com/wiki/contents/articles/1349.aspxhttp://social.technet.microsoft.com/wiki/contents/articles/1349.hyper-v-update-list-for-windows-server-2008-r2.aspxhttp://social.technet.microsoft.com/wiki/contents/articles/1349.hyper-v-update-list-for-windows-server-2008-r2.aspx


10/21


Table 2. Application server specifications


CPUsCPU

Number

of cores

RAM

(GB)

Cache

(MB)

Sun Fire x4150

(5 servers)2

Intel Xeon processor L5335

(2.00 GHz)8 32 12

HP ProLiant DL380 G7

(1 server)2

Intel Xeon processor X5660

(2.80 GHz)12 64 12

IBM System x3850

(1 server)4

Intel Xeon processor X7560

(2.26 GHz)32 512 24

Load Balancer

The load balancer/generator was a separate server, with its own copy of Parcel and a database to

store the results. The specifications are shown in Table 3.

Table 3. Load balancer specifications


CPUsCPU

Number

of cores

RAM

(GB)

Cache

(MB)

HP ProLiant DL380 G5 2Intel Xeon processor

X5460 (3.16 GHz)8 32 12

The benchmark test task (script) was executed on this server. The benchmark results were also

stored locally on this server.

Storage

The storage area network (SAN) used was anX-IO ISE-1(http://xiostorage.com/products/ise-storage/#specs_tab_1) with the following specifications:

Server connections: (max) 128 Management: web UI, ISE Manager, ISE Analyzer

Two DataPacs: 20 spindles each

Host interfaces: two 4 Gb Fibre Channel

(FC) interfaces

Cache: 1 GBsafe, mirrored write-back and

read-ahead battery-backed, 96-hour hold time

Storage volumes: (max) 240

Test DataThe lab simulated a real-world production database for a high-volume shipper with approximately

two months of shipping data. The size of the database used was 49.5 GB and contained more than

2.6 million packages, along with address and rating information.

All of the tests used a single data set in the form of a CSV file. The data set was generated pseudo-

randomly based on a configurable set of properties to simulate a typical high-volume shipper. The

data set contained 29,302 packages with the following specifications:

70 percent domestic, 20 percent international, 10 percent Canadian shipments
http://c/Users/tsmith/AppData/Local/Microsoft/Windows/Temporary%20Internet%20Files/Content.Outlook/GG33F13N/X-IO%20ISE-1http://c/Users/tsmith/AppData/Local/Microsoft/Windows/Temporary%20Internet%20Files/Content.Outlook/GG33F13N/X-IO%20ISE-1http://c/Users/tsmith/AppData/Local/Microsoft/Windows/Temporary%20Internet%20Files/Content.Outlook/GG33F13N/X-IO%20ISE-1http://c/Users/tsmith/AppData/Local/Microsoft/Windows/Temporary%20Internet%20Files/Content.Outlook/GG33F13N/X-IO%20ISE-1


11/21


50 percent single-piece packages, 50 percent multi-piece packages (with a maximum of 20

packages per multi-piece shipment)

50 percent UPS shipments, 50 percent USPS shipments

Performance and Optimization TuningBefore running the performance and scalability tests, the database and application servers were

tuned and optimized.

Database Server Optimizations

Following are some of the optimizations made to the database servers:

Pre-sized database files to avoid auto-growth

All the database data and log files were pre-sized to prevent autogrowth from occurring

during the test runs.

Moved transaction logs to a separate SAN disk pool

High latency was initially observed on the log file logical unit number (LUN) when the LUN

was located on the same disk pool as the data files. The log files were isolated on their own

disk pool to reduce latency.

Created multiple tembdb files

One tempdb file per processor core was created on each database server used in testing to

eliminate possible contention, in accordance with Microsoft best practices. For more

information, seeOptimizing tempdb Performance

(http://msdn.microsoft.com/en-us/library/ms175527.aspx)

Application Optimizations

Following are some of the optimizations made to the application servers:

Enabled hyper-threading

Hyper-threading can provide a very useful boost to performance. To verify the benefit of

hyper-threading for Parcel, an application servers processors were saturated with and

without hyper-threading enabled. It was found that hyper-threading improved

performance by approximately 18 percent.

Used two threads per application server core

The number of threads is very important for controlling the overall performance of the

Parcel application because it controls the utilization of the processors and the balance

between response time and throughput.

o If the number of threads used is too low, the processors will not be used to their

fullest potential and will have some amount of idle time. This will cause the

response times to be very low, but will also lead to a lower throughput.
http://msdn.microsoft.com/en-us/library/ms175527.aspxhttp://msdn.microsoft.com/en-us/library/ms175527.aspxhttp://msdn.microsoft.com/en-us/library/ms175527.aspxhttp://msdn.microsoft.com/en-us/library/ms175527.aspxhttp://msdn.microsoft.com/en-us/library/ms175527.aspxhttp://msdn.microsoft.com/en-us/library/ms175527.aspx


12/21


o If the number of threads used is too high, the processors will be saturated and run

more slowly. This will lead to slow response times, but this will also cause a larger

throughput to some extent.

Testers observed that physical and virtual servers performed best when using two threads

per core. This means that a server with eight cores (the number of cores that most of the

application servers in the tests had), is likely to perform best with 16 threads.

The best performance with hyper-threading was also seen when using two threads per

logical core. Therefore, for a 12-core server with hyper-threading enabled, the tests used

48 threads.

Parallelized the generation of tracking numbers

RedPrairie found that as the number of client connections increased, the amount of time

spent to generate a tracking number increased rapidly. This performance bottleneck was

analyzed and it was determined that optimizations could be made to better parallelize the

generation of tracking numbers for different carrier accounts. This improvement boosted

the throughput by about 13 percent and decreased the average response time by about 12

percent for four physical application servers with 64 threads.

Performance and Scalability Test MethodologyPreliminary tests were run prior to the performance and scalability tests to determine the optimal

number of threads to use in the lab. After the lab was set up, two baseline performance tests were

run, one with one server and one with four application servers.

After the baseline tests, more than 110 tests were run with various optimizations and tuning

iterations. A series of final benchmark tests were then performed to record the achieved levels ofperformance. All tests were run at the Microsoft Platform Adoption Center (PAC) in Redmond,

Washington. Initially a more powerful database server configuration (HP DL580 G5, four CPU [Intel

Xeon processor E7450 2.4GHz], 24 core) was used. This database server was not very stressed, so

the rest of the tests were run with a scaled-down database server (HP DL380 G7, 2 CPU [Intel Xeon

processor X5660 2.80 GHz], 12 core). This finding confirms that the Parcel application performance

is not database-bound, and means that customers can minimize their database hardware

investments.

The performance and scalability tests focused on measuring the scalability of Parcel on physical and

virtual servers, and on confirming the performance on a single entry-level server.

Additional tests compared physical and virtual machines performance, looked at the effect of

hyper-threading, and compared the performance of Parcel running on SQL Server Denali CTP3 with

the performance of Parcel running on SQL Server 2008 R2.


13/21


Test Results and ConclusionsThe data gathered in the tests met the performance and scalability goals, confirming that Parcel

running on SQLServer 2008 R2 Enterprise and WindowsServer 2008 R2 Enterprise can meet the

requirements of the largest tier-1 customers. In addition, tests repeated on SQLServer Denali CTP3

showed nearly identical performance. This section shows the detailed results of the conducted finalbenchmark tests.

Multiple Application Server Scalability Analysis

The data gathered shows that forming a cluster of application servers, either physical or virtual, to

spread the workload can greatly benefit performance with excellent scalability.

Physical Server Scalability Tests

Table 4 shows the data gathered for physical application clusters of one, two, three, four, and five

servers using identical hardware.

Table 4. Physical server scalability

Category Run 1 Run 2 Run 3 Run 4 Run 5

Number of application

servers1 2 3 4 5

Database server usedHP ProLiant

DL580 G5

HP ProLiant

DL580 G5

HP ProLiant

DL580 G5

HP ProLiant

DL580 G5

HP ProLiant

DL580 G5

Application servers

used

Sun Fire

x4150

Sun Fire

x4150

Sun Fire

x4150

Sun Fire

x4150

Sun Fire

x4150

Number of CPUs per

application server8 8 8 8 8

Application server

memory (GB)32 32 32 32 32

Number of threads 16 32 48 64 80

Total execution time

(milliseconds)852 961 1,018 1,068 1,182

Throughput

(packages/hour) 67,636 119,862 169,795 215,659 243,673

Linearity

(percent)100 89 84 80 72


14/21


Figure 2 shows the throughput results.

Figure 2. Throughout in packages per hour for varying numbers of physical application servers

The data gathered in the physical application server scalability tests shows that it is beneficial to

scale out and use multiple application servers. The data confirms that the solution scales very well

horizontally, though at some point, scaling up or updating the hardware to newer versions might be

more appropriate. It is also important to note that response time also increases as the number of

servers increase.

Virtual Server Scalability Tests

Table 5 shows the data gathered for virtual application clusters with various numbers of servers.

67,636

119,862

169,795

215659

243,673

0

50,000

100,000

150,000

200,000

250,000

300,000

1 2 3 4 5

Throughout(packagesperhour)

Number of physical application servers


15/21


Table 5. Virtual server scalability

Category Run 1 Run 2 Run 3 Run 4 Run 5

Number of

application servers1 2 4 6 8

Database server

used

HP ProLiant

DL580 G5

HP ProLiant

DL580 G5

HP ProLiant

DL580 G5

HP ProLiant

DL580 G5

HP ProLiant

DL580 G5

Application servers

used

Virtual

(hosted on

the IBM

System

x3850 X5)

Virtual

(hosted on

the IBM

System

x3850 X5)

Virtual

(hosted on

the IBM

System

x3850 X5)

Virtual

(hosted on

the IBM

System

x3850 X5)

Virtual

(hosted on

the IBM

System

x3850 X5)

Application server

memory

(GB)

8 8 8 8 8

Number of CPUs

per app server4 4 4 4 4

Number of threads 8 16 32 48 64

Total execution

time

(milliseconds)

578 614 672 800 851

Throughput

(packages/hour)49,819 93,766 171,426 216,058 270,889

The highest throughput achieved was 270,889 packages per hour in run 5, generating

approximately 19,500 batch requests per second on the database server; in this configuration, theaverage CPU utilization of the database server was approximately 33 percent, with 42 GB of

database server memory available. These results show that the database server was not the

limiting factor, and that there was significant headroom still available on the database server for

even further scaling out the application servers.

Figure 3 shows the throughput in packages per hour.


16/21


Figure 3. Throughout in packages per hour for varying numbers of virtual application servers

As with the physical application servers, adding virtual application servers to the cluster improved

performance. The data again confirms that the solution scales very well horizontally, though at

some point, scaling up or updating the hardware to newer versions might be more appropriate. The

response time does increase as the number of servers increase.

Single Application Server Performance AnalysisPerformance data was gathered by using a single application server (HP ProLiant DL380 G7) and the

smaller database server (HP ProLiant DL380 G7). Tests were run with varying amounts of memory

and number of CPUs and threads.

Figure 4 shows the test configuration used.

49,819

93,766

171,426

216,058

270,889

0

50,000

100,000

150,000

200,000

250,000

300,000

0 2 4 6 8 10

Throughput(packagesp

erhour)

Number of virtual application servers


17/21


Figure 4. Hardware configuration used in benchmark test runs

Table 6 shows the data gathered in the single application server performance analysis tests.

Table 6. Results of single application server performance tests

Category Run 1 Run 2 Run 3

Number of application servers 1 1 1

Database server usedHP ProLiant DL380

G7

HP ProLiant DL380

G7

HP ProLiant DL380

G7

Application servers usedHP ProLiant DL380

G7

HP ProLiant DL380

G7

HP ProLiant DL380

G7

Application server memory (GB) 8 8 48

Number of CPUs per application

server12 24 24

Number of threads 24 48 48


(milliseconds)569 746 708

Throughput

(packages/hour)151,931 231,630 244,072

Front-End Tier

(Application/Load Generation)

Back-End Tier

(Database)


18/21


The highest throughput achieved was 244,072 packages per hour in run 3, generating

approximately 17,500 batch requests per second measured on the database server; for this

configuration, the average CPU utilization of the database server was below 30 percent, with 17 GB

of database server memory available. These results show that the database server was not the

limiting factor, and that there was significant headroom still available on the database server. The

database server can potentially support a higher number of applications servers when the

application tier is scaled out.

Figure 5 shows the throughout in packages per hour.

Figure 5. Throughput (in number of packages per hour)

Figure 6 shows the execution time in milliseconds.

Figure 6. Execution time (in milliseconds)

0

50,000

100,000

150,000

200,000

250,000

Run 1 Run 2 Run 3

Throughput(pack

agesperhour)

0

100

200

300

400

500

600

700

800

900

1000

1100

Run 1 Run 2 Run 3

Executiontime(milliseconds)

1 second


19/21


The results of these tests confirm that, even with entry-level hardware, it is possible to get

excellent performance.

The data shows that on a smaller server with six cores and hyper-threading enabled, it is possible to

achieve a throughput of more than 150,000 packages per hour. If the number of cores is doubled,

the throughput can be between 230,000 and 245,000 packages per hour. A more powerful server

will likely perform even better. In addition, the response times are below one second.

SQLServer Version AnalysisAdditional tests were conducted to compare the performance of the Parcel application with

SQL Server 2008 R2 to that with a pre-release version of SQL Server 2012, SQL Server Denali CTP3,

with all other variables held constant.

Table 7. Results of SQL Server version-comparison tests

Category SQL Server 2008 R2 SQL Server Denali CTP3

Number of application servers 1 1

Database server used HP ProLiant DL380 G7 HP ProLiant DL380 G7

Application servers used HP ProLiant DL380 G7 HP ProLiant DL380 G7

Application server memory

(GB)48 48

Number of CPUs per application server 24 24

Number of threads 48 48


(milliseconds)713 713

Throughput

(packages/hour)242,529 242,441

Database CPU utilization 29.2% 28.1%

The throughput and execution time for Parcel running on SQL Server Denali CTP3 was in the samerange as that with SQL Server 2008 R2 (only the normal variations seen with all test runs). A slightly

lower average CPU utilization was observed with SQL Server Denali CTP3, which is beneficial if the

workload is CPU bound.


20/21


Results of Additional TestsData was also gathered to compare physical and virtual machine performance and the effect of

hyper-threading.

Parcel application performance was not greatly affected by the power of the database

server. This is because the Parcel application throughput is not database-bound.

As a general rule, more threads produce a greater throughput and fewer threads lead to a

faster response time. The data gathered shows that for both physical and virtual

application servers, the optimal number of threads to use per core in the application server

is between 1.75 and 2.

Carrier and service-level analysis shows that domestic multi-piece USPS packages have the

highest throughput and the lowest response times, while the Canadian UPS packages have

the lowest throughput and the highest response times.

The data gathered shows that properly tuning the virtual machines in a cluster can boost

performance by between four percent and six percent.

SummaryThe performance and scalability tests conducted by RedPrairie and Microsoft demonstrated that

RedPrairie Parcel scales and performs well on the SQL Server database platform and the

Windows Server operating system running on industry-standard hardware.

The tests exceeded the performance and scalability goals of achieving at least 100,000 packages

per hour with a response time of less than one second on a single eight-core application server. The

data shows that on an entry-level server with six cores and hyper-threading enabled, it is possible

to achieve more than 150,000 packages per hour. If the number of cores is doubled, the

throughput can be between 230,000 and 245,000 packages per hour. A more powerful server will

likely perform even better. The response times for all benchmarking tests were significantly below

one second.

The test results confirm the impressive throughput and response times with two different

hardware configurations: one with 12 cores and 48 GB of memory and another with six cores and 8

GB of memory. Tests repeated on SQL Server Denali CTP3 showed nearly identical performance.

The tests also confirmed the benefits of horizontal scale-out by adding physical or virtual


These results clearly demonstrate that RedPrairie Parcel customers can get excellent performance,scalability, and value with the Microsoft platform.


21/21

7343.RedPrairie Parcel and Microsoft SQL Server Benchmark White Paper (1)

Documents

Transcript of 7343.RedPrairie Parcel and Microsoft SQL Server Benchmark White Paper (1)