Rice Stock Market SimulatorCOMP 415 Final Presentation
RICE
What is Real World Software Engineering?Real projects require real structure and strategyHierarchical team structureDivision of laborPlanning, specifications, and documentation
How does consulting work?Receive project requirementsDevelop specificationsVerify specifications with customerDevelop prototypesProvide demonstrations
Regular customer interaction is necessary
Technologies InvolvedMicrosoft Visual Studio 2005 Team Suite EditionC# .NET 2.0ASP.NET 2.0Microsoft Windows Server 2003Microsoft SQL Server 2005Microsoft Team Foundation Server
Microsoft SharepointFIX 4.2QuickFIX open source implementationLog4Net
Industry Customer HBK InvestmentsCustomer Representative: Matt Jeanes, Rice 06
Who is HBK?HBK is an investment management firm:Employing approximately 325 individuals around the worldBased in Dallas, TX with branches in New York, London, Tokyo and Hong Kong
Established October, 1991$13B in capital under managementRanked in the top tier of large, successful hedge fundsGenerated a compounded annual return of 14.68% since inception to June 2006
What is a Hedge Fund?Similar to mutual fundsRun by an investment management firm (fund manager)Investors pool their money together into one fundThe fund manager invests the money based on one or more strategieswith some key differencesCan invest in broader range of strategies and financial instruments (e.g., short selling, derivatives)Limited in the number and types of investors that can investHBK pursues a multi-strategy trading approach
What comes to mind when you think about working for a financial firm?
This?
What comes to mind when you think about working for a financial firm?
Or this?
22% of HBKs personnel work in technology!
Chart1
85
61
54
16
21
12
Sheet1
Accounting2
Financial Reporting
Management Company Reporting7
Management Company Reporting (Europe)1
Fund Accounting717
Admin SupportDallas8
NYC3
London2
Tokyo114
HR22
Internal Audit11
IR22
Legal & ComplianceLegal8
Compliance19
Operations37.537.5
Quant Research44
Recruiting55
Risk Management2.52.5
Securities Lending1212
Strategic Planning1.51.5
Tax & Upstream44
Treasury2.52.5
IT1515
Development3939
InvestmentsCredit22
Developed Markets FI3
Equities24
Private Placements & Other6
Vol Driven & Quantitative1469
Management Committee1212
249249
Investments, Securities Lending, Quant Research85Front Office (Trading, Quant Research etc)
Operations, Accounting, Treasury, Tax & Upstream61Operations & Accounting
Technology54Technology
Audit, Legal & Compliance, Strategic Planning, IR, Risk Management16Legal & Compliance, Audit, Risk etc.
Admin, HR, Recruiting21Admin, HR & Recruiting
MC12Management Committee
249
0.2168674699
Sheet1
0
0
0
0
0
0
Sheet2
Sheet3
Sheet4
Why does a hedge fund need all these technical folks? Software Development
Decision support tools for trading, risk management, portfolio analysis, accounting, etc.Computationally intensiveVery low error and downtime tolerance
Datafeeds & Databases
Multi-TB databasesMillions of updates every day from hundreds of sources around the worldTools for cleaning, monitoring, management, and reconciliation
Information Technology
Design, build, deploy, and manage global network, server, and workstation infrastructureWork with users to improve productivityAutomate systems management tasks
Quick FactsWe have more servers than we do staff!Our total data capacity is approx 40 terabytes with usage growing at around 10 gigabytes per day.Our databases maintain around 156 million historical prices from over 26,000 issuers. Our systems programmatically price over 170,000 options each day.Real time prices require DS3 (45Mbps)
Our Technology ToolkitMost of our systems and tools are Intranet-awareASP.NET 2.0 , IISApplicationsVisual Studio 2005, C++, C#, .Net FrameworkInfrastructureService-oriented architecture, Web Services, XML / XSLTDatabaseSQL Server 2000 (replicated and clustered), DTSOperating SystemsWindows XP, Windows Server 20033rd party tools and technologies
Technology InitiativesTwo tiers -> n-tiersDirect market accessBetter user experience C# GUIs for tradingExpanding data coverage and capabilityStraight through processingCash management systemPersonalization technology Infrastructure improvementsGigabit networking, WAN optimization, Voice-over-IPBlade servers and Storage Area Networks (SANs)
How has the project evolved?Hedge Funds operate on the worlds stock exchange systemsTesting trading strategies could be costly on the real marketHBK looked to Rices COMP 415 class for an alternative
Introduction to Stock ExchangesExchanges themselves have very simple behaviorTwo actionsBuy (Bids)Sell (Offers)ParametersSymbolVolumePriceTransactions occur when all three parameters of a bid and offer match
Introduction to the Rice Stock Market SimulatorProgrammatically mimic market function as closely as possibleAbility to replay historical market dataStore bid, order, and transaction data in a databaseUser interface for interacting with the market
HBKs New SpecificationsFill ParticipantControlled by an ASP.NET GUIExtend the functionality of the Human ParticipantImplement dynamic button configurationFIX Protocol Communication
Fill ClientWhat is a fill client?What were the requirements?Configurable fill parametersSize of a Fill (Full, Random, Integer)Time between Fills (Random, in ms)Load parameters from configuration fileChange parameters on the fly from a GUISave the modified parameters to a file
Fill Client BehaviorIf an order enters the fill client system, it will be filledOther clients may be acting on these ordersGenerate partial fills after the desired delay for the desired amountWhen to place the next partial fill?Identical orders?Market makes no distinction between orders for the same stock symbol at the same priceBut all identical orders must be filled eventuallySo the fill client needs to make sure it doesnt lose any of them
Fill Client AlgorithmEach new order generates one unit of demand from the fill client for the given stock at the given priceTwo bids for the same stock at the same price -> twice as many partial fillsEach partial fill has a target orderThe market does not guarantee that the partial fill will transact with its target orderFill client should add a target order back to the order queue when the associated partial fill is removed
Financial Information eXchange GatewayWe want to realistically simulate a stock exchangeFIX: an industry standard communications protocolFIX Gateway for any system to connect to RSMS
QuickFIX EngineOpen source implementation of the FIX protocolSeveral valuable features:Object oriented designLoggingError CheckingMessage Checking
How are Orders managed for a client?Listing of orders associated with each clientQuote Server interactionTransaction listener
Processing Node StructureEasy interface for increased functionalityMarket Validation RulesEnhanced simulation of Market behaviorRetrieved from MarketRulesTick SizeLot SizeValid StockNode 1Node 2Node 3
Live Demonstration
ASP.NET ClientFill ConfigurationFill SizeTime between FillsStatisticsActive usersStocksUser ManagementAdd, modify, delete usersOnly people classified as managers can do this
Human ParticipantGUI RedesignOrder form moved from popup to static on left side of the GUIPost bid/Post offer/Create Button forms combined into one
Human ParticipantDynamic ButtonsCreate new buttons given parametersStock SymbolVolume of StockStock PriceAll or some of the fields may be filled outBlank fields filled in when button is clickedButtons may be modified or deleted
Human Participant Demonstration
PerformanceThroughput of Bids/OrdersCapacity for Simultaneous Connections
Fill Client Performance
Chart1
23.2
9.6
20.2
34.2
12.6
1.2
Time
Transactions per Second
100 Orders, 500 Shares
30Apr2007-100ord-500share
Report TimeTransactions in IntervalTransactions/Second
63313563862255500011623.2
633135638672399000489.6
63313563872240000010120.2
63313563877240000017134.2
6331356388224010006312.6
63313563887240200061.2
63313563892240200000
63313563897240300000
63313563902240300000
63313563907240400000
63313563912240500000
63313563917240500000
63313563922240600000
63313563927240700000
63313563932240700000
63313563937240800000
63313563942240900000
63313563947240900000
63313563952241000000
63313563957241100000
63313563962241100000
63313563967241200000
63313563972241200000
63313563977241300000
63313563982241400000
63313563987241400000
63313563992241500000
63313563997241600000
63313564002241600000
30Apr2007-100ord-500share
Time
Transactions per Second
100 Orders, 500 Shares
Fill Client Performance
Chart2
0
9.8
13
9.2
20.2
13.4
16.8
21.2
11.4
12.4
21.8
16.2
9
20.8
13.8
15.4
20.2
11.8
16.4
13.4
19.4
14.6
17.6
18.6
14
19
12.2
18.8
15
21.6
21.4
19
Time
Transactions per Second
500 Orders, 500 Shares
Chart1
0
9.8
13
9.2
20.2
13.4
16.8
21.2
11.4
12.4
21.8
16.2
9
20.8
13.8
15.4
20.2
11.8
16.4
13.4
19.4
14.6
17.6
18.6
14
19
12.2
18.8
15
21.6
21.4
19
Transactions/Second
Time
Transactions per Second
500 Orders, 500 Shares
30Apr2007-500ord-500share-1
Report TimeTransactions in IntervalTransactions/Second
63313564350121100000
633135643551211000499.8
6331356436012120006513
633135643651213000469.2
63313564370121300010120.2
6331356437512140006713.4
6331356438012150008416.8
63313564385121500010621.2
6331356439012160005711.4
6331356439512170006212.4
63313564400121700010921.8
6331356440512180008116.2
633135644101218000459
63313564415121900010420.8
6331356442012200006913.8
6331356442512200007715.4
63313564430122100010120.2
6331356443512220005911.8
6331356444012220008216.4
6331356444512230006713.4
6331356445012240009719.4
6331356445512240007314.6
6331356446012250008817.6
6331356446512260009318.6
6331356447012260007014
6331356447512270009519
6331356448012270006112.2
6331356448512280009418.8
6331356449012290007515
63313564495122900010821.6
63313564500123000010721.4
6331356450512310009519
63313564510123100000
63313564515123200000
63313564520123300000
63313564525123300000
63313564530123400000
63313564535123400000
63313564540123500000
63313564545123600000
63313564550123600000
63313564555123700000
63313564560123800000
63313564565123800000
63313564570123900000
63313564575124000000
63313564580124000000
63313564585124100000
63313564590124200000
63313564595124200000
63313564600124300000
63313564605124300000
63313564610124400000
63313564615124500000
63313564620124500000
63313564625124600000
63313564630124700000
63313564635124700000
63313564640124800000
63313564645124900000
63313564650124900000
63313564655125000000
63313564660125000000
63313564665125100000
63313564670125200000
63313564675125200000
63313564680125300000
63313564685125400000
63313564690125400000
63313564695125500000
63313564700125600000
63313564705125600000
63313564710125700000
63313564715125800000
63313564720125800000
63313564725125900000
63313564730125900000
63313564735126000000
63313564740126100000
63313564745126100000
30Apr2007-500ord-500share-1
Time
Transactions per Second
500 Orders, 500 Shares
Fill Client Performance
Chart2
7.8
12.6
10.2
18.2
7
4.6
15.4
14.4
9
9.4
4
18.4
17.8
7.4
4.6
14.8
20.8
3.6
9.6
11
16.4
11.8
8.8
4.6
17.6
18
7
6
11.6
16.6
10.4
9.8
9.4
13.6
15.2
8.4
6
14.4
16
10.6
6
12
13.8
14.6
7.8
6.4
14.2
16.6
8.6
8.2
9.2
17.2
13
7
8.6
14.4
16
8.4
8.8
11.8
15.8
11.8
8.4
11.2
13.8
14.8
8.4
8.4
15.4
16.6
11
8.8
13.2
14
10.8
11.6
10
16
7
2.2
1.4
2
4.6
3
2
0.4
1.2
0.8
0.4
0.6
0.8
0.8
0.4
1
1
Time
Transactions per Second
1000 Orders, 500 Shares
Chart1
7.8
12.6
10.2
18.2
7
4.6
15.4
14.4
9
9.4
4
18.4
17.8
7.4
4.6
14.8
20.8
3.6
9.6
11
16.4
11.8
8.8
4.6
17.6
18
7
6
11.6
16.6
10.4
9.8
9.4
13.6
15.2
8.4
6
14.4
16
10.6
6
12
13.8
14.6
7.8
6.4
14.2
16.6
8.6
8.2
9.2
17.2
13
7
8.6
14.4
16
8.4
8.8
11.8
15.8
11.8
8.4
11.2
13.8
14.8
8.4
8.4
15.4
16.6
11
8.8
13.2
14
10.8
11.6
10
16
7
2.2
1.4
2
4.6
3
2
0.4
1.2
0.8
0.4
0.6
0.8
0.8
0.4
1
1
Transactions/Second
Time
Transactions per Second
1000 Orders, 500 Shares
30Apr2007-1000ord-500share-1
Report TimeTransactions in IntervalTransactions/Second
633135662114574000397.8
6331356621645750006312.6
6331356622145750005110.2
6331356622645760009118.2
633135662314577000357
633135662364577000234.6
6331356624145780007715.4
6331356624645790007214.4
633135662514579000459
633135662564580000479.4
633135662614580000204
6331356626645810009218.4
6331356627145820008917.8
633135662764582000377.4
633135662814583000234.6
6331356628645840007414.8
63313566291458400010420.8
633135662964585000183.6
633135663014586000489.6
6331356630645860005511
6331356631145870008216.4
6331356631645870005911.8
633135663214588000448.8
633135663264589000234.6
6331356633145890008817.6
6331356633645900009018
633135663414591000357
633135663464591000306
6331356635145920005811.6
6331356635645930008316.6
6331356636145930005210.4
633135663664594000499.8
633135663714595000479.4
6331356637645950006813.6
6331356638145960007615.2
633135663864596000428.4
633135663914597000306
6331356639645980007214.4
6331356640145980008016
6331356640645990005310.6
633135664114600000306
6331356641646000006012
6331356642146010006913.8
6331356642646020007314.6
633135664314602000397.8
633135664364603000326.4
6331356644146030007114.2
6331356644646040008316.6
633135664514605000438.6
633135664564605000418.2
633135664614606000469.2
6331356646646070008617.2
6331356647146070006513
633135664764608000357
633135664814609000438.6
6331356648646090007214.4
6331356649146100008016
633135664964611000428.4
633135665014611000448.8
6331356650646120005911.8
6331356651146120007915.8
6331356651646130005911.8
633135665214614000428.4
6331356652646140005611.2
6331356653146150006913.8
6331356653646160007414.8
633135665414616000428.4
633135665464617000428.4
6331356655146180007715.4
6331356655646180008316.6
6331356656146190005511
633135665664619000448.8
6331356657146200006613.2
6331356657646210007014
6331356658146210005410.8
6331356658646220005811.6
6331356659146230005010
6331356659646230008016
633135666014624000357
633135666064625000112.2
63313566611462500071.4
633135666164626000102
633135666214627000234.6
633135666264627000153
633135666314628000102
63313566636462800020.4
63313566641462900061.2
63313566646463000040.8
63313566651463000020.4
63313566656463100030.6
63313566661463200040.8
63313566666463200040.8
63313566671463300020.4
63313566676463400051
63313566681463400051
30Apr2007-1000ord-500share-1
Time
Transactions per Second
1000 Orders, 500 Shares
Benefits of the RSMS SystemAllows trading strategies to be tested on historical dataSimulated Market Time allows long histories to be run at a higher velocityFill strategies can be developed using a feedback loop generated by interactions with real data running in syncAllows developers to test trading software under simulated market conditions
Special Thanks to HBK InvestmentsQuestions?
Comp 415 puts students in the drivers seat of a pretend software consulting company working for a real customerTraditional university level computer science class projects typically only involve one or two students per groupIn the real world projects are much larger, and thus take more manpowerWith larger teams, such as the 10 person COMP415 class, a hierarchical structure must be used to keep organizedWorking in a larger team requires much more planning so that a specification can be drawn up and used for development so each team stays on trackAdditionally, documentation of all work is that much more important should others need to understand an unfamiliar piece of the codebaseIn the real world, software consulting companies are hired to solve a particular problemThis involves regular interaction with the customer, first meeting to understand the problem and receive a basic outline of requirementsThen the team will go back and develop a set of specifications along with a timeline of what will get accomplished and whenThese specifications get verified with the customer and if satisfactory, the team begins developing the product, attempting to produce prototypes along the way in order to give the customer an idea of the progress being made 1.) so that the customer can see that something is getting done, and 2.) so that the customer can verify that the team is working in the proper directionFinally, once a solution has been developed to a satisfactory point the final product is packaged, demonstrated and delivered to the customerSo what tools did the class use to accomplish its goals for the semesterIndustry standard tools from Microsoft that are used at many large corporations including HBK
Visual Studio gives us a professional development environment to work in to write code in C# and ASP using the .NET platformWindows Server 2003 gives us a stable platform to run our services onSQL Server gives us an effective means for efficiently storing dataTeam foundation server source control allows for all source code to be deposited into one repository for versioning and redundancy, allowing the team to code such a large project much more easilySharepoint is used for collaboration: posting announcements, specification documents, maintaining calendars and timelines of events, etc.FIX is the Financial Information Exchange protocol which is the industry standard protocol for communicating financial informationLog4Net is an open source logging package that allowed us to maintain a central log system, which is especially useful for bug tracking
So now that Ive given you a little bit better idea of how the class operates, I would like to introduce the representative from our industry customer HBK Investments, Matt Jeanes who graduated from Rice just last yearSo, how did HBK investments and Comp 415 get to where we are today?
A hedge fund buys and sells an astonishing amount of securities and their derivatives on a daily basis. Millions of shares and dollars can be traded on a single day through the worlds stock exchanges: NYSE, Nasdaq, Euronext, etc.
If HBK develops a new trading strategy the current method for testing this strategy is to use it on a live market. This opens the possibility for a strategic flaw costing millions of dollars.
HBK tasked last years Comp 415 class to build an alternative, a stock market simulator that acts like a real stock exchange.
So the first question is how does a real stock exchange work at its core. In essence the exchange facilitates the meeting of buyers and sellers. It provides a black box where someone wanting to buy 1000 shares of GE at $35 can find sellers rapidly.
An electronic stock exchange, like NASDAQ, allows this meeting of buyers and sellers to be done entirely within a digital medium.
In this way the Rice Stock Market Simulator is a simulation of an electronic stock exchange.
The original specification for the RSMS that we inherited consisted of two parts, the actual exchange and its interface.
The goal of the exchange simulation was to mimic the market function as closely as possible. Additionally, HBK requested that the stock market simulator have the ability to play back a recorded day of trading.
The interface was supposed to be extensible, so that multiple different types of participants could connect to the market.
Here is a graphical representation of the original RSMS. The three client applications all interface with the market in the same way.
The Human Client is a GUI application that allows a trader to buy and sell shares interactively as well as sift through information about the history of an individual stock or the current best bids and offers.
The market playback functionality was encapsulated in the concept of an ITCH participant. ITCH is the industry standard for saving order tracing in a stock market, so the playback client loads in a saved set of trades and then recreates them on the stock market.
The C++ Participant is a set of classes that was conceived to provided a programmatic interface between HBKs proprietary trading software and the stock market simulator.
Internals: EXTENSIBLE, EXTENSIBLE
This year HBK approached us with specifications for adding new features to the stock market simulator. Whereas last years class had the luxury of designing their project the way they wanted, this year we had the constraint of being forced to start with an unknown codebase.
First, HBK requested the addition of a Fill Participant which is an automated trading client. Its purpose is to automatically fill orders placed on the market with parameters specified in an ASP.NET webpage interface.
Second, we were to extend the Human GUI client to incorporate some features HBKs proprietary trading software, in particular that the GUI can be customized on the fly by adding new dynamic trading buttons.
Finally, HBK desired that the RSMS implemented a subset of the FIX Protocol, allowing our market to speak the language of the worlds stock exchanges. This is a superior solution to the original C++ class interface to the market for HBKs software.
This is how the RSMS looks after incorporating the new design. The Fill Client and FIX Gateway are new modules added to the internal structure of the market. Just like the quote server there can be multiple gateways or fill clients running on any number of machines.
The FIX Gateway is the new entry point to the market through which orders must go. In addition to translating FIX to the Web Methods used by the Market Web Service the Gateway also validates incoming rules against the stock markets rules. These rules are held by the Market Web Service and distributed to all FIX Gateways
The Fill Client, or Clients are configured using the ASP.NET web page and then listen for new orders from a Quote Server. When it receives notification of new orders, the Fill Client begins placing orders to the Market Web Service as it has been directed.
Now that youve got an overview of our system, Adam is going to explain the Fill Client in more detail.
Fill client represents the actions of the rest of the market, filling the orders that the user of the system placesFirst point is an invariant: if an order leaves the fill client system, it must have been filledCant just blindly submit next partial fill because other clients can act on the orders that youre trying to fillOne unit of fill client demand = one partial fill of the specified amount placed after the specified delay
Each partial fill is generated in order to fill a certain target order placed by a userBut the market has no notion of targetingTo make sure identical orders are filled, the fill client should have this notion of targeting and re-add the target order after its partial fill is goneSuppose that a user places a bid for 500 shares of stock, and the fill client fills 100 at a time
On this slide, explain the whole processOn this slide, emphasize the parts that have changed: looking up in the tableThis process repeats itself until order #1 is no longer in the market, thus ensuring that it is filled before it is removed from the fill client systemEach data point represents an average over a 5 second interval
This first graph is rather shortLonger, shows that the performance hovers in the mid-high teens
Kind of spiky; probably because the buckets dont match up with quote server updates, which come every 2 secondsSpikiness is even more apparent here
Average TPS seems to be dropping; several possible bottlenecks, including the size of the order queue and the load on the quote server/MWS
Top Related