Ermanometry Research: The Quest for Accurate Data: Why It is Important

15/06/14 1:17 PMErmanometry Research: The Quest for Accurate Data: Why it is Important

Page 1 of 6http://ermanometry.com/articles/the_quest_for_accurate_data_why_it_is_important.php

The Quest for Accurate Data: Why it isImportant

From the Bridge Trader - Jan/Feb 1999

"Data geeks" is a term often applied to our personnel here at ErmanometryResearch. We've even been called data freaks. Sometimes the languagegets still more colorful, and we love it! From our perspective, the strongestlanguage is the greatest compliment, provided that it refers only to ourobsession with accurate data. We consider it confirmation that we aredoing our job.

Our research requires us to be compulsive about accurate data. One of thefoundations of our work is that market movements are not random. Thisapplies to all freely traded markets, cash and futures, from grains andmetals to financials and equities. Our thesis that all markets conform tospecific dynamic patterns, both in price and time, was not a preconceptionfor which we sought evidence. This thesis was developed fromoverwhelming evidence uncovered through painstaking data analysis. Thebook, Ermanometry-The Perfectly Patterned Stock Market, containshundreds of pages with this evidence and the methods used to decodemarket moves. Ermanometry measures moves of more than 60 years usingincrements no larger than a single trading day. We do not count in weeks,months or years. The permissible error factor on these massive moves isless than one/thousandth of one percent. Accurate data is imperative in thisanalytical environment. For example, Ermanometry Research hasprojections for more than 16 time periods of major support or resistancefor the DJIA and S&P 500 during 1999. Among the most significant arethose centered on April 12 and September 1. If the indices exceed thehighs of January 8, 1999, we expect them the be making historic highsabout April 12. These projections result from the application ofproprietary algorithms to the number of trading days between previousmajor turning points. We consider turning points to be those days onwhich the market reaches new intraday high or low extremes and thenreverses. Closing prices are not considered. Some of our algorithmsrequire multipliers as large as four. Assume that a projection was basedupon applying a multiplier of four to a move counted as 100 days. Assumethat the true turning point actually occurred on day 99, but faulty datacaused us to believe the turn occurred on day 100. Multiplying theincorrect total of 100 days by four, and then adding the resulting 400 days(accurate data would have given a total of 396) to day 100 of the previousmove, would actually create an error of five days. A four-day errorresulted from the multiplication, and adding the result to day 100 insteadof day 99 of the previous move increases the error factor to five days.Obviously, this is unacceptable. Thus a data error of only one day couldcause our high probability projection of a major trend change in theindices to be shifted from the time period centered on April 12 to one a

Articles Section

Table Of Contents

Precise Projections withBasic GeometryCompound Pivots andMarket SymmetryLog Spirals In The StockMarketThe Quest for AccurateData: Why it is ImportantAtop Mountains of Data -Atlantan is Unlocking TheMarkets Secrets

Ermanometry ResearchHome Overview Alerts Concepts Examples Articles What They Say Own The Book Contact Us About



whole week later, centered on April 19.

The extent to which Ermanometry Research requires accurate data maynot apply to the average trader/analyst. We believe in the KISS principle(keep it simple, stupid) and a trader should never get so involved in"details" that the big picture is obscured. A favorite expression of ours is...some people are so fervent over details they get caught in their ownunderwear. Nevertheless, Ermanometry has found many errors in theofficial records of major exchanges, regarding both the actual count oftrading days and daily high/low prices, and all market participants shouldbe aware of the potential for errors and the results of using bad data. A fewbad ticks may not have much effect on moving averages and oscillators,but errors have a cumulative impact. Trendlines can be terribly skewed ifthe bad ticks include an important high or low.

Figure 1 illustrates an erroneous daily high that still resides in data banks10 years after it occurred. It contains a "spike" that occurred on October31, 1988 and shown on the five-minute chart of the S&P 500 Index. If theanalyst was using real time data and small increment time charts, the spikewould have been obvious, and a correction made. However, on an hourlychart the spike would not necessarily be evident. The error would bealmost impossible to detect on a daily bar chart.

Figure 2

CME S&P 500 Index-Nov. 88Time and Sales-10/31/88

2:212:222:232:242:25

2787927878278772793927871

2788827878278762787327871

2788827877278752787227872

27878

2787527872

27939



2:26 27871 27871 27873 27873 27873

Please note above that the last tick at 2:23 was 279.39,64/100 away from the previous tick...the first tick at 2:24repeated this aberration, and then prices returned to"normal". The 279.39 ticks were almost certainly an error.

NYFE NYSE Index SpotTime and Sales-10/31/88

2:212:222:232:242:252:26

156821568215681156791567915679

156831568215681156791567915679

156831568215681

1567915679

1568215682

1567915679

This is the time and sales data for the NYSE Composite, thebest surrogate for the S&P 500. This index did not reflect asudden rise and fall at 2:23. This is conclusive proof that thedata for the S&P was faulty.

Figure 2 is a "time and sales" listing for the S&P 500 Indexand the NYSEC for five minutes on 10/31/88.

Figure 3S&P 500 Index, Monday October 31, 1988

Total call volume 4,942Total put volume 4,581The Index: High 279.39

Total call open interest224,261Total put open interest229,841Low 277.14; Close 278.97,+0.44

Figure 3 shows the statistics printed in all of the financial paperson the next day. Please note that the erroneous tick is shown asthe high for the day. This error is "forever" embedded in everyhistorical data bank that Ermanometry has investigated. Vendorsof historical data are in a difficult position. They may know oferrors but if their data conflicts with the "official" data, the clientwill most likely assume that the official records are correct andthe vendor's data is wrong. Therefore, the vendors will usuallyretain the faulty data rather than conflict with the officialrecords.

A bad tick in an index is usually caused by a bad tick in one of theindividual stocks in the index. The NYSE will normally correct theerror in the individual stock data. However, the indices arecalculated by outside vendors. Therefore, unless the outsidevendor picks up the correction message sent by the NYSE for theindividual stock and then recalculates the index and sends out acorrection message to be inserted at the proper time, the indexwill remain uncorrected. It is a mistake to assume that these



corrections will be made.

There is one recent development that may alleviate the problemof bad ticks in the DJIA. Dow Jones & Company, Inc. has recentlycanceled old licenses which allowed a multitude of outside datavendors to compute and distribute the various Dow Jones &Company, Inc. averages. Dow Jones & Company, Inc. will computethe averages and the Chicago Board of Trade will be the exclusivegateway for redistribution of the calculations to other vendors. Atthis time we do not know if the averages will be recomputedwhen bad ticks in individual stocks are corrected and thecorrections in the averages then distributed. Even if thesecorrections in the averages are made, there is still the problem ofinserting the corrections into individual data bases.

The type of error represented in figure 1 is particularly insidiousbecause if the analyst corrected the spike on an intraday chart hemay have assumed that the bad tick had been permanentlyeliminated. Unfortunately, since the bad tick represented thehigh for the day, those data feeds that recap the daily high/low,often received from other vendors, would show the bad tick as ahigh. Thus the error would show on the daily chart even thoughthe analyst had eliminated it on the intraday charts.

Some errors are innocuous but Murphy's law appears to haveundue influence upon when the most errors occur. Adisproportionate share of errors occurs at the end of explosive orpanicky moves. These are the most chaotic moments and theenvironment in which errors thrive. The "end" of such moves oftencontains the extremes for the period and price action analyzed.Therefore, the analyst must consider all extremes suspect untilverified. Remember, errors at extremes affect not only timing,but trendlines, oscillators, and almost every tool in the analyst'sarsenal.

It is impossible to truly appreciate the large number of pricecorrections, insertions, deletions, etc. without having had theexperience of watching the data stream printed out on the yellowpaper tape from an old Western Union type ticker. Correctionswill appear almost every few inches. Sometimes the entries are assimple as changing a bid or ask quote to an actual trade, or viceversa, and other times entire strings of trades are deleted. Veryoften these deleted trades actually took place, but they are"busted" (deleted) because the trades shouldn't have beenexecuted.

Busted trades are most frequent in the futures pits. When tradingis frenzied it is possible that a pit broker might not hear or seeevery bid/ask in the pit and the market will trade "through" aprice that a broker is legitimately, diligently bidding or offering.

Assume that the market is trending down from 105 in very active



trading. Conditions may or may not warrant a "fast market"designation which would invoke a different set of parametersgoverning pit rules. Fast market conditions will not be covered inthis article because it would be an unnecessary complication:

Broker A is diligently bidding for 10 contracts at 101.Across the pit, Broker B bids 100 for two contracts.Broker C, standing next to Broker B, receives an order to sellfour contracts "at the market."Broker C sells 2 contracts to Broker B at 100. Broker B thendrops his bid to 99, and Broker C sells him two more at 99.News hits the market, there are no more offers, and bidsrise to 103. The next trade is 104.Broker A never got a chance to buy any contracts at 101.Conditions were such that Broker C neither saw nor heardBroker A's bid at 101. It was an honest error.Broker A's client, seeing the prints at 100 and 99 rightfullyassumed that his order to buy at 101 had been filled.The pit committee would most likely bust the trades at 100and 99, and a deletion message would be sent.Broker C's sales at 100 and 99, would be given to Broker Awho had been bidding 101. Thus the selling client would geta better price, and Broker A's buyer would have been filledon four of the 10 contracts he wanted to buy.

Murphy's law not only makes sure that the most errors occur atthe end of runs, but also that deletions or insertions will hoveraround the "even" prices, such as 100 or 150. This means that thepoint and figure chartist may have filled a box that should not befilled, or not filled one that should have been filled. Erroneousdata can cause charts to read like comic strips, and causeoscillators and moving averages to generate false buy or sellsignals, particularly in short-term trading.

The analyst can take measures to protect the integrity of his data.The obvious answer is to be constantly vigilant. The bestmeasures are to understand the differences between various datafeeds and charting software. If short-term in and out trading isdone, it is helpful for the analyst/trader to have a data sourcethat automatically transmits all error messages and deletes,inserts, and corrects the data used.

There are many real-time data feeds available. Unfortunatelythey are not all equal in their performance concerningcorrections, and speed. Another important factor is whether ornot the analyst/trader can access the vendor's data base with hisown computer, i.e. two way communication, or is merely apassive recipient of the data. As a passive recipient it is necessaryfor the analyst/trader to manually make any corrections that hemay find. The kind that he would know of would probably belimited to daily high/low prices or obvious spikes. Manualcorrections are time consuming and aggravating, and thetrader/analyst would still be unaware of the vast majority of



corrections.

There are some data vendors that make corrections in the client'sdata even though the data is stored on the client's computer.However, the type of corrections made are usually not ascomprehensive as those available to the client who has constantdirect communication with the data base of a vendor thatcorrects every erroneous print.

The intent of this article is to make you, the reader, aware ofdata aberrations but not to make you paranoid about them. Badticks are not going to "make or break" your trading.

Trading discipline and money management are more important,and they should not be neglected while you get distracted or"caught in your own underwear" in the attempt to clean up everysingle tick.

Technical note: we are receiving sporadic reports of problems viewing our PDFs on Internet Explorer7. We have found that right-clicking, and then saving the PDF to your computer before viewing it is a

good workaround if you are experiencing this issue. Users of other browsers do not appear to beaffected.

HomeCopyright 2007 Ermanometry Research

Ermanometry Research: The Quest for Accurate Data: Why It is Important

Documents

Transcript of Ermanometry Research: The Quest for Accurate Data: Why It is Important