Evidence-Based Technical Analysis: Applying the Scientific Method and Statistical Inference to Trading Signals

UPDATE: 10/1/21 15 years later- Still the BEST BOOK on security trading you can own. IMAGINE— That in one hand you held a bag full of returns from YOUR technical trading system.... .... In the other hand you held a bag full of RANDOM returns from the market over the same period. NOW IMAGINE— That there was a 500 year old scientific tool which would allow you to COMPARE each bag of returns to determine if YOUR trading system actually works better (makes more $$$) than a system driven by pure LUCK. The tool actually TELLS YOU IF YOU ARE “WINNING” SIMPLY BECAUSE OF LUCK or YOUR TRADING SYSTEM IS ADDING ANY VALUE, ANY VALUE AT ALL, TO YOUR TRADES. Aronson’s book explains, in thorough detail, THAT scientific method. ###########. ORIGINAL REVIEW OF 12 YEARS AGO ############ What you take-away from a reading of this book really depends on where you're coming from. For STATITICIANS with an interest in trading markets--You'll likely walk away with the feeling: "Yeah, that's what I've been thinking for years, nice to see someone took the time to debunk the TA myth." For TECHNICIANS (traders) with an interest in statistics--You'll likely walk away thinking "You gotta be kidding. There are a hundred good books which can show you how to use TA to make money. This book sucks." Aronson suggests that the truth does NOT lie in between-- He is firmly in the camp of the Statistician. But a close reading of this powerful book does not "close the door" on profitable TA, it simply confirms what every first-year MBA learns: "No OBJECTIVE black-box trading strategy CONSISTENTLY beats the market AVERAGES over the LONG TERM." But hard-core TA fans take heart. You will find something interesting in this book also. I'm certain Aronson would agree with the following: "Sure you can add COMPLEX rules to the black-box, and PERIODICALLY find runs of profitability with TA. If EXCESS returns exist only in the SHORT TERM, hey, that's good enough for me." For those not willing to take the time to digest the painstakingly presented statistical concepts, there will be little value in this book-- This is a serious study with lots of math. Its not hard math. But math best understood after fully internalizing a college level stats class. Even for those with a Stats or Econometrics degree statistics are tough--both computing and interpreting statistical data takes a little work. To complicate the issue, market-related statistics are fraught with half-truths, mind-bending math, and wall-street lore. This book goes a long way to put bogus TA lore to rest by presenting a clear, scientifically sound procedure to test Technical rules. For those seriously considering buying this book let me suggest that you find Aronson's website [...] and download and read Dr. Timothy Masters' .pdf "Monte-Carlo Evaluation of Trading Systems." The document, both in tone, and sophistication, mirrors Aronson's book. If you like Masters' 43 page doc--you will love Aronson's 500+ page book. The Review I break the book up into four parts, each with various degrees of usefulness depending on your background--ie: Technician or Statistician. Below, I'll simply give what I thought was the "money-quote" from each part, plus a couple of observations for those considering buying the book. ***** Part 1: Chapter 1 - 31 pages Objective Rules and Their Evaluation "The isolated fact that a rule earned 10 percent rate of return in a back test is meaningless. If many other rules earned over 30 percent on the same data, 10 percent would indicate inferiority, whereas if all other rules were barely profitable, 10 percent might indicate superiority." - Aronson, page 23 Constructing Rules - Intro to bi-modal rule construction and trigger thresholds Data Transformation - Nice review of position-bias, log-differences and testing biases Benchmarking Rules - Good review of why "Relative-Benchmarking" is important Beating the Benchmark - Why a profitable back test is not conclusive proof of good rule ***** Part 2: Chapters 2-3 - 130 pages The Illusory Validity of Subjective Technical Analysis The Scientific Method and Technical Analysis "Statistician Harry Roberts said that technical analysts fall victim to illusion of patters and trends for two possible reasons. First, the "usual method of graphing stock prices gives a picture of successive (price) levels rather than of price changes and levels can give an artificial appearance of pattern or trend. Second, chance behavior itself produces patterns that invite spurious interpretations""-- Aronson, page 83 The Eye Deceives - Charting a random process and the representativeness heuristic Subjective vs. Objective -- Why its important to be able to "hard-code" a TA rule The Role of Logic - Why "Falsification" is more important than "Affirmation" in TA Astrology vs Astronomy - Pushing the TA boundaries from pseudo- to science ***** Part 3: Chapter 4-7 - 230 pages Statistical Analysis Hypothesis Tests and Confidence Intervals Data-Mining Bias: The Fool's Gold of Objective TA Theories of Non-Random Price Motion "Informal data analysis is simply not up to the task of extracting valid knowledge from financial markets. The data blossoms with illusionary patterns whereas valid patterns are veiled by noise and complexity. Rigorous statistical analysis is far better suited to this difficult task." - Aronson, page 172 Hypothesis Testing--Good review of probability and statistical inference The Traditional Solution - Actually put your college-level stats knowledge to use The Monte-Carlo Solution - Putting computer randomization and re-sampling to work The Data-Mining Problem -- Why traditional MC solutions don't work Inefficient Markets - How, where and why profitable TA rules should STILL exist ***** Part 4: Chapter 8-9 - 100 pages Case Study of Rule Data Mining for the S&P 500 Case Study Results and the Future of TA "Few rule studies in popular TA apply significance tests of any sort. Thus, they do not address the possibility that rule profits may be due to ordinary sampling error. This is a serious omission, which is easily corrected by applying ordinary hypothesis tests." - Aronson. page 449 The Operators - Reviews: channel-break-outs, moving averages, channel-normalization The Indicators -- Reviews: price, volume, breadth, spreads, yields The Rules - Reviews: trends, inverse trends, reversions, divergence The Results - Analysis of why 0 of the 6,402 tested rules produced no significant results The Bottom Line Aronson's book reminds me of that masked-magician on TV who has given away the secrets to all the best stage illusions. Novice magicians and apprentice conjurers will undoubtedly be "pissed-off." But true professionals are liberated. The best in the field can focus on new and potentially MORE exciting illusions--not the same old tricks.

David Aronson's Evidence Based Technical Analysis ("EBTA") is a fantastic book, and one which our industry has sorely needed. It is a "How to Do Research" book that details the scientific method with regard to the markets. Everyone in the field should both read the book and practice what it preaches. But that won't happen, which is both bad news and good news. The bad news is that the vast majority of market traders who do not practice what the book preaches will lose money. The good news is that those who do will most certainly prosper. As the numbers of the former outnumber those of the latter, the few will earn a lot from the many. The long (over 100 pages) psychology "preface" is extremely important to Aronson's body of work. I found it hugely interesting, but fear that others may not, or worse. In fact, the psychology preface itself indicates that this work will be reviled (my words) by the multitude. People do not like their sacred cows criticized. The problem is that most market practitioners use methods with little or no scientific basis. Even if shown evidence of faulty logic, people continue to believe its validity. This is also true in the medical profession as Aronson illustrated and which scared the daylights out of me. For anything to be scientifically testable, it must be possible to prove it wrong. However, many of the technical analysis disciplines cannot be defined. Thus they cannot be disproved. Consequently they have no scientific validity. They may have some anecdotal importance, but true science is lacking. Let us say that one of the market gurus espouses that when the chart of XYZ resembles "Pattern A", the stock is destined to rally. To test that we have to define Pattern A and we have to define "rally", and we should provide some time parameters in which to work or fail. The trouble is that the guru cannot define any of that. But the guru still believes in his work, and all of the investors who pay monthly fees for his expertise believe it also. Anyone who criticizes the guru or the validity of Pattern A is looking to get flamed. EBTA preaches that technical analysis research should be conducted like quantitative analysis research. Those who treat TA as a casual discipline will get casual results. The book is not an easy read, but it is an easier and much more interesting read than the "bible" of the CFA community, Quantitative Methods for Investment Analysis (DeFusco, et al.). I own both books and certainly consider EBTA more valuable than the CFA manual, worshipped by thousands. Don't expect to download all of Aronson's knowledge the first time - read it again. I did and learned more the second time through. Aronson is meticulous and provides "service after the sale". I had recently traded emails with him about an article he had cited. He was prompt to respond and discus the implications of our expanded research. I have the feeling that he is like this with everyone. In conclusion I have to say, that if you cannot do what EBTA preaches, at least get yourself a money manager who does. Bill Rafter President Mathematical Investment Decisions, Inc.

This is a tough book to review. The material covered is simply not covered anywhere else I've found, and it is absolutely crucial in building a scientific approach to building trading systems. As such, you pretty much have to read this book if you want to trade and not lose your shirt. On the other hand, it's got some fairly serious flaws. The author seems to be a "seat of his pants" proprietary trader who eventually got science-religion, and became a scientific trader. As such, it is probably more or less oriented towards people like him; people who may not have been exposed to ideas like "standard deviation" or "statistical distribution" before they read this book. I'm not sure it succeeds in explaining this issues. I found the explanations to be excellent and extremely clear; but I have a Ph.D. in physics, and have been thinking in these terms since I was a teenager. Will some 40 year old knuckle-dragger who has never heard of the Student-T distribution get anything from this? I don't know. I kind of suspect he won't. Can I think of a better way to explain these concepts to an older student coming to the ideas for the first time? Nope; certainly not. I'd probably just give them this book and hope for the best. The other flaw is also kind of a strength: the author "talks" too much. This book is over 500 pages long. The crucial material in it; the explanation of White's reality check and the Monte-Carlo analogue by Tim Masters is really only a couple of pages. Most of the other text is interesting and well written as the author is a learned and experienced man, but, well, Aronson could use an editor. I believe Ambrose Bierce once reviewed a book with, "The covers of this book are too far apart." This is unfortunately sort of true here. I'll say it again: this book is the only one I know of which deals seriously with the issue of data mining bias. This is what separates the men from the boys. It's easy to build signal processing techniques which find real signal in financial time series (and yes, they work a lot better than the lame TA signals the author uses), but more difficult to find out when these techniques are lying. I'm planning on giving away a piece of software you can use to find some kinds of signal fairly painlessly: I probably won't give away the "reality check" stuff, because that's the hard part. What would I have liked in the ideal world? Maybe a little less Popper and bad history of science, drop the specific test he did and add more technical stuff on the various forms of reality check. For example, the reality checks described here deal exclusively with simple entry points: how do you deal with more complex entries and exits and money management? There are ways of doing this for certain, but this book is only the beginning in figuring them out. What do you do about signals which have the Markov property, or, for example, what do you do with signal-finding algorithms which have the bootstrap property baked into them already? What about a data mining reality check for Sharpe ratio? What do you do when you have a signal with varying probability of being true? By this, I mean, you may have a signal you have determined has a 51% chance of being correct, and in some cases, you may have a signal which you know has a 54% chance of being correct (you probably will never have a 99% correct signal; not in finance anyway); what you do with such signals is different. Sometimes you have a signal where you have no idea what the probability of success is: these need to also be handled differently. There are also issues with correlations between trading systems, bet sizing ... I supposed there are lots of issues like this which I would have liked to see addressed in a book like this, but until someone writes such a book, we have to make due with this book, Grinold and Kahn and SSRN. Speaking of Grinold and Kahn, while this is probably outside the author's field of expertise, application of these ideas to classical macro/microeconomic models used in the "alpha plus" investment funds would have been incredibly awesome. Those fields use plain old regression to build their accounting based models. G & K's book doesn't mention much beyond Student-T tests for backtesting (Elton and Gruber does mention the bootstrap without telling much on how to use one). Applying the machinery of White's reality check to this "arbitrage pricing model" sort of thing would have been a huge win: far more interesting than using it on various technical analysis methods as he does in the second to last chapter. Anyway, that's how I would have rolled.

This is far and away the most practical and unique trading book ever published. While some may not like the fact that unlike most books about trading it does not contain, nor does it claim to contain, a misleading "holy grail" set of entries and exits, it does in fact contain the key to building a successful trading model from the ground up. Anyone wishing to succeed at trading system design must first develop a skill which is surprisingly subtle and exceptionally rare: the ability to distinguish between a good result and bad result. This seems so obvious on the surface that it is overlooked by the vast majority. However, it is absolutely not trivial and this skill is a key differentiator between the successful few and the majority who continuously struggle with trading. This book contains a series of tools and concepts which, when mastered, will equip the trader with the ability to understand when they're likely to have found something real and when they are simply fooling themselves. These techniques are also extremely useful in evaluating the ideas of other authors in a much more realistic way before attempting to deploy them in a live trading account. Because of this, Evidence-Based Trading Analysis complements a collection of trading books quite nicely and should be included in every serious trader's library. While it is never pleasant to find out that despite having worked tirelessly on a trading system that one's methods of development and analysis are faulty (I believe this accounts for the majority of the negative reviews this book has received), it is far better to learn and apply proper methods BEFORE deploying poorly constructed models rather than discovering one's error through mounting financial losses. In summary, while this book may deliver a painful message to some it also provides the tools that enable the reader to truly progress in a way that translates to real-world success. Anyone who wants to elevate their trading system design process from the realm of the weekend hobbyist to that of the professional absolutely must master the techniques contained in this book. I cannot recommend it highly enough.

I'm not a writer, but after reading a lot of TA book this book is nothing but a piece of garbage and have no value. He repeats himself dozens of time on the same issue, his philosophy simply makes no sense. Just to give one example when he wants to prove DATA mining is bias from the old testament codes, he wants to know why the coder could not predict the 9 11 attacks, after all he was able to find it in code after the fact. Being familiar with the Torah codes, its not possible to predict anything because no one is able to know whats going to happen, so if we don't know what to look for we can't find it , we can only find something if we know what were looking for. Its like telling someone in a big bags of millions of items go find something & I'm not telling you what to find, do you think he will find it ?. The way he makes away with pattern recognition with absolutely no evidence, just because he he wasn't able to prove that it works (so if i wasn't able to prove it, it must be of zero value. The fact is that 1000's of people have made money using Edward methodology. There is a lot to write but i will leave it here. PS by the way I'm selling the book

You are not going to find performance summaries or Tradestation code. But, what you will find is in depth thinking about the scientific method and statistics applied to trading. Aronson gets deep into both the how and the why. I do agree with reviewers and find it disheartening that a test of 6400 binary buy/sell rules did not find a tradable system. But they were simple, single rules. In chapter 9, he does make suggestions on next steps. This book is going to save me time in my backtesting. I found several things that I will do differently. Related to this, I liked his explanation of data mining and the right and wrong way to go about it. The 40 pages of footnotes give you an idea of the amount of thorough research that went into this. I added a couple books to my reading list from the notes. If you found books by Pardo, Kaufman and LeBeau valuable, then you are going to like this book. I have read many dozens of trading books, most of them systematic, and this has been the most important one for me.

The author mentions: "[while] the set of rules I tested did not give significant results many footnotes point to studies that do" and "I certainly don't claim that there are no TA rules that work. In fact I cite references to numerous peer reviewed studies that discuss TA rules that have proven out in rigorous testing. However, for the rule set that I specified, which I felt at the outset would contain some good rules, none had sufficient performance to reject the null hypothesis". In other words, the chosen rule set did not contain some good rules as the author expected. The question is then: why not to include in the book (= to submit to test and select those that in fact worked) some of these "numerous peer reviewed studies that have proven out in rigorous testing"? Any fear that getting them published would make them no longer useful? Well, but they have already been published as cited in the book. The author does mention that "the purpose of the book was to present a method of testing rules". But instead of just using this method to disprove every single study presented in the book, it would make a lot more sense to use the very same method to confirm at least some successful studies too. This would make the book a lot more useful, well-balanced and positive.

This is the only book out there that challenges the dogma of TA that many new traders fall prey to!. Not only it does that with sound logical arguments but also explains in painstaking detail how to test your rules to make sure that the returns are statistically significant. The author explains in detail the basis for the book and the ideas and how to implement them. Definitely its requires some sort of intensive study and it not for casual trader who is looking for some magic tricks. Ask yourself are the head and shoulder, the triangle etc etc patterns statistically significant?, Are you trading a pattern or a rule because you heard it works somewhere or did you test to see that it works better than a trade based on coin toss?. In short are you always taking the statistically significant trades?. If you cannot answer this question then you need to read this book. This book also has detailed references to various papers and articles which you can try to read to get that edge that every trader needs to find for himself. If you are a programmer like me you can implement software to test out your trading strategies!!. In short for system developers based on TA this book is a must read.

Testo didattico scritto da uno dei tanti teorici di borsa che probabilemnte non hanno mai aperto una posizione in vita loro. La prima parte consiste in una demolizione sistematica di tutta l'impalcatura teorica dell'analisi tecnica classica e francamente non dice cose del tutto sbagliate anche se si prende la briga di scomodare filosofi e naturalisti. Nella seconda parte va, in modo piuttosto inconcludente e contradditorio, alla ricerca del sacro Graal, quello che tutti vorrebbero avere ma che forse soltanto un veggente dotato di capacità metafisiche potrebbe avere. Da aggiungere alla collezione, come molti altri del genere.

An overdue examination of technical analysis from a scientific perspective. For too long TA practitioners have used overly vague terminology and methods for predicting the market. Presumably many thousands of investors have tried to put these into effect losing themselves money and causing heartache in the process. There is a role for TA but one that is based on reasonable testable propositions, this book is a major step in this direction.The sections on statistics were some of the clearest explanations on this topic that i have ever seen and helped me with my finance masters. This is not the typical TA book with 20 surefire ways to double your profits. It is more realistic but at the same time a little depressing ....success in trading (as in life) requires a lot of hard work , study and training ...there are no shortcuts. NOTE: this is not the typical TA book, do not buy it if you are looking for that killer chart formation (that worked in the past but probably won't in the future) but do buy it if you genuinely want to learn something about the market and how to backtest stock movements. From this perspective this is the best book available and should be on the shelf of any serious investor.

There's a lot of useful material in this book - there's also a lot of pseudo scientific bigotry. The scientific method is held up as the Holy Grail and without doubt it has it's uses - but it's only part of the story. As Einstein said - imagination is the most important thing. Once you've hit upon some innovative idea then the scientific method is merely a process of shaping it up. Half the book can be dismissed as the author attempting to constrain the world within the scientific method - the rest of the book is very useful - particularly for avoiding the hunt for fool's gold.

Good introduction on reviewing Technical Analysis from the scientific perspective