On June 11th the McClellan Oscillator (as measured by TC2000) dropped below -200. That night I published a system related to this oscillator which bought the SPX on a drop below -200 and sold when it moved back above 0. The system was 17 for 17 since 1986. The exit trigger came on 7/17. In this case the system would have lost about 5.6%.The CBI is another breadth tool I use. On July 1st it reached 10, which I will many times use as a buy signal. Going back to 1995, buying the SPX when it hit 10 and selling on a return to 3 or lower would have resulted in 18 winners out of 18 trades. On Friday the 18th it returned to 3. This would have resulted in a loss of about 1.9%.
So now that the perfect records are ruined, how does this change the risk / reward moving forward?
First a story…
After I got married my wife and I went to Hawaii for our honeymoon. There was a concierge in our hotel that offered all kinds of 1-day packages of things to do – snorkeling, trips to other Islands, etc. One thing we thought looked exciting was a helicopter tour of the island. She told us that the helicopters were all flown by former military pilots and all new and state of the art. The company had been flying for 15 years and never had a crash. We were looking to sign up for something in two days. We told her we’d talk about it and come back the next day to sign up.
After we left we agreed the helicopter sounded the best of the options for that day. We were sight-seeing the next day and figured we’d sign up in the afternoon after we got back. When we got back to the hotel, before going down to the concierge, we flipped on the tv. The news was on and the top story was a helicopter crash that took the lives of 7 people. It was the company we were planning on signing up with and the tour we were planning on taking. It was pretty alarming to say the least.
My thought after seeing the news story was that tomorrow would likely be the absolute best time to take the helicopter tour. One crash every 15 years, and it happened today. No way they crash 2 days in a row. My bride had a different perception of the risk.
We went snorkeling.
In reality the crash had no effect on the risk of flying in the helicopter. Perhaps extra precautions would have been taken following the crash, but for the most part if the chance of crashing was around 1 in 5,000 on Friday, then after the crash on Saturday, the chance was still 1 in 5,000. It wasn’t any more (as my wife perceived) or less (as I perceived) dangerous.
Systems with perfect track records should be viewed in much the same way as the Hawaiian helicopters. The chance of a McClellan Oscillator signal or a CBI signal generating a profitable trade was never 100%. Now that they’ve encountered their first loss, should the risk be viewed differently? Not in my eyes. It should have been viewed at less than 100% before the failure and it should be viewed around the same after. I’ll treat breadth indicators in general and these two in particular in the same way I did before. Should they begin to provide false signals more often than expected, then it may be time to re-evaluate.
Traders should attempt to use any edges they identify to their advantage. They should not view any edge as a guarantee, and they should not be swayed too much by a small sample set. It’s been almost exactly 8 years since the helicopter crash and I don’t believe that company has had a crash since. It’d be nice to see the breadth indicators put in a perfect track record for 8 more years. If they did it still wouldn’t change my approach.
A wise and successful man told me many times "Facts are our friends."
ReplyDeleteHi, thanks for this. Am curious - how would you differentiate this situation from the turkey problem? Throughout its life you'd think that since it has been alive all days previously you can continue to count on it being around...until it gets slaughtered. Missing once after going 18 for 18 can be ok, but what if it is 0 for 4 after that?
ReplyDeleteThe question is - what are your implicit assumptions about risk? You seem to making the assumption that all helicopters fly with some baseline risk due to maintenance issues and the probability of pilot error, and that the recurrence rate of events would follow some sort of distribution that would place the probability of two events happening close together way out on the wings, so to speak. (What distribution would describe this?) But these events will happen - and - the "baseline" rate of accidents will change as new trends are established, helicopters age, maintenance schedules change, pilots have issues and hit the bottle...
ReplyDeleteAs an example I would submit Taiwan's China Airlines...
this is just a tack-on to the previous two comments, i suppose -- but one of the more effective arguments about risk management made in taleb's last book (or at least i think it was there) was the contraposition of three interpretations of a string of coin flips which turn up heads 20 times in a row when asked, "what will be the outcome of the next flip?"
ReplyDeletethe first (call it naive) is that the next flip MUST be tails, because "it's due!"
the second (call it statistical) is that one cannot predict -- the odds haven't changed, and the odds are and were 50/50.
the third (call it pragmatic) is that the next flip will very probably be heads -- as, following such an aberrant string, one can safely assume the coin isn't fair and is weighted to produce heads. in other words, the test conditions were not as advertised -- the setup is a fraud.
i agree that no indicator is perfect and one has to assume false signals are in the pipe. but whenever something like this happens -- particularly when two separate reliable breadth indicators give false signals in close conjunction -- i think it behooves a technician to question whether or not the test is still as advertised, or if in fact something different (or at least not seen in some time) is happening in the marketplace.
Any methodology that provides an edge, once published, will go through a stage of diminished return until an equilibrium point is reached. I believe the same thing is happening to your method of using the CBI and McClellan.
ReplyDeleteHenry
But it is important to remember that result of the financial market are not explained by the normal distribution (bell curve). Or at least it is not under the time of ONE person's lifetime.
ReplyDeleteOne signal might work perfect during bear markets, and another during bull market, or some other more diffuse filter.
So the next couple of years the McClellan signal and the CBI signal might not work since we are under totally new market regimes (a term that Steenberger uses in his site Traderfeed).
Some signals are also prone to trend, and the reason they worked so well is that the trend for that signal has been intact until now.
I'm wondering, since some of the previous commenters are ever so slightly poo-pooing quant trading, what the alternative is?
ReplyDeleteAre you all advocates of completely discretionary trading?
I just think that there is a big and largely unknown picture from which small parts of it can be discovered from time to time.
ReplyDeleteRational decisions are based on a) available evidence and b)some kind of predictive element on what the rest may be.
Given that the evidence is small and b) is guesswork anyway, one can't be too confident about putting too much weight on one or even very many indicators, be they quant or qualitative.
Does that mean its discretionary? No, it feels more like its forced because the alternative is to do nothing until you can get the entire picture, which is an impossible task.
Seems to me all of you, save Woodshedder, are somewhat missing the point.
ReplyDeleteOB/OS systems are based on emotionalism. This is a constant of human nature and never changes.
Any time a “reliable” system like an OB/OS algorhythm or a Seasonality bias fails to yield the expected result, rather than questioning the underlying rationales, one more appropriately “goes META”.
One tries to look at the situation from one “quantum-ring” higher. Or put another way, as the technician Carl Swenlin puts it, one “drills down” another level.
What does it MEAN that this is occuring? What is the Market “saying” with this anomaly?
Something unusual is going on-- what does that MEAN? What does the anomaly or disparity portend for the near future?
I believe there are two big events that all investors should hard-wire into their consciousness, should think about 24/7, waking, sleeping and dreaming-- with about 1% of their minds. That is, always somewhere in the back of one’s analysis, keep that awareness. The way a small bird is always aware of the possible location of the hawk. The way a gazelle or buffalo is always conscious of the possibility of tiger or lion.
The first is the Japanese stock market’s decade-long flatline, and how that came about. The second is the crash of 1987.
In 1987, prior to the Monday massive single-day decline, there were several periods in the preceeding days where OB/OS oscillators all hit levels of oversold so extreme that there were “setups” all over the place which had previously been 100% successful in the past.
These trades all got blown to Kingdom come.
For years afterward, the great pioneer econometric forecaster Norm Fosback ran a periodic Feature in his Newsletter called “Indicators that Successfully called the Crash -- Where are the Now?”
These were a handful of time-honored Indicators, NOT of the OB/OS variety, which were solidly negative (but in no way perma-bearish) well before the black Monday event.
Until human emotionality goes “out of style” it is not possible for a cleverly designed OB/OS system to go “out of style”. But it is possible for them to produce a losing trade, and if that happens, it usually means that the Market is signalling.
I employ a proprietary variant on Mr. Fosback’s classic Seasonality Trading system, and I usually bank coin with it each month. When I bank a loss instead, I don’t scratch my head or question the underlying rationale, I “go meta” and ask if this might portend a weaker month ahead than average-- and usually that is the case.
I believe that from the May highs, to the recent lows, we’ve had a mini-crash... just in slow motion. Had it occurred in 2-3 days it would’ve been spoken of in such terms. I also believe that the atypical performance of Rob’s superb setup-models was a harbinger of a special kind of weakness.
If one believes that, and sees an emerging OB/OS failure, they should then bounce to a secondary set of indicators based on valuation or cycles of the moon or whatever... but NOT sentiment or breadth-- as Mr. Fosback advises.
I took the loss in the McClellan Oscillator system trade, in my Aggressive Portfolio. I didn’t second guess it. However, I also let the emerging failure guide me into an even DEEPER level of caution in my core Conservative Long-Term Portfolio. And that stood me in good stead, as I contemplated the timing of a periodic rebalancing.
I believe this is the way to use such OB/OS systems. The trade is a trade, but it is also a longer-term indicator.
(By the way, Rob, I was a guest of the US Navy on their Kauai base, in August of 2000, and I was guided by my friend there to a helicopter tour company with ex-military pilots and took the ride-tour and it was awesome. You mentioned “8 years ago” which would be 2000. WHEN did you see that article about a crash (ie. when were you there in Hawaii) and by any chance do you still recall the name of the company?)
Daniel
Too much jargon for me I'm just starting out here.
ReplyDeleteAre you saying that if you get a fail you need to consider if the current model is a glitch and can typically continue to use it.
Unless you think there is (and one must always be aware of) a greater systemic problem (a "meta" indicator if you will) in which you will have to change the model entirely (jump turkeys if you will)?
Sorry, Ryansoh, most readers of Rob’s blog are NOT just starting out, and so we tend to sling terminology, and so your confusion is quite understandable.
ReplyDeleteYou wrote, “if you get a fail, you need to consider if the current model is a glitch...”
No, quite the contrary. The MODEL is fine. The individual OCCURRENCE is what is atypical, or in your term, a “glitch”.
All oversold markets can get more oversold, before they reverse. All overbought markets can get more overbought, before they turn down. They can get WAY more extreme.
The art of constructing an OB/OS system is to find some logical average point where MOST of the time the market reverses fairly soon after you take action.
When it doesn’t then do what you “expect”... that’s when it may be offering a clue to some unusual deeper problem, or set of circumstances... which is what you then attempt to “drill down” after and explore, using OTHER types of indicators, not based on overbought or oversold.
Of course, Ryansoh, you may be right that a particular OB/OS system that one is using DOES have a key design flaw... and THAT is the real problem. Not some rare market environment.
But one does NOT find that with a system that Rob has designed and extensively tested. It’s rare to find someone you can trust like that-- but readers of this blog know that carelessness in design or testing is not a problem found HERE.
However, all systems that attempt to be “anticipatory” of a trend reversal will sometimes be wrong. Even the best. And such failures are not design flaws. However, one can do more than just record one’s gain or loss. One can also ask WHY... And then the ‘system’ also becomes a broader ‘indicator’.
Thanks, I only just found this blog over the past week (thanks to Google's recommended blog feeds). I think that reading investment blogs like these will help me in my career as a buy side analyst (em equity, but with a quant macro theme going), so am looking to learn as much as I can, as well as provide feedback (in whatever noob capacity).
ReplyDeleteI think a lot of what you said makes sense (apologies if I have misunderstood again). The idea is that the OB/OS indicator that has been constructed here is pretty robust and it's not easy to expect that something would be overlooked/not considered.
Rather, if an occurrence is not what you'd expect it makes sense to kick the tires, check the water ("drilling down"), instead of abandoning it entirely. The vehicle itself is well built and works fine, but sometimes it is not going to get you there, so one should then consider other forms of transportation (other indicators).
One last response, then I must go...
ReplyDeleteNo, Ryansoh, you’re still not quite getting it.
One DOES NOT kick the tires, or question the system or the model.
One uses it again, next time, exactly as one did this time. And again and again. Just as it is. These models work on probabilities and odds. The model is treated as OK, just as it is.
Another analogy would be a reliable meteorological testing instrument.
If it starts giving strange readings, say barometric pressure or somesuch, and it truly is tested and reliable, what it might be saying is that a VERY unusual storm is brewing up.
One does not then throw away, or even tweak, that barometer. One instead looks away from it-- the device has told you all it can-- and instead starts looking elsewhere for more useful information.
Maybe one then squints at the horizon, looking to see if there are any suspicious funnel clouds way in the distance.
This is of course a crude analogy. But hopefully it helps illustrate the key point, which is that ALL models based on trying to get the odds, the edge, the probabilities on one’s side, will sometimes result in a loss rather than a gain, and that is not necessarily a defect in the model.
This is why one always speaks of results over many instances, results generally speaking, results on balance. On any one occasion-- even when simply rolling dice-- pretty much any result can occur.
Good luck. You seem to be a quick learner.
I'm wondering, since some of the previous commenters are ever so slightly poo-pooing quant trading, what the alternative is?
ReplyDeleteAre you all advocates of completely discretionary trading?
not at all -- but it is important to know the limitations of systematic trading all the same. to apply the model blindly is no better than applying your "intuition" blindly. one has to understand why the model works and what assumptions its function is predicated on. only then will you be able to realize the weaknesses of the model.
this is why, imo, the best technical traders i've ever met are often also the most experienced with their model and the most willing to study its conditionality.
I employ a proprietary variant on Mr. Fosback’s classic Seasonality Trading system, and I usually bank coin with it each month. When I bank a loss instead, I don’t scratch my head or question the underlying rationale, I “go meta” and ask if this might portend a weaker month ahead than average-- and usually that is the case.
ReplyDeleteI believe that from the May highs, to the recent lows, we’ve had a mini-crash... just in slow motion. Had it occurred in 2-3 days it would’ve been spoken of in such terms. I also believe that the atypical performance of Rob’s superb setup-models was a harbinger of a special kind of weakness.
right, that's exactly what i'm trying to get at. when highly reliable setups fail, there may be a message there -- the conditionality of the models may have been violated, and something out of the ordinary may be afoot. one has to be aware of the outlier possibility, because it can be very damaging!
One aspect of all this that no one has mentioned is the small sample size of the "survey". This is really the most limiting factor of trading systems.
ReplyDeleteWe've only had about 100 years of developed financial markets with statistics attached. Even if a trading system had one signal a year, 100 trading events representing 100 years is still a very small sample size.
As a avid poker and blackjack player, card simulation "sample sizes" always run into the millions to get an accurate picture of the "edge" of a card playing system or a combination of cards in your hand.
A Phase III drug trial would be heavily criticized if the patients in the trial only numbered 100.
BTW Rob, didn't the Celtics lose 18 games in a row the year before last? :) We could also be prone to make erroneous assumptions about the Celtics if basketball was a brand new sport. Such as having no chance to win the championship this year? :)
A final word, and then I must exit this ‘Thread’ permanently..
ReplyDeleteIf G.Marius should return and review a final time-- or for the benefit of the self-described novice, Ryansho, should he return-- I want to call his attention to two very right-on observations by G.M.
First, his keen comment about ‘outliers’ (colloquially, ‘Hundred Year Floods’, for ex.), and the need to be aware that the ‘extra-ordinary’ does happen occasionally, and snake-eyes does come up occasionally in dice.
Secondly, his comment to Woodshedder (with whom I agreed from the start, see above): “..One has to understand why the model works, and what assumptions it is predicated on.”
George Soros, the great Hedge Mgr, said in his worthwhile book ‘The Alchemy of Finance’, that what made him nervous was when he didn’t know what to be nervous about. In other words, if he could not spot the inevitable existing weak spots in a Model, or the exit-trigger of a Strategy, or what would ultimately prove the fatal flaw of an evolving “belief system” moving the market, if he did not know these things IN ADVANCE-- then he would not initiate a position in the first place.
Rob, however, factors these things into his modelling, as best he can of course. He clearly distinguishes research studies from actionable algorhythms. (By this last, Ryansho, I mean complete rule-sets, so objective and unambiguous that they could literally be programmed into a computer to execute.)
So I would disagree with G.Marius on one point. He wrote, “..to apply the model (Note- EITHER CBI or McClellan Oscillator,it’s the same point) blindly, is no better than applying your "intuition" blindly......”
Actually, this is a non-logical statement, along the lines of Betrand Russell and Logical Positivism and linguistic philosophy. Apples with oranges.
These kind of models MUST be applied “blindly”, mechanically. That is their point. Intuition, which can also be called ‘trading sense’ or ‘trading feel’, is by definition blind and can't be applied any other way.
To me the most interesting Models have always been ones that allow the user a limited add-on module of discretionary over-ride or alteration, in response to the promptings of intuition-- should they occur.
With that I leave it be, and bid you all good trading.
Daniel
Excellent discussion guys.
ReplyDelete