Trust the Math, not the Force
April 11th, 2010 by Eric Kesselman in General Guidance, TheoreticalSo Chris Liss (coupled with an annoying refusal to give me Alberto Callaspo) has been saying a number of things I want to respond to. The debate begins in my last post 'Intuition v Quants' and continues in the comments area, and then on to Chris' post on Rotosynthesis.
Chris has a number of objections, but the one I'd first like to tackle is his complaint that baseball is not a math game, and that since a baseball season is "organic", or non-repeatable (2010 only happens once) all of our projections will be 'wrong'.
I think its hard to argue that baseball is not a math game. We count statistics for all sorts of outcomes, and can say some pretty meaningful things about them. For example, I can tell you that if an average batter is facing a 1-2 count instead of a 2-1 count he is about 15% more likely to get a hit on the next pitch. I can tell you if a runner is on 2nd base with no outs, this is worth on average a little over 1 run. These are mathematical statements with quite a lot of probative value. In the sense that we are counting things, and can make probabilistic predictions about the future with a fair amount of accuracy its hard to argue that baseball is not a math game. This is true even though no math, or even much thinking necessarily is involved in the actions themselves. Math is a way of describing and predicting the game, its inputs, and its outcomes. Baseball (and I would argue pretty much all of life) is a math game in this sense.
I don't think Chris can argue that there are all sorts of situations in baseball (and fantasy) where I can say mathematically meaningful things like this. To whatever extent I can, these statements have real value.
What I think is really bothering Chris is the lack of complete information. He craves certainty and exactness. We will never know the real 2009 Derek Jeter, or his real baseline performance. Did he overperform expectations in 2009? Or did we underestimate his baseline performance? We can never know for sure. Worse, as Chris notes, baselines are often changing.
Chris seems to take from all this that because we can never know for certain, all projections are "wrong." This isn't true, and isn't really the point. The question is how useful they can be. Sure, I won't be able to tell you exactly what Derek Jerek will do in 2010, any more than I can tell you precisely whether the batter with the 2-1 or 1-2 count will get a hit on the next pitch. However, that doesn't mean I can't say some meaningful mathematical things about his likely outcomes. That is all a projection is. To the extent that I can tell you with a high probability what a player will do, that knowledge has value. For Chris though the lack of complete accuracy seems to mean the complete lack of value. Not true. To the extent that I have better projections than the field, I have an advantage on the field. I have more accurate information. I don't need perfect information to gain a big edge. Accurate projections aren't completely 'right', they're just a bit more useful than some others. Chris seems to suggest that since we can't know anything for sure, everyone is equally wrong except for those few people who miraculously happen to get it exactly right. Not true.
Chris also tends to note that these seasons are not repeatable and therefore that probabilistic forecasting isn't appropriate. It's really just another way of saying 'we can never know for sure if we were right.' Clearly probabilities of any given thing don't change depending on whether or not you can or do repeat them exactly in the future. Chris is just hung up on the fact that we can never point to something in particular and say 'this had exactly a 53% chance of happening." It's too inexact, too filled with variance, and uncertainty for him. In actuality, the better way to approach the issue is just to ask whether or not you can say something useful about the predicted outcome. I suspect Chris has a hard time saying that you can't. If you want to quibble over how useful or accurate a model or a projection is, fine. But clearly suggesting that all forecasts are "wrong" is not the way to get anywhere insightful.
All Bill has been claiming is: If we do agree on reasonable projections (and on most players we more or less generally do), but other owners can't price their expectations accurately- that costs them. Bill also likes to say 'If you guys aren't pricing Paul Konerko right by now, something is seriously wrong with you." Its hard to argue with this. If we more or less agree on who the player is, but cant agree if he's a 10$, 13$, 17$ or 19$ player someone is just flat out wrong. Being wrong costs you edge.
If our projections are meaningful (they are, subject of another article) then the model's move from projections to accurate values is also valuable. To the extent we agree on who a player is, the people with good models will price him better.
Tags: Bill Phipps, Chris Liss, Eric Kesselman, intuition, quantitative analysis, quants, rotosynthesis, rotowire
Loading ...
Two points, Eric – my point isn't that because your projections are necessarily wrong, and untestable in any event with any precision – due to the uncertainty about what's variance and what's a new baseline – that they're not useful in the abstract. Let's say if your life depended on setting a home run total that Derek Jeter would not exceed, but that some other player would, I think you'd be pretty safe with 40. But they're not useful under relevant conditions, i.e., competing with me in a fantasy baseball league. Because your margin of error is so large, it renders any precise conversion system worthless – again – false premises followed by sound logic do not lead to true conclusions any more than false premises followed by faulty logic do. Or – in other words – if your projections for the players you end up drafting – not the entire diversified pool – are off by 40 percent or more – that your conversion method which might be 3 percent better at translating them than some fantasy guy's (assuming that's even they case) might actually keep yours vastly inaccurate, but his conversion errors could just as easily move him closer to the real number, assuming such a number really exists, which it doesn't. But it could at least move him closer to the actual numbers the players will put up. Or it might move him further away. But I don't see how the conversion accuracy confers a major advantage.
Finally, I really think you guys miss the key point which is that not everyone plays this game the way you do. Not everyone needs to consider precisely what Derek Jeter will do in order to make a bid on him. You think this is either not true (I secretly have very specific numbers in mind that I inaccurate translate into Jeter's dollar value) – or I'm just totally like George Bush – going by my gut with no particular idea of what I'm doing.
Both premises are false. Unlike Bush (and I don't want to get political), I KNOW the underlying facts as thoroughly as just about anyone. And that's really the difference. You cannot use your gut the way Bill absurdly mischaracterizes it in his post – you can only do that once you've had a lot of experience and done a lot of tedious prep work. The difference between your approach and mine (and actually Eric, I think your actual method seems to agree with what I'm saying here rather than with what you're arguing), is that I trust in my own intelligence to steer me appropriately here, and Bill does not trust in his or anyone else's.
For a game like chess where a computer can just bludgeon you with processing power and contemplate billions of possible moves in a second, I get it. But when your projections are so inaccurate that only 40 percent of the consensus top-15 picks actually live up to their billing, I'm going to trust in my intelligence. I know the limitations of the math in cases like this. You guys assume the fantasy experts have no idea how to translate stats into dollar values, but that's not remotely true. We use standard deviation above replacement to generate our values, and believe it or not. I built the model! But whether you can improve upon it slightly – who really cares. I've learned from experience that it's not where the bread is buttered anyway when it comes to winning fantasy leagues.
As for Paul Konerko, I have no idea in the abstract what he's worth in our 10-team league. But I feel very confident that were we to throw him out there, and were I to need a first baseman, I would look at my remaining budget, look at the remaining first basemen in the pool and bid appropriately based on what other players had gone for and what Konerko's general skill set is. 25 HR, .270. ish player. If I had to guess I"d say he's a $12-13 player in our league depending on when he came up and who was left. If he were the first player, and someone got him for $9, and it was going twice, I might price enforce, I might not. I don't really care if someone saves a couple bucks on a guy I don't really want. But if he went for $6, I would say $7 for sure. Once he gets anywhere within range, I don't care unless it's one of my guys.
And keep in mind this is off the top of my head. I could check our projections and our converted dollar values, but I don't even bother. I don't believe our made-up projections and dollar values make any difference in my ability to win the game. And I certainly don't want to feel compelled to buy Konerko for $12 because my spread sheet says he's worth $13.
I totally concede that my actual method is VERY similar to yours (notes, fresh look, stats). For the majority of players I start with what they did last year, and I largely decide whether I expect something similar, slightly worse, slightly better, break out, etc. I generally think in two likely outcomes, for example player X will very likely perform similarly to last year, but theres a 20% chance of a big break out. I then come up with a price that is largely a modified market price from recent auctions, using my expectations.
That being said, I often feel during auctions that Bill has an advantage on me. I often look at his bids after the auction, and realize I was off, sometimes by just a few bucks. Every once in a while by a significant sum.
Can you expound a bit on your 'key point' that not everyone approaches this stuff in the same way, and why that matters? Sure, nothing stops anyone from bidding randomly, having no idea what or why they're doing it, having vastly different projections or valuations, or (my wife's preferred method) just bidding high for guys with cool sounding names. That might effect my bids in some places (like closers) but it really doesn't change my opinion of Derek Jeter's real value.
I think there are a lot of 'reasonable' bids given all the uncertainties you mention. But there are also times when you are completely wrong, and you might not realize it so cleanly without a model.
Oh my God, this stuff is tremendous. Thanks to Peter Kreutzer for turning me on to it.
Chris is making me realize (as I read all of this in one fell swoop) that maybe I'm not as agnostic as I think. He's also making me realize that – in my better auctions – the genius tends to poke his head out and get a word in over the agnostic. I've had auctions where I could have simply price enforced endlessly but maybe this isn't the best strategy at the time. However, perhaps its not as simple as it being genius vs. agnostic, but more of a holistic thought process. I'd say that its the genius in me prodding the agnostic toward a better result.
i didnt read into how they came up with this projection but just lokonig at the AL West i dont know, LAA under .500 and SEA at .500? i think SEA may be the front runner as of now, with their offseason. OAK and LAA, im not sure where they’ll fall. of course, id like OAK to finish first, realistically dont think itll happen, but its more of a wait and see, lots of questions to be answered (i could write a book going into the ’10 season). LAA, its a guessing game. i think itll be very close for 2nd. i think TEX will finish last, i havent followed them close this offseason, but they sending millwood to BAL and signing vlad and harden. whos gonna catch for them? dont see them finishing ahead of OAK or LAA.so for SAG1. SEA 85-90wins2. OAK above .500 without major/long injuries. maybe 83-87wins3. LAA 77-83wins4. TEX near .500, 76-80wins.bailey’s college # retired? thats a total shock to me, no offense. glad wuertz will be around for a couple years i hope.
Nice work on the blog. Hopefully you don’t get much comment spam, besucae comment moderation is an absolute chore, I used to maintain the Microsoft forums and it was effort. I like how Digg comments work, user moderation does half the work for you then and you don’t need to babysit trolls and spammers!