Entering the Fray
April 15th, 2010 by Derek Carty in General Guidance, Prediction, Theoretical, UncategorizedThe on-going debate between Chris Liss, Eric Kesselman, and Bill Phipps is a terrific one that makes me very happy I was invited to participate in the CardRunners league this year. These are the kind of theoretical discussions that I love to have, and I feel like it’s being framed and discussed much more intelligently here than it often is among baseball bloggers.
Anyway, I thought this would be a good time to jump in, introduce myself, and give you a feeling for who I am and what my thoughts are about fantasy baseball.
I’ll start by saying that I’m far more on the “quant” side than “intuition.” Some may argue that this is out of necessity because I’m just 22 years old (incredibly young in fantasy years – were I instead playing poker I’d probably find myself one of the senior members at some tables) and don’t have the experience of my fantasy counterparts. In fact, I’ve been playing fantasy baseball at the higher levels for just two years, giving the other fantasy guys in this league exponentially more experience than me.
I don’t view this as a disadvantage, though. I view it as an opportunity to distance myself from some of the preconceived notions my fellow fantasy analysts may have, from the groupthink they may unwittingly be involved in, and from the habits they may have slowly and unknowingly developed over the years. It allows me to take a step back and forces me to think rationally, logically, independently, and quantitatively about things because I don’t have that experience to guide me and to fall back on – and I don’t necessarily think that’s a bad thing.
When I won LABR as a rookie last season, I had never participated in an auction before. Not one. All I had to guide me was theory, and I wound up doing pretty well. That’s not to say that I now consider myself a perfectly adept auction drafter (I’m not), and that’s not to say that experience wouldn’t have allowed me to do even better (it would have). I’m still learning the ins and outs of reading the market, and with more experience I may not have had quite so much money left over at the end of Tout Wars this year. Still, I think the fact that I’m so much younger than my fantasy counterparts leaves me with a unique perspective on things.
What I find to be quite interesting is that my views in almost all respects of this debate seem to conflict with those of Chris Liss – the other fantasy guy in this debate and a guy whom I have a great deal of respect for –and, from what I can tell, seem to mesh more with Bill and to an extent Eric (or perhaps just with Eric’s devil’s advocate stance). I think this quote from Chris best exemplifies where we differ:
“It's true I have a vague projection, but who cares if I have Jeter for 22 steals or 25? I know it's just a guess anyway. I put him at 22-ish steals if I bother to think about it which I don't when I'm drafting. The more important thing in many ways is my overall impression of Jeter compared to the shortstop pool, not just in numbers, but in reliability, durability, etc. I'll target him as a nice piece of my overall puzzle. I'll determine how important he is based on league depth, quality of shortstops, etc.”
I’ve heard this line of reasoning from fantasy experts (even those who have a more quantitative reputation) before – Ron Shandler, for example, has adopted this approach in recent years– but I just can’t agree with it. Sure, all projections are a guess, and we can only achieve a certain level of accuracy, but by watering our projections down further, all we’re doing is reducing that accuracy. Even if all you have is a rough idea about a player in your head without a precise projection, you implicitly do have some sort of projection for that player. And whether you have an explicit numerical projection for a player or not, you’re still obligated to pay for that player’s numbers. And at the end of the year, those numbers will have been worth X dollars (whether we know – or care to know – precisely what X is or not).
This game is inherently about numbers and getting as many of those numbers as you can for as little money as you can. Whether we adhere to the notion of “projections” or not, at the end of the year we will still be judged by it (if only implicitly by our league standings). And by ballparking our expectations, we will necessarily be removing some of that accuracy.
Even if I were to concede that projections are “necessarily wrong” as Chris and many others claim (and I will only concede this point for the sake of argument – I actually consider this to be a vast oversimplification) – what is the alternative? Chris has said, “But when your projections are so inaccurate that only 40 percent of the consensus top-15 picks actually live up to their billing, I'm going to trust in my intelligence.” Let me ask you this, though. Based on your intelligence (which I’m in no way questioning), who would your top 15 players be? I’d wager that, assuming most projection systems actually do adhere to this 40% rule, your top 15 players (selected by whatever method you deem acceptable) would do exactly the same. It’s not as if forsaking the use of projections is going to lead you to something more accurate than them.
In pointing out his qualms with projections, Chris also said, “So your model either has no outliers or random outliers. That's the trouble with projections – they're impossible to do right not only because of variance, but because they're either too timid, or too speculative. So I've found it's better to just practice identifying the breakout players, and go the extra buck for them at auction (within reason).”
But why can’t I indentify the breakout players in my projections? What are you doing differently?
“If you could build a model that synthesized those variables better than my brain, I'd be concerned.” How could it not? Chris, what if you took whatever is coming out of your brain and wrote it down – quantified as best you could. Then combine it with whatever truly objective data you find appropriate, and put it all into a model. Why wouldn’t that beat whatever rough estimation your brain (or anyone’s brain, for that matter) comes up with? The ideal projection system, in my mind, will have a foundation in the numerical, and based upon scouting data and context (that is, whatever context can’t be expressed numerically – most of it can be), will be altered where appropriate. I don’t see how a human brain can come up with a more precise and accurate expectation for a player than that. The human brain, as magnificent a thing as it is and as much as it can do, has severe limitations, especially when it comes to taking a confluence of factors into consideration and trying to reconcile all of them. It’s just not built to do that.
The last point I’d like to make relates to Chris’s description of his intuition method.
“Moreover, I don't even bother to do projections in preparation for my auctions. I merely research all the facts about every relevant player (past performance, playing time, team context, age, health, historical context, etc.) and put him in a rough order on a list.”
“I try to know the player pool deeply (from historical performance, to health to team context), use past experiences with pricing in the given format and adjust for market conditions on the fly.”
Why are these – and other similar, context-driven factors – not something that can be incorporated into a quantitative person’s game, and more precisely so? Health, team context, historical performance, ballparks, lineup position, etc. are all things that can (and should) be incorporated into any good projection, even if it takes some intuition to do so. Being vague about your expectations only serves to decrease your eventual accuracy.
Finally, I know it seems as though I’m picking on Chris a lot here, but that’s not the case at all. Chris is a very talented and accomplished fantasy baseball player/analyst, and I have all the respect in the world for him. I enjoy healthy debates, though, and I find it quite interesting that our views differ so greatly here.
Loading ...
First off, Derek – thanks for weighing in – good to have you joining in the debate even if you do agree with everyone else, and I'm left to groupthink all by myself.
But seriously, let me address some of your points:
Sure, all projections are a guess, and we can only achieve a certain level of accuracy, but by watering our projections down further, all we’re doing is reducing that accuracy. Even if all you have is a rough idea about a player in your head without a precise projection, you implicitly do have some sort of projection for that player. And whether you have an explicit numerical projection for a player or not, you’re still obligated to pay for that player’s numbers. And at the end of the year, those numbers will have been worth X dollars (whether we know – or care to know – precisely what X is or not).
I think you're jumping from present to future but not quite acknowledging the leap. At the end of the year, a player's numbers are a certainty. They can be translated into dollars, depending on the exact makeup of the league (obviously if players are apportioned so SB is very tight, then Rajai Davis had more value). But let's assume an average amount of looseness or tightness across categories, assuming there is such a thing in our format that transcends the quirks of the league's specific owners and how they change over time, yes, you can get a sense of how valuable a guy's stat line was. And you can work backwards and figure out pretty well what you should have paid for him.
But going forward in the present, that's not the situation presenting itself. We can only guess. And so you're not playing for a player's numbers – almost certainly not the ones you're guessing about – you're playing for his potential numbers. And so there's the question of whether it's better to estimate his potential numbers by removing the human, the player you've observed over the years and replacing it with a stat line. You can try to put all the information you've ever known about that player (and historically comparable players and players generally and the league and humanity, physics, etc.) into that set of numbers you have for him, and once you do, the player disappears. It's just x HRs, y RBI, etc.
You know this is just a guess and liable to be wrong not only due to variance or injury luck but also do to an unanticipated change in skill set. But be that as it may, you can't a model without a stat line, so this one is it.
So I don't get how converting the information you know about a player into a single stat line increases precision or accuracy. This is a totally separate question from whether Bill's conversion model of numbers into dollars is better than yours or better than nothing. I'm simply asking whether you think settling on something specific is the same thing as increasing accuracy. it seems to me by settling on something specific you just render it more able to fit into a model.
This game is inherently about numbers and getting as many of those numbers as you can for as little money as you can. Whether we adhere to the notion of “projections” or not, at the end of the year we will still be judged by it (if only implicitly by our league standings). And by ballparking our expectations, we will necessarily be removing some of that accuracy.
Is the game inherently about numbers? Or is it about players? When you buy Rajai Davis, you didn't buy 40 steals. You bought Rajai Davis, and you're damn well not assured of anything except whatever he ends up doing. Players produce numbers, of course, and those numbers determine our fate. But you can't buy numbers directly as you seem to be implying. If we knew the year-end stats ahead of time, then yes, you could buy numbers. But at auction time, you're buying players.
This might seem trivial but I think it's a big leap that a lot of people make. We have some tools to solve problems, and we're eager to use them, so we make some subtle or not so subtle assumptions that convince us beyond any doubt that these tools are optimal.
I'm not saying they don't have a potential use. But I think we're very eager to equate players to specific stat lines not so much because we have a great basis for doing so, but because if we don't, we can't think of a reliable way to use our model.
Chris, you're absolutely right that when we buy Rajai Davis we're buying Rajai Davis and not 40 SBs, but that's the nature of the game. No method – ever – is going to tell us with absolute certainty what a player will do. Never. So we do the best we can and let it be. Because we don't have that 100% certainty doesn't mean we should abandon method and reason, though.
No matter what method we're using – genious or agnostic, intuition or quant – we will always have that uncertainty. So it becomes a matter of choosing the lesser of two evils, doesn't it?
I'm not abandoning reason – on the contrary, I think you need to have good reasons to bid on players. And I most certainly have a method – it's just not the same as your method.
The question the quants have not answered – or at least to my satisfaction is this: why is it optimal to translate facts about players into projections in the first place?
The answer I think Eric gave is that our results are measured in numbers. That's true, but how is that an answer to my question?
What we need to do is gather facts and buy players. We don't produce the numbers, the players do. We don't aggregate the numbers, Sportsline does. We need to understand what our players are capable of, but it's impossible to know precisely. If we decide to write down a precise number, we're not getting closer to the truth. We're just writing down a number. From a knowledge standpoint, we're in the same boat whether we write down the number or not.
So why write it down? Why get specific? Because we have models that can work with that specific number and tell us what it means in dollars in this context. So we're settling on specific numbers not because it's more precise as you allege (clearly, your knowledge of the player has not increased just because you picked a number), but because it's a useful input for our pricing model.
I suppose the idea is – well our best guess projection is as good as it's going to get, so let's just throw that dart and translate it really precisely in dollar values.
So the model isn't doing anything to further your understanding of the players, or increase your knowledge about them. All it's doing is forcing you to make a specific call ahead of time with a number, and promising to give you a really good dollar value for that specific call.
To the extent your call is right, (and we know you haven't learned anything new by writing it down), the pricing model will give you a good dollar value. To the extent your call is wrong, it will give you a bad dollar value.
I'm just going to try to call the dollar values directly from my knowledge of the player. And not even dollar values but bids – facts to action. To the extent my call is right, I'll get a bargain. To the extent my call is wrong, I'll get a bad price or miss a bargain. I don't see why guessing Derek Jeter to be a $28 player is any more perilous than guessing his precise stat line. Both are estimates based on facts we know about him.
So I don't really see the point of going through the projection exercise unless you have a fetish for pricing models.
Now if I did bother to make projections (took the leap at that phase), then the accuracy of my pricing model might matter. But I just make the leap at the auction table, while you make it at the input side. I'm not translating from facts to projections at all – because that step doesn't gain you anything except to make it suitable for loading into your model.
It seems like people like using spreadsheets or perhaps more complex software, and they derive enjoyment out of building models so they want to make their bet earlier. I just make it at the auction.
I think this debate is shaping up to some seriously awesome sidebets throughout the year. It seems that their are no lack of opinions going on.
"If we decide to write down a precise number, we're not getting closer to the truth… So why write it down? Why get specific?" This is where we disagree, I think. We write down a number because if we're rounding numbers off and just coming up with a rough estimation, we're necessarily going to be losing some measure of accuracy. To your question of "why is it optimal to translate facts about players into projections in the first place?," it's for this reason. In the end, all of our picks will be implicitly (at the very least) judged for their accuracy. By making rough estimations, we're losing some of that accuracy. I'm not one to get hung up on precision ("My projection is to the ten-thousandth digit so it's better!"), and precision is surely a very different thing than accuracy, but I do believe that if you're too imprecise you are giving away some measure of value.
"I'm just going to try to call the dollar values directly from my knowledge of the player." This is my biggest qualm. Don't you think, Chris, that there is something lost in the translation here? Is your mind making that translation, from raw data to price, with 100% efficiency? I don't believe it is, because I don't believe it can. How can the human brain process all 8 years worth of Nate Robertson's career, his stuff, the fact that Comerica decreased Ks by 5% and Florida increases them by 10%, that Comerica increased HRs by 1.5% and Florida decreases them by 2%, that the Marlins aggregate road park schedule will boost HRs by 4.5%, that the NL boosts K/9 by 0.57, that John Baker seems to boost K% and decrease BB% by a couple percent each, and a whole cacauphony of other information and context – and NOT lose something in the translation? I don't see how it's possible.
Even if you're taking all of these things into account (and I'd bet you are), you're necessarily doing so with rough estimations. That leads to a heck of a lot of rounding. When it all ends up being combined, in your mind, how can you trust that your brain will assemble it all more accurately than a computer can? The computer will do so with 100% efficiency, every single time. Can you honestly say that your brain will assemble all of that data with 100% efficiency evey single time?
Finally, even if your aren't using projections, it seems to me at least, that a model like Bill's may actually be picking up value twice-over (how much value is another matter entirely). Once for the less-than-100% efficiency in considering all the data and context, and again for the less-than-100% efficiency in converting that data and context into a price.
I'm going to raise one more qualm about Derek's recent post. The recitation of adjustment factors as a way to translate skills into some sort of neutral value is a subject of great interest, but I think as science it is seriously flawed.
Those numbers are all small samples, subject to considerable variance, and by the time they can apply to a single player that player has likely moved on to another phase of his career path. Are they irrelevant? Hardly, and we're better off for having them for the way they help us look at the issues of individual players. But I haven't seen an improvement in player projections because they incorporate all this information. The noise in the data is substantial.
I'm not arguing against data, but I think identifying what is meaningful, what is suggestive and what is hokum is kind of a key issue.
I'm going to weigh in one more time, though I don't seem to be having any effect shaping this debate. The bottom line is that there are a plethora of issues here, and reducing them to Chris's "intuition" vs. the virtues of having the best numbers possible is pointless.
It makes no logical sense to say that less information is better than some information, but that doesn't mean that having the best information available can be reduced to a set of numbers that give you a strategic advantage. Or, maybe it's better to say, that knowing what stats are worth is a good thing, but drafting off your projected stats is almost always a bad idea.
Why is this? Because the information that matters more than player projections is the market's read of a player's upside. When Chris bought ARod in our league, he was buying a 12 team price in a 10 team league. That's a discount. It's also a reach. Will it work out? Half the room says no, half the room says it might, but wasn't willing to go the extra buck. Thus is value established.
Fantasy baseball is a different game than it was when we started. In the 80s we spent a lot of time trying to figure out what mattered, and how to measure it. We spent a lot of time collecting the stats to play the game.
In the 90s we discovered how to make projections that were as accurate as possible, and how to price a player's value in the game. But even then there was an asymmetry of information about injuries and team moves. If you lived in a particular city you had more information about the players on that team than those that lived out of town. In those days we went to the Out of Town Newsstand in Times Square and read the Boston, Philadelpha, LA, and other papers, looking for an edge. As Rotoman, my job was answering questions about these situation that were reported nowhere else.
In the aughts, we have information symmetry. Everybody knows everything that everyone else knows. We're playing a game of perfect (or as perfect as it gets) information. There are better players of course, who process info better, and bad players who process it less effectively, but that doesn't change the main issue, which is that there is no sure edge from projections (the difference between the best and worst projection systems is slim), from pricing (we worked that out a long time ago), and from the news (everybody has access to all new info).
This discussion is pitting the post-quant Liss, who feels he has the whole of baseball information in his soul and can apply it organically to his draft and team management, and the quants, who actually think they can derive meaningful numbers in a game where all information is shared.
I would still like to hear from the quants about their methods, or why they think they have information those of us who have studied this game for a long time have missed.
Oh, another thing about Derek's post.
Shandler's move away from stats and numbers, as Derek noted, is the opposite of a move away from stats. It is rather a commitment to the component stats and their ability to identify the players who are going to break out. I'm not convinced about Mayberry yet, but I'm sure that the rationale is rational and of interest. It is empirical in a surprising way.
We write down a number because if we're rounding numbers off and just coming up with a rough estimation, we're necessarily going to be losing some measure of accuracy.
How so? Let's say Pujols should hit anywhere from 34 to 48 homers 80 percent of the time. How is that less accurate than giving him 41 homers? I'd say it's more accurate because it's closer to reality which is uncertain.
How can the human brain process all 8 years worth of Nate Robertson's career, his stuff, the fact that Comerica decreased Ks by 5% and Florida increases them by 10%, that Comerica increased HRs by 1.5% and Florida decreases them by 2%, that the Marlins aggregate road park schedule will boost HRs by 4.5%, that the NL boosts K/9 by 0.57, that John Baker seems to boost K% and decrease BB% by a couple percent each, and a whole cacauphony of other information and context – and NOT lose something in the translation? I don't see how it's possible.
Of course, I'm not precise on all this stuff. And to the extent that you can create a model that makes better projections than my brain can (or the impressions that are implicit non-specific projections), I think you're a dangerous player. I conceded that from the beginning – if you're better at predicting than me, I'm worried. I doubt it – not you personally, but that these models which use past performance and project it into the future – are going to get the job done, but if you could aggregate growth and decline factors for individual players better than I can, then you can beat me. But that's got nothing to do with translating projections into stats. That's at the creating projections step.
The reason I think it's much harder to create a model that does that is you still have to tell it what's relevant and in what proportion. I have certain indicators I look at that seem to be reliable, but it's not robotically applied across the board to every player – it's situational what I'll look at. Depending on pedigree, arm strength, control, ground ball rate, velocity, health, stats, etc. There are different players that strike me as candidates for growth or regression for different reasons. I'm not taking into account as much data as you, but just trying to hone in on relevant stuff that I believe has worked and constantly updating my understanding of how those variables interact.
But again, that's got nothing to do with the question of translation from stats to dollar values. This is all in the realm of prediction. The reason I don't think the prediction model can be built is that otherwise it would be used to pick stocks, and one could print money. But most mutual funds don't even keep up with the index funds, and tons of hedge funds went out of business. Is there someone who can print money by picking stocks over the long haul with a computer model? I don't know. But it seems that's the level of complexity when you're talking about predicting surprising growth, not just beating the suckers who pay for wins and ERA and don't know K/9.
I don't think the chief task in predicting surprising growth or regression (where the money is really made) lies in crunching a bazillion bits of data – this is not the same task as winning at chess, for example. It's more about having a feel for what kinds of players are ripe for the price. I can't really describe it any better than that. I could tell you the factors I look at, or say why I targeted certain guys, but it's just a matter of having a good sense of it.
sorry that last comment could use a serious edit, but I can't edit from the admin area…
but:
This discussion is pitting the post-quant Liss, who feels he has the whole of baseball information in his soul and can apply it organically to his draft and team management,
I like that characterization of it. But I prefer to say it's not me auctioning fantasy baseball players, but the Lord auctioning through me… And the Lord does not go 16 on Johnny Damon, Eric.
One odd thing about when I put Erickson's projections through my model a few years back, it really seemed to beef up the middle tier players, lots of mid-level guys like Damon (current version), Vernon Wells, Alex Rios – they went for a lot. And the ARod's only went into the low-to-mid 30s, if I recall correctly. Everything was much more condensed. And it seems most projections drafters have whole teams of those mid-tier guys. Whether it's Bill or Derek. They never get a big star. I have just the opposite philosophy – I try to pay for stars who get you more production per slot and upside scrubs who have the most potential profit given their cost. But I have rarely seen a projection model that got anyone a $35 player. I guess Bill bought Ellsbury for a lot. Erickson and I were debating his second round pick of Ellsbury in a mixed league on our show, and I defended it in principle, but said I had a bad vibe about him for some reason and would have stayed away.
Bill, what kind of odds can I get that Hermida outperforms Ellsbury this year?
The most "accurate" projection systems regress all outliers to the means. So if you make bids off the projections you get a team full of mid-level guys. But the reason a team of mid-level guys might win is because of the outliers, the guys on the team who performed way above their projections.
I think it's important to point out that we don't have any real evidence if there is an advantage to a spread the wealth strategy vs. a stars and scrubs strategy. We know precious little about what actually wins leagues, if there actually is an optimal approach, which seems to me to be a ripe area for discussion.
I agree Peter – we don't know which approach wins leagues, only that in practice a model gets you all $25 players and cheaper for the most part.
That's another reason I scrapped the idea of drafted by this kind of model. One of the great things about an auction is that you can buy any player you want, unlike a draft where you're stuck with your draft slot. So if you love Justin Upton this year, your model will say you have to let him go at $27 or so unless you give him his 90 percentile projection. So you either project everyone else at 50 percent and Upton (and your other pet players at 90), or you realize that you're just fudging to get the results you want, cut out the middleman and just go the extra buck at auction.
I agree with Derek. I think he just missstated when he said the game was to get the most numbers. In terms of the auction it is to get the most potential numbers. If I start the season with more potential numbers then the next team, on the average, I win more times then the next team. I agree with Derek also that anything I can do with my mind I can add into my projection to make it more accurate. Once I have done that to every projection for every player that will be drafted the computer can do a better job then I can of creating a dollar value for each player from what my mind produced. No I do not know more then I did before I create the dollar values, but the computer can do the job more accurately then I can. And no, a projetion system does not result in buying middle teir players all the time. It will only do so if middle teir players are undervalued by other owners. I bought Lincecum in an NL only league this year using projections for example at $37. For projections the phrase "more accurate" is the key. None of the projections will be spot on, but the closer you get your projections to the actual results then other peoples projections, the more advantage you have in buying players. Somebody mentioned knowing the market. Your projections can not know the market in your league, but I do not see why that matters. Your projections reflect what you feel the market should be for every player. If your prices are more accurate then your league you will win. If your league overvalues a certain style of player your projections will tell you not to buy them, and to spend your money on a different spot your league undervalues.
Oh yeah, the other thing projections models do is overvalue starting pitching. That's because the SP's stats are worth way more than the SP who is extremely volatile in the categories we track and a bigger injury risk. Moreover, it's always easier to get pitching breakouts out of nowhere cheap, so it's easier to get by with what seemed like a shaky starting staff when you bought them.
As for adding all these factors, that's great, but you have to know how to weight them. As Peter pointed out, a lot of these park effects change over time. In theory, you want every possible factor quantified in its proper proportion for your model, but in practice, you're usually guessing how much weight to give each factor.
Finally, the market does matter. It won't work to buy ARod for $41 unless I can fill my OF with a $4 Scott Podsednik and a $6 Carlos Guillen, at this point (knock on wood) two guys getting full-time at-bats. If I'm stuck with players who aren't playing, the ARod purchase will hurt me. Remember, you don't acquire your whole team at once – so there's an information gap during the auction. You learn as the auction goes on what's available and at what price. The reason I'm able to be agnostic late and get full-time hitters is that I save my money late after spending it early. But the model is agnostic as to that.
Oh yeah, the other thing projections models do is overvalue starting pitching. That's because the SP's stats are worth way more than the SP who is extremely volatile in the categories we track and a bigger injury risk.
I don't understand why you think this is a valid way to argue. You don't know anything about any of the models people are using in this league, yet you make and endless number of assumptions about them. There is absolutely no reason why using projections and a model would lead to systematically overvaluing starting pitchers. Do you have any evidence to back this claim up?
And of course the market matters. You cannot do well without recalibrating your model during the auction based on what is happening. Is this just another thing you assumed the "modelers" were overlooking? You basically have assumed they overlook everything, even trivial points which anyone with any mathematical skills whatsoever would have noticed very early in the process.
Chris P. I think you're misreading what Chris L. is saying. Or why he's saying it. Those of us who make player projections know the problems. If you make pitching projections that incorporate all the injury risk, you get weak-ass projections for the best pitchers and the projection model clusters in the middle tier. If you make projections that look at all like the end distribution of stats, you end up with to top pitchers overpriced, because while the end of season stats will end up like your model, the names attached to many of them will not be the same. Thus, a projection system that actually tries to pick top pitchers will overvalue them. I don't think this is a matter of opinion, it is how it necessarily works.
Of course, if you incorporate the injury risk into the projection model (which is a legitimate option), your prices won't allow you to buy any of those top pitchers. Maybe that's a good thing, maybe that accurately reflects what expectations are, but it opens you up to exposure to other teams that choose to buy the top pitchers and pays for them. Some of these teams will fail, but some will succeed and these teams have equal access to the cheap pitchers who are most likely to have a big impact on your league's standings. I can't argue that it's wrong to buy all middlin' starters, but it certainly doesn't give you a noticeable edge.
Finally, Chris L. was arguing that the market does matter against a poster who said the market didn't matter. He was responding to a specific point and was hardly assuming anything.
Oh, one more thing. It is possible that you guys have a projection or pricing model that moves the ball forward, but the assumption that you don't is based on the same skepticism one has about perpetual motion machines. To allay suspicions, show something new.
Couldn't have said it better, Peter. Thanks for following up.
One thing you guys should realize is that I've done models for this stuff. I'm not claiming my model couldn't be improved upon – it was pretty simple – but it was credible. And so the problems I saw would be common to most models that started with 2600 dollars for 280 players. Being incrementally better at saying what a SB is worth (if in fact you can do that not knowing how the distribution of players would affect the sensitivity of the categories this season) would be unlikely to affect some of the basic issues that I came up against and that ultimately led me to scrap projections-based modeling for my intuitive approach.
As Peter said, it's possible you have come upon something that moves the ball forward that somehow avoids these issues, but like Peter I'm skeptical because of what I take to be the nature of fixed projections for 280 players that add up to 2600.
But if anyone wants to explain how their model avoids this, I'm sincerely curious to hear it.
When I did the basic model using Erickson's numbers (which in my opinion were as credible as any), I simply figured out what replacement value was for a given league in each category, subtracted it out, then looked at the standard deviation in each category for each player above or below replacement (and adjusted rate stats for ABs/IP). We then added up the standard deviation scores in each category to form a total. We then summed all 280 totals to get a number. We then divided 2600 by that number and got our constant. We then multiplied each player's total by that constant to get his dollar value.
It was a simple way to figure out how far above replacement a player was in each category and to compare the totals across categories and generate a dollar value.
It turned out SP were very high, hitters were clustered and closers were junk.
If I could have any player's stats in the AL from last year, it would be Greinke's. But I would not draft him ahead of half a dozen players at least for this year for the reasons Peter says.
I also think closers might be undervalued because so few players produce saves. And it's hard to reliably get them off the wire because the players producing them are so universally identifiable. And most closers-in-waiting are already owned. Someone will get them off the wire, but it's hard to ensure it will be you.
Maybe there's more to it than that, but the idea that only 14 players at any one time get a stat might be meaningful. Whereas anyone can get RBI or runs. I'm not sure how to account for oddities like that in the pricing model.
The thing that stops closers from regaining any of their value is that teams don't have to buy them. You can dump the category and still compete in most leagues, and have some chance of even earning points in saves if your $1 closers in waiting come in. That changes the value of saves considerably.
Wow, lots to address. Let's see…
I’d never argue that the market isn’t important. The market is critically important. It doesn’t matter if you have Colby Lewis valued at $15 and expect to get him for $1-5 if it winds up that the market values him at $30. The market will dictate what you do, especially for an agnostic player. It’s the combination of your strategy, player assessments, pricing, and the market that matter.
Where I disagree is here:
“…that doesn't change the main issue, which is that there is no sure edge from projections (the difference between the best and worst projection systems is slim), from pricing (we worked that out a long time ago), and from the news (everybody has access to all new info). “ – Peter
Agreed on projections (to some degree – while overall accuracy is similar, I do believe there is room for projections to differ enough to levy an advantage to one over another) and news, but I don’t think we can glance over pricing so quickly. I think that’s Bill’s whole argument, that pricing hasn’t been optimized, and I tend to agree. Fantasy experts often find themselves saying that it’s hard to acquire value in a top-notch expert league because the market is necessarily efficient among top competitors, but I don’t believe that this is completely true.
In this league and in Tout Wars, for example, I felt that the elite hitters went for more than they should have. Of course that’s just my opinion. I don’t claim to have a perfect valuation system for converting projections into prices, and I certainly don’t have the Wall Street/options-trading background and experience that Bill possesses. Yet, if you look over the CardRunners auction results, Bill seemed to be in the same boat as me. His most expensive purchase was Ellsbury for $31 and then didn’t buy another player above $17. I don’t know what exactly Bill and I were doing similarly, but our teams are structured very much the same way. I’d be interested to hear Bill jump in here. I guess my point here is thatif there's still some dissentment over what perfect pricing is, I don't think we can say that we've achieved perfect pricing.
“I would still like to hear from the quants about their methods, or why they think they have information those of us who have studied this game for a long time have missed. “ - Peter
I’m not saying that I have information others have missed. I do think I look at a few things that others don’t (but would be able to should they choose), but for the most part I think I differ from people like Chris mostly in that I don’t believe the human brain can aggregate so much information as efficiently as a computer can.
“Shandler's move away from stats and numbers, as Derek noted, is the opposite of a move away from stats. It is rather a commitment to the component stats and their ability to identify the players who are going to break out. I'm not convinced about Mayberry yet, but I'm sure that the rationale is rational and of interest. It is empirical in a surprising way.” - Peter
I’ll just glance over this quickly as it's really only tangential to this debate, going on the record saying that I disagree. I’m not a big fan of Mayberry. Seems too simplistic and falls in the same boat to me as Chris’s methods – you’re robbing yourself of accuracy.
“I have certain indicators I look at that seem to be reliable, but it's not robotically applied across the board to every player – it's situational what I'll look at.” - Liss
, I love Upton this year), why should we go more if that’s where the bidding ends up? If we selectively use our 90th percentile projections for a player because we “like” him, we’re just fooling ourselves. Whether we like the player or not, the fact remains that it's his 90th percentile projection, and we're selectively focusing on that 90th percentile for him only. Forcing things into “the results you want” will only end badly. Trust the process (whatever that process may be).
I wouldn’t argue that robotically applying a formula across the board is the best way to go. Every player is different, so treating them as such is surely the best way to go, assuming we do it properly. And again, “pedigree, arm strength, control, ground ball rate, velocity, health, stats, etc” are all things that can be quantified.
“And it seems most projections drafters have whole teams of those mid-tier guys. Whether it's Bill or Derek. They never get a big star.” – Chris Liss
“The most "accurate" projection systems regress all outliers to the means. So if you make bids off the projections you get a team full of mid-level guys.” – Peter
“I agree Peter – we don't know which approach wins leagues, only that in practice a model gets you all $25 players and cheaper for the most part. “ – Chris Liss
I don’t think this is true. In LABR, I paid $38 for Prince Fielder. Last year in LABR, I paid $40 for Jimmy Rollins. This mid-tier approach just so happened to be the case for me in CR and Tout because I felt the top tier guys were going for too much, in these specific instances. That won’t always be the case (again, it’s about the market). In LABR, I thought the top tier guys were priced perfectly fine. Pujols might have even gone too cheap. I think why I generally go for more middle-tier guys, though, is because you aren’t going to make a profit on Prince Fielder or Jimmy Rollins or top tier players of that ilk. You’re just hoping for even value there, which is perfectly acceptable, but I try not to load my roster with these kinds. Get a few to anchor your team, hope for even value, and then grab a bunch of undervalued guys in the middle tiers.
As to regressing to the mean, I don't think we can argue that this isn't something that must be done. It must. But which mean we regress to, I think, is very important. Regressing to league average isn't optimal for all players.
“So if you love Justin Upton this year, your model will say you have to let him go at $27 or so unless you give him his 90 percentile projection. So you either project everyone else at 50 percent and Upton (and your other pet players at 90), or you realize that you're just fudging to get the results you want, cut out the middleman and just go the extra buck at auction. “ – Liss
So why fudge? If after going through our process and ultimately deciding that we like Upton as a $27 player (and, I’ll say, despite all my complex models and my heavy mean-regressing tendencies
“If you make pitching projections that incorporate all the injury risk, you get weak-ass projections for the best pitchers and the projection model clusters in the middle tier. If you make projections that look at all like the end distribution of stats, you end up with to top pitchers overpriced, because while the end of season stats will end up like your model, the names attached to many of them will not be the same. Thus, a projection system that actually tries to pick top pitchers will overvalue them. I don't think this is a matter of opinion, it is how it necessarily works.” - Peter
I’m not going to get too detailed into my thoughts about pitcher projections and this whole middle-tier clustering thing, but let’s say, if for no other reason than for the sake of argument, that this is truly the case. So what? While we may want to be able to select the elite pitchers, if they’re probabilistically a poor play, let them go. Maybe you think you want a top pitcher, but really, the only thing we should want is to win – however that must come to pass. Sure, some team will buy those top pitchers and hit on them, but that’s the nature of playing against a field. This is akin to arguments I’ve heard against taking Pujols first in a draft. Because #1 picks fail to produce #1-pick worthy stats x% of the time, taking the consensus #1 pick is doomed to fail. Sure, but it’s doomed to fail less than if we took someone other than Pujols #1. It’s the same thing here.
“I can't argue that it's wrong to buy all middlin' starters, but it certainly doesn't give you a noticeable edge.”
Sure it will. Working under all the same assumptions, it’ll give you an edge over those teams that overpay for the top-tier pitchers and fall on their faces and will leave you in a neutral position compared to those who overpay for top-tier pitchers and succeed. So it does convey some edge in that you’ll be beating at least a few of the teams in this arena.
“The recitation of adjustment factors as a way to translate skills into some sort of neutral value is a subject of great interest, but I think as science it is seriously flawed.
Those numbers are all small samples, subject to considerable variance, and by the time they can apply to a single player that player has likely moved on to another phase of his career path.” – Peter
“As Peter pointed out, a lot of these park effects change over time. In theory, you want every possible factor quantified in its proper proportion for your model, but in practice, you're usually guessing how much weight to give each factor. “ – Chris Liss
I’m going off-topic here, I know, but I’m not sure I agree. Why would a park effect change over time? Unless the park itself has been changed, why would it? Weather and atmospheric effects may be slightly (randomly) different, but it’s the same park. We're just seeing variation. It may seem that way if we’re just looking at single year, unaltered park effects, but that’s far from optimal. We should look at several years and regress to the mean to account for the “small samples” and “considerable variance.”
I’m not sure what you mean, though, Peter, when you say that by the time they can apply to a player he’s moved on. Isn’t that the whole point of context adjustments, to account for this very thing? Robertson, for example, has moved onto Sun Life Stadium, so we apply that effect to him. He’s moved to the NL, so we apply that effect, and so on and so forth. And Chris, why is weighting the factor difficult? If after coming up with a proper park effect, for example, we see that Sun Life increases Ks by 10%, what’s there to weight? Just bump up Robertson’s Ks by 10%.
Again, sorry for straying from the usual topic here. I love context adjustments (and talking about them), yet there still seems to be some controversy with them.
I think I differ from people like Chris mostly in that I don’t believe the human brain can aggregate so much information as efficiently as a computer can.
Actually, said the opposite – that there's no way I can take all the information into account that a computer would. I just don't think this is a game like chess where processing power is the key to identifying value.
I wouldn’t argue that robotically applying a formula across the board is the best way to go. Every player is different, so treating them as such is surely the best way to go, assuming we do it properly. And again, “pedigree, arm strength, control, ground ball rate, velocity, health, stats, etc” are all things that can be quantified.
How do you quantify pedigree or experience level without applying it uniformly across the board. Say pedigree (where he was drafted for example, or where he was on BA's top prospect list at his peak) was quantified as some kind of prospect rating number – in some cases it matters more than others – with Jeremy Hellickson, you're double-counting because he's only a prospect due to his results. With Hochevar, he's the No. 1 pick, so it matters more – it's a separate factor apart from his so-so minor league results. I don't buy that all these factors are equally relevant for all these players. There's a choice you have to make for what to combine and emphasize with each player depending on who he is. I suppose you could create a formula that said if velocity less than x, then use these factors, if velocity greater than x, use these, but it's not very nuanced.
I love Upton this year), why should we go more if that’s where the bidding ends up? If we selectively use our 90th percentile projections for a player because we “like” him, we’re just fooling ourselves.
If the bidding ends up at $27, you'll still get Upton for $28 even if you project him for 100 HRs and 50 SBs. So that's not the concern. The concern is that if you really love Upton, you either project him for what any rationale person would see as his 90 percentile season (you think it's his new 50% baseline), but the rest of your players are in line with the market more or less, or you say screw it, I'll just go the extra buck – why bother to write down something that says I'll get him (and affects the rest of the players in the model?). The model is supposed to be scientific to an extent – so unless you're building in hunches for accelerated growth-rate baselines, you're stuck with very vanilla numbers that look like everyone else's. So you either have to push Upton's growth higher – on a hunch – or just accept that you're a hunch bidder, scrap the model and its pretend objectivity and go the extra buck on Upton. Because I agree – if you just bump his projection to suit your preferred result and pretend your a rigorous modeler, you're fooling yourself.
Why would a park effect change over time? Unless the park itself has been changed, why would it? Weather and atmospheric effects may be slightly (randomly) different, but it’s the same park. We're just seeing variation.
Wrigley played as baseball's best hitter's park and best pitcher's park in two different seasons the past decade. This is variance, but it's so wide as to be very perilous for use in a given year. Are Ks really universal to every pitcher in a given park? Are lefty power pitchers affected to the same extent of right-handed submariners? What about home runs – are all lefties or righties affected the same way? If someone's a dead pull hitter, is that different than if someone's a gap-to-gap hitter with growing power? A lot of this stuff lacks nuance and often better off either ignored, or just taken into account in extreme cases – Petco, Coors, etc. if a guy goes from Comerica to Florida (what it's called these days), I'm not going to go crazy with the park effects, though being in the NL is obviously significant.
The other factor that I think is underrated – if someone goes to Safeco with that defense behind him, do these factors not work synergistically in some cases? Nibblers trust their stuff more, throw strikes, get more Ks, less walks. But pitchers who already throw strikes benefit, too, but less. There's a lot nuance, and I think it's hard to build a model that does nuance well. Not impossible, but difficult. It's like building a program that make art – player evaluation is part science, part art in my opinion.
My main issue at this point is that Phipps is silent. He may have the greatest pricing model in the world, but a few days of chatter and asking nicely has provoked no response. Not even a, hey, I'm thinking about it how best to present it, I'll get back to you shortly. At this point I want something from the quants side to chew on, to react to. It may be as tasty as that hanger steak I had a dinner a couple of weeks ago, but right now I have no idea.
Derek, I'll try to get some of your points tomorrow.
Questions. If you were given the end of year stats and told to draft players for that season, would there not be a correct price to pay for each player? Of the available players in the league player pool, would there not be a set of 280 best players to draft? If one team picked a player that was not one of those top 280 players, would they not lose to the other teams that picked those top players by a small margin? Derek keeps using the word opinion when he's talking about how much a player is worth. Is it a matter of opinion because, even if given the end of the year numbers, nobody here can figure out the most mathmatically correct price to pay for each player? If we can't that's unfortunate because you'd think someone good at math could. Assuming you could then the problem of not knowing the stats is a stand alone problem, its a matter of who can make the best set of likely numbers for the player pool. In short, the projections that come the closest to matching the eventual end of season reality.
There is a post in another thread, I think, about this problem. The answer is that there isn't one set of values. Players faced with this problem would recognize that there is no advantage playing the same game as others, and game the system. Some would drop one category. Some would drop two. Depending on the skill of the players, their ability to manage their drafts (do they have hard numbers, or their memories of the players's stats, for instance), and how active the alternatives strategies were, the respective player pools would vary widely. I first did this in 1996-97, and it taught me that prices are mutable, dependent on a context that changes at every instant during the auction.
I'm not saying they're unimportant or unknowable, at least in a broad sense, but depending on a set of prices to dictate your decisions during the auction is a mistake. (I do think there's promise for some sort of dynamic pricing model, which we haven't really talked about. And I bet we do.)
If you were given the end of year stats and told to draft players for that season, would there not be a correct price to pay for each player?
You could narrow it down. But let's say we figured out roughly the most valuable 280 players. And we auctioned them. Let's say Bill's model was really good, and put him in first place. But then Eric and I made a trade, his SB guy for my power guy who were both deemed to have equal value. Suddenly, I pass Bill in SB and runs, Eric passes him in average, HR and RBI. Now Bill's in third place.
Let's imagine instead of making a trade, Eric and I noticed this happening as the auction was going on and originally bought the guys we needed to pass Bill.
But Bill's model would try to take this into account during the auction, and he'd have to get different players, too. So the optimal team is changing as it goes. Maybe Bill would have to pay $1 or $2 more to fix the standings in his favor than the player "earned" under his original dollar numbers.
So the players (even when we know the stats) don't have fixed dollars, right?
Its a matter of who can make the best set of likely numbers for the player pool.
The interesting thing is you projections could be the most off and still be the best for the purposes of winning the league. Let's say I think Justin Upton will go 50-50 this year, and I have him as a $60 player. My projection is most likely further off than anyone else's. It's a terrible projection. But I'm still going to get him for just $1 more than anyone else is willing to bid. So if he goes 40-25-.315, I've got a great value, even though my projection was the worst.
I also might project Zack Greinke to have an ERA of 6.00. If he has an ERA of 3.70, then my projection helped me because there was no way I'm going to own him.
These are extreme cases, but the point is it's very hard to say what the most accurate projections are. Are they the ones most similar to the actual numbers in the aggregate? Or are they the best ones for picking and avoiding the right players?
Some notes about Derek's questions and some of the ideas above:
Ballparks change constantly. Many ballparks play differently in the spring than the summer. And while it would be of interest to see month by month park effects, the imbalance of the schedule, the variance of the weather and the usual small monthly sample, mean that whatever results there are can't be fully trusted.
Ballparks also actually change subtly from year to year. Dimensions are altered, a wall is made higher or lower, a billboard is added or subtracted, a humidor is acquired. Some of these changes are well documented and commonly known, others are mostly invisible. It would probably be possible to track all these things, but even then you would have to adjust for the weather. An unusually warm April or a blustery September or an unusually cool July change everything. So does a schedule concentrated on pitching poor or pitching rich teams. Once you start slicing and dicing the data, the small sample variance starts to wreak havoc.
I think the annual park factors we generally use are good enough, and I think it's fine to do general adjustments from them for players who change context, I better trust my sense to give a bump or knock to a guy who is changing context significantly, than I do saying he's getting a 5 percent bump in power vs. lefties but a 6 percent knock in batting average, plus his line drive percentage was down last year, so he gets credit for regression of 8 percent there, while we adjust his HR rate down 4 percent because he was a little juiced in HR/F rate.
Maybe I'm being a willing ignoramus here, but I don't think we know enough about the interactions of all those numbers and what happens in the real world, to be comfortable doing all that micro adjusting. Or maybe I'm just lazy. When you add all those little factors in, you end up with basically the same numbers that everyone else has by saying, hm, he's going from a pitchers park to a hitters park, I'm going to bump him up a couple bucks.
And even then, the damnedest thing is that there is no way to prove it whether I'm right or you're right, since the normal variance of players makes it impossible to determine that one system of projections works significantly better than others. (And isn't that what all these normalizing calculations are meant to do, give you an unbiased scientific baseline of a player's actual ability?)
I just deleted a long bit I wrote about paying, not "overpaying," for the top tier pitchers, whose prices are generally cheap compared to the mid-level pitchers and their collective capabilities. That can wait for another day, once when we learn something more about the possibilies of improving player pricing. But my point was that in most leagues the top tier pitchers are cheap compared to potential earnings (not adjusted for risk), and have a similar rate of return on investment as middle tier pitchers (once they are adjusted), but they use up fewer slots. Increased risk, meet increased opportunity. As I said, I think you can overpay, but in most leagues you don't have to because others won't.
One more thing: Derek wrote: I’m not sure what you mean, though, Peter, when you say that by the time they can apply to a player he’s moved on.
What I meant is that a player's skills change throughout his career. Youngsters are spry, fast, impatient. The talented ones are glorious in their prime, retaining their athletic gifts while utilizing great wisdom acquired from years of practice. And those that survive into the downside of their careers, who don't fall suddenly off the map, do incredible things to marshall the resources that remain and kick their failures down the stairs. Each of these phases is roughly four or five years long, and by the time you've started to home in on one the player has moved on and become someone related but different.
If all followed that arc it would be easy, but a great many don't. And when they fail far enough they don't decline gracefully, but are abruptly shown the door. Or not, depending on the team they play with.
Age is one of the most important factors for player projection, so there is information in the numbers about the shape of a player's career, but we're again dealing with small sample sizes. A long time ago I tried to match players by similarity score to other players with similar career shapes. I've written about this lots over the years, it's a fun trick for looking at future potentialities. The problem is that for any given player at a given age, there aren't that many players with similar histories, and when you look at their future outcomes they vary wildly. The best predictor for future performance, it turns out, is their median output, which is pretty much a weighted average of the individual's past performance.
The point is, you can do all the work to individualize the projection to all sorts of factors, and the results will be about the same as a three year weighted average (and really, with a constant, a two year average is nearly identical to the three year version). That tells me that on average, at least, there isn't that much actual information in the adjustments.
I'm sure I missed some, but this is way too long. Am I right, that these questions and our various responses to them go to the heart of the game and its ongoing interest to us?
Please forgive me, a stranger stepping into a debate as developed as this. Here is my take, as an outsider:
1) Projections:
Everyone uses a model. Liss's "intuition" is a model, albeit undefined. There is so much variance in fantasy baseball that the advantages of a defined model are not necessarily visible inside of one fantasy baseball season. While Liss's supreme knowledge of the game, feel for breakout players and playing time situations, etc., all give him an advantage over the so-called "quants," Liss would, in the long run, be an even stronger player with a tested and defined model that incorporated his knowledge into projection making. As Derek says and the traders agree, accuracy is advantage, more discernible in the long run. When you are trading thousands of shares per second, or whatever you crazy finance whacks do, the 0.5% (or 0.00000000005%) predictive advantage your model provides is all the more evident (thank you Thesaurus).
2) Valuing Projections
Art Mcgee has done good work on the valuation of projections, and I assume our "quant" friends have as well, likely going into even greater depth. Valuing projections is not about accuracy. It's about doing some math. Given league settings and final standings, we should be able to know exactly how much Andrus's 2009 season was worth. If Manager X has the player valued correctly in the context of the league, and Manager Y is off by $5, X gains an advantage. Now, Liss's point seems to be that this hypothetical $5 advantage is negligible because of the inherent fallibility of projection systems. While projection systems are fallible (25%-30%, say), if the valuation of my fallible system is also flawed, I become even less accurate and drop down into the 30-35% fallibility range. While there is a chance one's flawed valuation of one's projection turns out to be closer to the actual outcome, there is just as great a chance that it will be even further away, thus making the proper valuation of projections no less important.
3) Conclusion
Liss's intuition is a model. When he goes to $27 for Jeter, he is making calculations, based on the environment of the auction, position scarcity, his own budget, his own valuation of Jeter, etc. In the long run, when the "quants" define models (and test them) for all of the calculations Liss is making (not just in the draft, but throughout the season), they will win. We will have to look at a hundred thousand fantasy baseball seasons and the average place finish for each camp, not one or three. There are so many variables in the equation of a fantasy baseball season, however, that experience still wears down a "quant" over a full season, despite the inaccuracy in these calculations of experience. "Quants" might not have enough models yet, but theoretically, given equal knowledge, defined and tested mathematical systems will give "quants" the advantage over time.
I agree with the small sample size above and that the advantage of a model is small over an experienced player who basically can model in their head. But I agree that it is there over many seasons.
Similarly, "That tells me that on average, at least, there isn't that much actual information in the adjustments." Many small advantages add up to wins.
I think an important point to say is that your model only gives you prices, but there are other factors that make a winning team. If I used my model to draft 8 speed deamons because they were undervalued I'd lose, because my team structure would limit my power category totals.
I also happen to believe that the replacement level of the league should be adjusted above part of the player pool, essentially making a large group of $1 players at teh bottom. I set the replacement level to the average waiver wire player for the league, because all players drafted who are below that level can be replaced on the average. I am curios if anyone agrees with this strategy. I think its a hole in current models that many people use and perhaps an intuitive player is making that adjustment in their head and gaining advantage over a model. There could be other factors models are doing poorly as well to make the intuitive approach better. But as the above poster said, if you did you model right, you'd gain advantage.
I would like to see some evidence that many small adjustments add up to wins. Over the past few years we've seen a tremendous increase in the amount of pitch, situation, park, game, hit, field, weather, younameit, data in baseball, but we haven't seen an increase in the accuracy of projections. At least not in any way that says, I am an improved version! I'm not saying that the information won't contribute in a significant way someday, but that will be when we actually know how to read it. Right now a lot of the data are experimental. That's why MLB is giving PitchF/X away. They want us to kick the tires for them and find out what is working and what isn't. It's exciting and will change the future of both baseball and the fantasy game, but I don't think it's all that meaningful (or maybe I should say reliable) yet.
I would like to point out again, maybe more forcefully than I have before, that I have a model that I created in 1987 and have improved constantly since. I make projections, which scored well in Tom Tango's Forecaster's Challenge last year, and draft off a price list. I am not an intuitionist, at core, and I can see easily that a good model will help you win the game.
But given all that, I don't think a good model can be built off of player projections, unless it involves improving the projections. I don't think projections contain the pertinent information you need to price players reliably. This is a disagreement, but it isn't one about intuition versus data, but rather one about whether the data itself can really be considered that. I'm perfectly willing to change my mind if someone can explain to me how they might.
But so far the argument has been: If everybody is using projections as the input into their model, and my model is better, I have an edge, right? Right. But then that leads to you to say, so if Chris Liss used my model that created prices from projections and combined that with his awesome draft sense, he'd have an advantage, right? And I think he correctly answers, Maybe.
There is lots of information in the baseball data we have, but an awful lot is missing. If you're using projections as your inputs and they aren't the best input, won't your model be handicapped?
I'm not clear on your question about the number of $1 players at the bottom. I think that the players who are drafted at the bottom who aren't catchers are usually there because they aren't very good, but might get playing time, or because they'd be okay if they had a role, but they don't have a role. I think it's fine to create a cluster of these $1 types, and then decide what to pay for them in the endgame, when you know how much money you have left.
Depending on your strategy and ability to pull it off, you might have no money and no slots, so they're irrelevant, or you may have a little too much money, and can bump the best $1 a buck or two, depending upon your strategic situation (needing reliable AB means one thing, seeking risky potential talent another).
But replacement value is always the quality of hitter or pitcher available to replace a player who goes down. There are reasons to play around with this some, especially in shallower leagues (and cardrunners is a little shallow), but the principal is that the replacement player is the one who replaces the one who is lost.
Interesting series of threads. A lot more discourse than any 'expert' league I've been part of.
The question I’d like to ask is this: In a 12-team single league format, what percentage of success is attributable to projections vs. converting projections to $ vs. draft strategy/execution vs. in-season pickups + trading vs. luck?
I consider myself more on the quant side than most in fantasy baseball and invest a lot of time tuning projections and developing formulas for converting these projections to $. But at the end of the day, I think the two amount to about 1/3 of potential success.
I've found draft strategy/execution might account for another 15%. Having a 'feel' for the draft room (will they overspend early, are they gunshy and then will speed up as inventory gets scarce) is an important variable. One of the reasons that I dislike the 'stars and cheap upside' draft strategy is that you don't have the ability to adapt once you've made the big purchases which invariably are early in the draft b/c top players tend to be nominated early. I've been in several deep mixed-league auction drafts this year where multiple teams banked there would be solid $1-$2 players late in the draft only to watch in anger as the most attractive of these players would go for, say, $3 by teams like mine just because I can afford to with the savings I accrued throughout the draft.
I'd consider in-season moves to be 5-10% at most – more in NL where there is some prospect/rookie depth in free agency (in AL-only, Mike McCoy, a 27-year old rookie utility player on the Jays was the best prospect available after the draft).
The remaining 40% or so feels like luck. I've got an AL-only team with an OF of Granderson, Snider, L. Scott, David Murphy, and Eric Byrnes with Desmond Jennings on the bench. If Granderson gets hurt, my team is screwed. If everyone stays healthy BUT Upton's shoulder pops out and Jennings gets 20 cheap SBs, my team is great. I wonder how many leagues and/or years of play it would take to separate the 'skill' from the 'luck' aspect when it came to evaluating fantasy baseball players.
Rudy
http://www.razzball.com
Hi Rudy,
I like your site.
My personal feeling is that the draft is a bit more important than that. I also think in season moves are more important than that, but perhaps only true if you had a solid auction. I generally feel that if you have a solid auction, you can find ways to deal with bad luck easier. If you whiffed at auction, you're often left just trying to find lightning in a bottle in claims.
I feel like luck is less a factor in your team doing very well to the field. If you are really talking about finishing 1st only, I think its a lot like a poker tournament. You generally need to play well and get lucky both.
Hey Eric -
Thanks. I agree that to finish 1st, you need to play well and get lucky. And I agree that a good draft (and I'd add a BALANCED draft in terms of roster and risk) will allow one to deal with bad luck easier.
Rudy
It's interesting you say that. As I am obviously new to expert leagues, let me ask you if my impression is correct. I get the sense that many of these leagues are not very active in trades, possibly because so many of the owners are in SO many different leagues. I think if that is true, and its hard to get trade talks going constantly, I would agree with you here.
I've found that trading is difficult in single-league formats because the free agent/waiver wire is barren. I had an offer in one league where I could trade Luke Scott and David Murphy for Chone Figgins. You'd think I'd jump on it but I would've been in a position where I was punting 2 OF slots. Combine the free agent scarcity with the fact that drafters inherently value their picks more than their competitors so trades from both sides may seem lopsided. I suppose the multiple league aspect doesn't help but probably is a secondary driver.
In my deeper mixed-leagues (14-16 teams), there have been a decent amount of trades. But generally it wasn't with another self-proclaimed 'expert'.
So the percentages I laid out were for a single-league format where in-season moves are limited because of deep rosters + tough trading environment. It adds more importance to the draft but also increases the role of 'luck'.
Hi Rudy.
I like the conversation you and eric have going here, but it is kind of buried in this thread. I think Eric should move it to a thread of its own.
For now, I'll take a very quick stab at your original question.
I don't think it can be broken down into percentages because of both league variation (I've been in leagues with very active trading) and because of synergies. As much as I have emphasized the importance of a good model, the value of it goes up exponentially with better projections and feel for the players. I actually agree with Chris Liss that you can play with just player knowledge and lean on the room a bit for pricing aspects. I think you are unnecessarily surrendering edge and passing up on an aspect of the game I find fascinating, but you are much better off than the guy with the world's greatest model and zero clue about the player pool. If we were instead to just break it down generally to 'knowing the players well' and 'having a good model and knowing how to use it' as long as you are at least reasonable on the first part it is probably a 60/40 or so break. The 'luck' component is just how volatile this game is and that is going to depend on how much edge you can extract from the rest.