Word Lovers and Baseball Fans
April 12th, 2010 by Robert Dixon in General Guidance, TheoreticalHi, my name is Robert Dixon. I am a co-owner in the league. I've been a serious games player and bit of a math geek pretty much all my life. I really enjoy fantasy baseball and am excited to be part of this league.
Back in the early 1990s when I was in college I took up tournament Scrabble. I'd already been in the tournament chess scene for a few years and assumed that Scrabble, like chess, would be filled with a bunch of games loving people looking for an outlet for intellectual competition. I was very wrong.
Scrabble clubs and even tournaments were primarily filled with incredibly smart interesting people who mostly shared a deep love of words. I really liked those people and enjoyed their company, but after just a few short months and with a vocabulary significantly smaller than almost everyone else's I was destroying almost everyone at the Seattle Scrabble Club. Why?
Where they loved words, I loved games. I didn't waste time on definitions or oddball words with almost no utility in the game. I memorized lists of important words to the game (2s, 3s, high frequency 7s and 8s and such) but I also tried to figure out what made a play correct. Even the good players out of the word lovers thought defense was worth way more than it was. They also made serious mistakes valuing the letters left on their rack after a play. There were enough gamers like me in the Scrabble community and computer programs were becoming available to tell people what the real value of plays were that by the time I gave the game up two years later play had already started to improve noticeably.
I think the current state of fantasy baseball is very similar to what Scrabble was back then. The vast majority of participants are hardcore baseball fans and not much of gamers if at all. A lot of these guys know major league and minor league rosters with more detail than I know my extended family. This is not a bad thing. Fantasy baseball is a great hobby for hardcore baseball fan who can simultaneously satisfy a desire to be both a scout and a GM, jobs that there are literally thousands of times more people who want them than can ever have them.
Now, I am part of a different group of people who play fantasy baseball. I am a baseball fan, but not a huge one. What I am is a gamer. Now in my late 30s my early trajectory of playing chess and Scrabble tournaments has just continued and fantasy baseball is among the many games I have enjoyed immensely over the years. And while there are certainly aspects of the game that have some resemblance to being a scout or a GM, they are crude imitations of these roles set up within the framework of a game.
I think there are some aspects of the game where there are widespread mistakes that get reinforced by a level of groupthink similarly to how the word lovers almost all thought defense was an important part of Scrabble.
For starters, saves are incredibly overvalued. Not just closers, but saves themselves. Also, I don't think anybody can eyeball a stat line and tell you within 3% what that is worth. As someone who is facile with numbers and has been using a good model for five years I think my eyeball estimates are close to 10% off and for most people, even those who play a lot of fantasy baseball, that error is probably more like 15 to 25%. If I am right about that, which I will happily defend, we have reached the point where someone operating without that 15+% error has clear edge over someone with slightly better projections.
But what answer can I give to someone who says that projecting players and using a model is not the right approach at all anyway? Well, most likely I have no answer that will satisfy that person. As much as I enjoy the company of word freaks and baseball fans there are areas we are far enough apart in that we are unlikely to ever convince each other. I will attempt to answer that question, but in reality, I am probably only going to be able to write for gamers who enjoy fantasy baseball. But, for what it is worth, here is my attempt.
Taking this auction as our example, there is a set pool of players we are dealing with. There are ten teams, each with 260 auction dollars to use. The value of the entire pool of players must on some (possibly unsolvable) theoretical level sum to 2600. I say it is impossible to speak intelligently about the value of players without first taking that entire pool and using some method to divide the 2600 dollars among the players. I just don't see how it is possible. I understand that having auctioned enough times people can develop an intuition for it, but there is no way that intuition can be the ideal approach. Use all the experience and intuition you want when determining values, but any truly correct strategy has to start with valuing the entire pool and making it add up.
This is where it starts. For those who can agree with these points there is a conversation to be had. For others, I am still happy to play some fantasy baseball with you, go out for drinks when the season is over, and do it all again next year!
Loading ...
Question for you Robert – would you rather have a $10 player or your choice among 10 $1 players, whichever ends up doing best. Let's assume the first $1 player is the one you buy. The nine others (his rough equals) go unpurchased.
Do you see how this screws up your model? You think there are $2600 to spend on a given set of stats. And the only task is to apportion that money to get the most bang for your buck. So if I buy ARod for $41 and then get stuck with a $1 slot, and you buy two $21 players, your model tells you you've got the better bargain. But that's because your model dupes you into thinking the meager projections for my $1 player is all I'm going to get out of that slot. When in fact, I'm going to make moves all year to get more than that. I get ARod plus x$, and x is almost certainly greater than $1.
In other words, replacement value is higher than you think. And when your model which attempts to value stats across categories by using value over replacement and standard deviation – as all models must – it's going to have the wrong inputs. It's going to think replacement value is the $1 player, but really it's what we do with the roster spot over the course of the year.
The higher the replacement value, the more valuable the star players. This is something I know from the math but also from playing in a variety of leagues. So your model will undervalue the stars if you attempt to build projections for every purchased player and divvy up their stats into dollars.
That's but one example, but it shows that you can't just take the pool of drafted players, give them stats, and expect those stats to convert to $2600 accurately.
I don't know enough about scrabble to comment, but I'm pretty sure this is more like the stock market. If you tell me your models can tell me which stocks to pick and crush the market, then I think you can crush fantasy baseball. But this stuff you're saying $2600 to apportion based on specific projections across the pool? i looked into that years ago and concluded the game was more complex than that.
Sorry, that last line sounds pompous – tried to edit/delete it, but couldn't figure out how.
Your assumption that we either didn't see the issue with relpacement value or find a way to address it is very telling. I think you divide the world into things that can be clearly calculated with an absolute answer and things where we must abandon math and trust to nothing more than intuition, creativity, and experience. With this view I can see where you'd be dismissive of models.
I don't think we have completely solved it. Actually, we are likely still further off than the 3% that you claim to be able to eyeball your way to. I just think that a robust approach involves best attempts to answer these questions instead of either claiming to already know the answer or calling them unanswerable.
Like I said in my original post, I don't think I can convince anyone who has a very different view of things that this is the right approach. What I think I can do is write interesting articles for gamers who enjoy fantasy baseball. If you choose to read them (which I hope you do) you might find yourself gradually swayed over a little, but I think that is the most I could hope for.
Chris, you do realize these guys are among a select group of highly paid top traders at a bazillion dollar firm with a veritable arsenal of mathematical tools at their disposal? Their job is in fact to crush the stock market, and they do it quantitatively, with math and models.
To paraphrase what I just wrote in our discussion in your rotosynthesis column :
I can certainly understand some skepticism, but I think you're being rather close minded with your insistence that since you looked into it years ago and ran into problems, that it essentially can't be done. These are a pretty extraordinary pair of guys, with training in just this kind of area, who have spent a great deal of time on the issues. I don't think you have anywhere near enough information about how their model works, or how they approached these problems, to say something as bold as they don't have replacement value right (which pretty much implies everything else is wrong too.)
I don't doubt that these are smart guys, Eric. But if someone tells me that they're starting with the premise that you can project $2600 worth of players and distribute it across the pool, I already know the model is going to have some problems. Because you spend $2600, but it only reflects the pool of players purchased and doesn't not accurately account for the price of each slot. That's going to skew replacement value.
Can this be incorporated – sure – to an extent. But honestly, Robert and Bill – have you incorporated it yet before I mentioned it? I'm all for models – I built one, and I don't doubt you guys can improve greatly on it. And I wanted to delete the last line as I realized it implied that somehow my trying it was relevant to whether a model can work. (It's why I don't do it anymore, because I realized that it was too tied to assumptions based on last year's stats, and not reflective of a lot of factors).
Moreover, not only am I pretty good an eyeballing a stat line and picking out a dollar value in a given format, (though as Peter wrote it varies depending on circumstances more than one would think), but also I don't usually need to as there are people with models in our auctions doing that all the time – we know the general projections and pricing for all the players. And in my example about the GM who knew roughly what the prospect had earned last year, and the one who knew exactly, I don't think the latter had a big advantage, either.
I appreciate that Robert, you admit your more than 3 percent off, even we were drafting for last year's completely known stats. If you consider that, and add that the stats for 2010 are not known with any precision, then you can see why I'm dubious your model has a major advantage.
And I'm curious about the extent to which their models "crush" the market. Is it because they prey on the suckers that don't trade well? Or is it because you can predict the performance of specific stocks? If it's the latter – I'd be more worried. Can you beat the market? Or is the goal simply to profit from the mistakes others make in trying to beat it?
I don't know how they approached the problem. But I'd certainly like to hear about it. You really seem very determined to stick to your 'it can't be done' argument before you've heard any real details whatsoever.
I'm not sure why you seem to think smart guys spending time on the problem wouldn’t be able to come up with the realization that further talent comes into the pool later, or that these slots' value change over the course of the season before you suggested it.
I think its a bit muddled to suggest that because of these factors too, nothing very meaningful can be said about the initial task of how to divvy up the $2600 among the initial pool.
Like I've said in other posts, the questions really are whether mathematically meaningful things can be said here, and how much edge is to be gained from them? Bill and Robert are saying that, while they haven't solved the problem (and it might not be fully solvable) they've made serious inroads and this is providing a significant amount of edge. I’m surprised at your apparent confidence that they’re wrong.
I'm also surprised by your suggested reliance on other peoples' models. You really seem to be generally arguing that all models are flawed, so why rely on those ones? Furthermore, it seems to be part of Bill/Robert's argument that these other models are flawed.
Personally, I think at this point it makes the most sense to wait to hear more details than to just insist their methods are flawed in advance.
That’s a knowing answer to a difficult questoin
I'm not relying on other people's models, Eric, but we're talking about an auction where the models are influencing prices. Shandler once wrote that an expert league is easy in some ways because no matter what, a novice can always just pay $1 more than an expert would. That the sense in which the market is always going to limit the errors to the point where I think your improvements in valuation will be on the margins. I'll still buy according to my gut, but you can see how the models price various stat-lines and what the ballpark or market is.
And I didn't say nothing meaningful can be said about divvying up $2600 among the purchased players. I said that there's more to it than that, and I've already seen models that do it pretty well insofar as one wants to do it. So between what we already have – the ground that's already been trod over – and the issue with slots/versus dollars, in-draft inflation and variable value of players based on budget and situation, I think the improvements would be on the margins.
Is it possible that your improvements could be substantial – I suppose, it's possible. But again, unless you purport to generate BETTER PROJECTIONS, not just better conversion into dollar values, I don't think you guys have much of an edge. That's why I'd be much more concerned if you guys could actually beat the market, not just game it.
Chris's quote from Shandler reminds me of the Popeye League, which a bunch of us started some years ago in the Times Square Popeye Fried Chicken store. We started a league where each bid was made blindly by writing it on a piece of paper, then simultaneously holding it up to your forehead. High bid wins.
I have a pricing model, my buddies did not. We played an oddball format, so there we no published prices. The first year there was outrageous variance in the prices and I won easily by sticking to my prices.
The second year there were no more ridiculously out of whack prices and I didn't win.
The third year there wasn't any fun, as the bidding was competitive on every player, no one wasted money and everyone agreed it was a good experiment that had outlived its usefulness.
Personally, I would love to hear more about the model Bill and Robert are talking about. It has been my impression by the way they've couched the discussion that they are rehashing issues that some/many of us tacked many years ago, which is why I think the discussion has careered the way it has. But when I think about it I realize I don't know any details, so maybe I've got it all wrong.
But Eric's right that it's unfair for me to presume they didn't take into account the value of the slot and not just the numbers projected for the cheapest rostered players. So sorry about that. Would be curious to know if the presumption was correct though.
And I think the scrabble analogy presumes we're not serious gamers who have long modeled/studied the game theory aspects of the this game, but are just baseball enthusiasts who know a whole bunch about players. I at least have reasons for thinking your model isn't going to be much better than the ones from the industry guys (it's not like Peter or Shandler are idiots). Why would you presume we didn't have an understand of the game itself? Are not all of these posts essentially demonstrating that? Does one have to model the game into projected stats in the way that you say and draft according to them to be a "game theorist" about it? Why is projecting specific stats the best way to decide what a player is worth?
Finally, does you model take position scarcity into account? Or are all stats created equal? In other words, you must have a SS, and you cannot have more than three SS in your lineup at any one time. Same with 3B. Does that factor into your pricing model? Is Joe Mauer's batting average compared to everyone else's (adjusted for at-bats, of course), or is it compared to what others are getting average-wise from their catcher slots?
I know their model does account for scarcity and position. I don't believe it works quite like you suggest, comparing Mauer's average to other catchers. You care about the baseline production you get by position, but not the type. There's no reason you have to get your batting average from catcher, and your power from your 1b, etc etc.
I do understand how people can be miffed at some of the presumptions on both sides of the discussion, but I think the real difference is that expert leagues (and much of their methods) have been publicly available to study for quite some time. Robert and Bill have studied what the experts have done, and are on sound footing offering critiques, however anyone criticizing their methods really can't know any details of what they're talking about.
Let's get some more details before we start poking holes.
I for one would like to hear how they address your projections issues. I think you're overstating the situation (in both uselessness of projections and lack of importance of accurate pricing) somewhat in your argument, but I definitely think this is a serious argument and I would like to hear a response.
You don't have to get your batting average from catcher, but you have to roster two catchers, and that has an affect on your batting average since the typical catcher hits for a significantly lower average than your typical first baseman. So if you have Mauer, he's worth more for average than a .330 first baseman with the same number of at-bats.
Chris, I didn't presume you were or weren't anything. I said if you see a complex mathematical game like fantasy baseball and (even after attempts to approach it differently) decide that the expert way to approach it is to trust your gut because anything else is just too complex then you are not a serious gamer. I stand by that.
I am truly offended that you would ask (twice even!) if my model addressed the baseline question before you'd brought it up. Yes, it did. Besides, if I hadn't thought of it before then there is no way I could have incorporated something that complex into it in the last few hours, so answering your question a different way, no, I didn't lie about that.
I don't take into account where production comes from, just net production. I do take into account baseline production by position. I don't see how you can possibly make a case that it does matter, but you are welcome to try.
Robert, you never answered the question, so I asked it again. Not sure why you're offended by that or why you felt I was accusing you of lying! It seemed the simple $2600 for 280 players given their projections would have a hard time with what players 260-280 would earn when they're so fungible. I'd be interested to hear how you do that.
But clearly you think that if someone doesn't give every player specific projections (which are of course fictional) and create dollar values from them, then they can't possibly be a serious gamer. I suppose Shandler's Mayberry method makes him unserious as well. (And he used to do it very specifically).
The bottom line is that at some level you have to make an assumption whether it's as to the player's projection, or as to his dollar value. Where that assumption is made isn't that important – it's still a gut call. (Unless you build software that does the projections for you, too – which is much harder). That's why I asked whether you can beat the market by picking stocks – if you can do that, I'm impressed.
And your understanding of using your gut is really oversimplified. A person with an excellent grasp of the underlying facts can make judgment calls on "gut" that are extremely accurate. There have been studies which confirm as much. Could a model ultimately be more accurate? It's possible, but I'm dubious unless you can create one that projects and converts.
You guys are framing the question the wrong way in my opinion. You see 22 HR, 89 RBI, 80 Runs, 23 SB and .301 average in 566 at-bats and think – try to name the value in a league with our parameters. (Off the top of my head – $28. But let's say according to your model I'm $5 off. You think – aha – we have you beat.
But maybe I'm not good at translating those stats into dollars because that's an artificial step. My brain goes from facts (not fictional projections to dollars). I hear Derek Jeter at 26, and I know to go $27. Not because I have his specific stat line in my head. I just know most of the relevant facts about Jeter and translate them into action. I don't need to make the step that you guys think is so important.
If you read the post on RotoSynthesis,you can see what I'm saying with the language analogy.
The thing is I grasp what you're trying to do. You cannot see past your method and grasp what I do. You dismiss it as "gut" when it's actually an algorithm in my head. How does Tom Brady know how much velocity and at what direction to throw the ball? It's his gut, based on experience. Can someone build a program as responsive to all the factors that would deliver the ball more accurately? Not easily. This might not be as complex as real NFL football, but you get the point. The auction happens in real time and it involves projecting future performance of human beings who are often unpredictable.
I think we're starting to go in circles.
While there are a few small questions outstanding, the debate has really presented two main ones to my mind. This is personally what I'd love to see answered next:
1) How good is the intuitive player's internal algorithm really? Chris certainly seems to think he's zoned in, and maybe he is. However, this claim is easily made. Like I said on Chris' rotosynthesis page, how do we move beyond a bunch of guys all claiming to be intuitively fluent, or to have perfect internal translators? It seems we need to start testing this claim as the next logical step. How should we proceed with that?
2) Chris's claim that player projections' volatility dwarfs the value of any hyper accurate pricing needs to be addressed. In other words success is largely determined by being right about reasonable guesses like whether or not Arod hits 28, 35, 41, or 48 homers and less about how accurately you price the value of 28, 35,41 or 48 homers. Naturally i'm simplifying a bit, and he clearly thinks he can accurately price things as well, but I think this is the heart of the objection.
Pretty good framing, Eric, but there's actually one more issue: that we're all going by gut, just at a different step. Even if you build a perfect stats to dollars model, where do you get your stats? To give Jeter 18 rather than 20 homers is an assumption – a gut call. And does it really matter whether you're using a gut call on that step and then a model for dollars, instead of a gut call directly to action at auction? Ok, Bill says he's not using gut, but using CHONE or Shandler or whatever. But how do they do their projections? Either by gut call or algorithm. If it's the former, then you're relying on someone else's gut. If it's the latter, their algorithm So your model – no matter how perfect on your end – is only as good as the inputs – some other guy's algorithm. Maybe it's great, maybe it's not. Maybe it's calibrated to score great in overall projections, but not for assembling fantasy teams. And I'd be surprised if it can adjust to all the facts like projected playing time changes in spring training, etc. It's likely you're flying blind there, too.
Again – find me an algorithm that can accurately pick stocks, predict the weather, etc. and then I'll fear your model. Otherwise, you're taking a leap along with the rest of us no matter whose model your use.
I think that's all in number 2, but I agree I’d love to see it discussed in depth.
not really as you guys assume your model is somehow scientific when the inputs are clearly not. either you place blind faith in someone elsels projections, or make a gut call by creating your own. do you not concede that if your projections are poor, then proper valuation of them is pointless? why are your projections any less suspect than my decisions at auction?
I really don't see how that isn't clearly covered by what I wrote for #2. " In other words success is largely determined by being right about reasonable guesses like whether or not Arod hits 28, 35, 41, or 48 homers and less about how accurately you price the value of 28, 35,41 or 48 homers." ?
It's not covered because you're seeing 2 steps and a balance between being better at Step 1 to the extent it matters and Step 2 to the extent it matters. But my point is we all have to make 1 great leap of faith or intuition or gut, and there is only one step. My leap is that I can translate what I know about the players (the facts I've gathered into action at auction). Your leap (even assuming a perfect conversion model) is to generate projections from facts. Facts to numbers, facts to actions. Are not both faith/gut/intuition? There is no science from facts to numbers any more than there is from facts to dollars. There is no Step 2 for me. And assuming a perfect model, no step 2 for you, either. Once you have your numbers, your decisions are made. Once I have my facts plus reading the table, I'm making a decision. The only difference is that you know what you're going to do in advance of the auction, and I don't. And what's what gives you the illusion of it being scientific. But your inputs are not based on science, and your outputs only have value to the extent your inputs do.
I don't know Chris, everytime I read what I wrote it seems to basically say the same thing. The question is really how we arrive at our projections, how good they are, and whether the issue dwarfs the other parts of the process. I think we're going in circles again.
There's only one common part of the process, and that's going from facts to assumptions. Mine happen on the fly during the auction in real time. Yours happen before the auction and take the form of numbers which you then apply a bunch of math and science to in order to predetermine what you'll do at auction. Your math and science doesn't give you an advantage, it just makes you feel better about your decisions because it resolves them ahead of time. But there is no science in how you got your projections, any more than there is in how I got my "feel" for what a player's worth on the fly. I don't know how I can explain this any clearer. The only model that rests on a firm foundation would be one that could PROJECT AND translate to dollars. Then your inputs would be scientific, too. All your TRANSLATION model does – to the extent it's tight – is ensure that whatever your crazy projections are, that's what you're going to live and die by. You will get a team that certainly reflects those crazy projections, right or wrong. I'll get a team that reflects my crazy intuitions about player value on the fly. We're both taking a leap – you that the projections to which your model so aptly ties you are good, and me that my intuition is sound.
Unless you can build something that uses science to project stats, too – there is no advantage to being tied to a particular set of projections.
"Again – find me an algorithm that can accurately pick stocks, predict the weather, etc. and then I'll fear your model."
I don't really understand this statement at all. I have a model of what the average role of a six-sided die is, but somehow every time I roll the die it fails to come up 3.5. Does that mean my model is wrong? I think you are confusing variance with accuracy.
With individual baseball players, you can't tell the difference. Is Jeter's market-beating 2009 due to variance (he had his 95th percentile year), or because the market was wrong? We'll never know. But after a while you have to get some things right – you can't just keep arguing that your projections were perfect, and variance was to blame. So build a model that can pick stocks like Buffett over time, and you have something. But if all you can do is say – "assuming our earnings forecasts for company x are correct, then I expect the stock to rise," well that's not really going to do much for me. Tell me which companies are going to beat earnings estimates over the long haul. If I have that, I'll know to buy the stock.
I guess I don't understand your point. Is your main point simply "having a superior system of converting projections into dollar values is not a signficant advantage"? If so, could you quantify that somehow?
If your point is simply you'd rather be the world's best projector rather than be the world's best at turning projections into dollars, then I agree with you, but I don't think that was your point.
Also your Buffet analogy does not really work for many many reasons. Buffet buys many many stocks, most go up, some go down. Almost none of the do exactly what he thought they would. To me that sounds like what I would expect the Phipps group of baseball players to look like at the end of the year.
"having a superior system of converting projections into dollar values is not a signficant advantage"? If so, could you quantify that somehow?
Chris, converting projections flawlessly into dollar values is only helpful insofar as your projections are good. If your projections suck, then so will your dollar values. If you have bad projections, and your conversion to dollar values is distorted, it might even help you.
All having a good conversion model does is ensure that you're buying players exactly according to whatever projections your gut or faith has told you to adopt. Without knowing the accuracy of one's projections, I don't see how it's possible to say whether having a tight conversion model is an advantage.
Maybe Buffett was a bad example. I'm sure you could argue that he just takes the bargains that the market leaves behind (like Bill tries to do) rather than pick specific companies to blow up. I don't know the balance there, so not a good analogy, you're right. My point is a model that could create projections that were more accurate (obviously perfect accuracy is impossible due to variance as you say), that would be really useful. A model that just priced players according to consensus market value and had you get whatever players went for less than that is less so.
I guess it comes down to this – if you're trying to be the house and collect whatever vig comes in the form of whoever goes cheap because squares overpaid for their pet players, that's great so long as the squares are using your same market values, or are random as to which players they pay more or less for. Or if by some miracle the market was accurate, accounting for variance. But if you come up against a sharp, and you set the line at current market value and let him pay the vig to have his pick, he will clean you out more often than not. Because you will have your dollar values that reflect the market or your projections, but he will have better projections (or intuitions).
I think that's why Vegas doesn't mind if the public is on the other side, but if a sharp makes a big bet, it'll move the line in a hurry. It's dangerous to set lines against people whose projections are better than yours. Even if you have a vig.
So maybe Eric's right, (and we are going in circles) and it comes down to whether getting the vig (according to the market) is a bigger edge than choosing the players. I think the semantic confusion is whether you think there's wisdom in crowds, so to speak, and that the market's consensus (whatever that is in this case, assuming it exists) is a good starting point for projections. In that way, maybe it's not really as big a leap of faith as going with CHONE or Shandler, making up your own projections or going with intuition would be. Other the other hand, it might be worse than those, and choosing where to set the line, i.e., what your projections are, is dangerous unless you really know what you're doing.
And I'm confusing two threads anyway. (1) I'm actually an agnostic, too – buying ARod and Longo because my "model" told me they were undervalued in this format for many reasons, not because I think they're especially likely to earn a profit and (2) when I do target players (genius picks), it's ones I think have risk/reward ratios not properly priced by the consensus.
Maybe I just need to get some sleep (this 8-11 am daily show is killing me) or get my thoughts in order before I start vigorously *and publicly* arguing a point.
I agree that having a model doesn't help you if your projections are terrible, that is a fairly trivial point. For that point to be important, you'd have to show that some people in the league have significantly better projections than other people in the league. You certainly haven't done that. So I'll ask my question again- assuming everyone has equally good projections (and I don't mean identical, I mean equally accurate on the whole) would you still say that having a superior pricing model does not give you a meaningful advantage?
Can we all just agree that the better the inputs, the greater the value of the pricing model? That really seems hard to argue with.
Let's save the rest for an in depth discussion of the inputs.
Chris Liss you have completely misread me. I started with the simplest possible assertion to show where a theoretical discussion begins. I said we have a set pool of players and a set amount of money to spend on them and any rigorous approach to the problem involves assigning values to everyone in the pool and having it sum to the correct total. That is the problem presented before an auction. No value exists in a vacuum. Yes, you can play the game without doing this but you are not taking a gamer/theoretical/mathematical/logical approach to it. You want to go from there to assuming my model is constructed of simply divvynig up all the projected stats with no accounting for baseline problems, positional issues, volatility of players performance or any of the many things that must be addressed to have a reasonable solution to the problem. I wanted the simplest possible starting point and we could go from there.
Actually, Robert, it's you who have misread me. How am I not taking a theoretical/gamer approach just because I don't have specific values?I've got more theories and reasons why certain things should or shouldn't be done than almost anyone. Why must one specifically project numbers first? The problem presented at auction is spending your $260 on the best team. It does not necessitate creating specific projected stats for every player to do so. Given variance every year and the impossibility in this case of separating it from projection errors (unlike in poker where you know exactly when you got your money in good whether or not you won), there are no perfect projections – or at least no knowable ones. So is it possible your approach which necessitates specific numbers and values for each player is not optimal? And I have retracted the assumption that you don't account for these things, and simply pointed out that there's a lot to account for.
And Chris – you agree that having bad projections and a good conversion model doesn't help you, but you think it does help if everyone has equally good (or bad) projections? How so? Let's say our projections are equally bad. Why does accurate pricing according to my bad ones beat inaccurate pricing according to your bad ones. Isn't it possible my inaccurate pricing helps me offset the badness of my projections?
And what about someone like me who has no specific projections? How does pricing accuracy with necessarily wrong projections give you an advantage over me?
I'll give a rigorous pricing model based on whatever the inputs are credit for one thing – that may be an actual advantage, albeit in the way a placebo actually makes people get well faster sometimes: it makes you feel confident in your decision-making at auction, and maybe that helps cut down on some egregious errors that you'd make if left to your own impulses. The illusion of science (even though the inputs are largely faith in some system or the accuracy of the consensus) might be worth something. I don't need it at this point because I'm confident in doing it on the fly even though I know I'll likely make some mistakes (at least minor ones) every auction.
I would say no, it is not possible that having inaccurate pricing helps you if you have bad projections.
What if I told you that my model for projections had means and variances? So for Albert Pujols it says he is expected to hit between 38 and 48 HRs a 80% of the time. Would that give you more faith in my projections or pricing? Do you really think that would change the pricing signficantly from just saying he is going to hit 43 HRs?
It's not possible? Let's say I overvalue steals, but your projections undervalue young, upside players. So I draft a team that skews young.
With Pujols, it doesn't matter as much. With Cam Maybin or even a Ricky Romero, your ranges are suspect to start with, and your expected dollar value is likely to be off, too. Does it matter whether you use a range, or strict value? I don't know.
That isn't an example of your inaccurate pricing helping you. That is an example of both of us making a mistake and then you making up a story where this somehow works out okay for you.
Yes it is – I misprice steals, and it helps me skew young and take advantage of your bad pricing on young players. If I didn't overvalue steals, my team wouldn't skew young.
Of course we can construct a chain of events where having a bad system helps you. I can construct a series of events where through sheer luck and chance a system of pulling bids out of a hat can end up helping you. I'm talking about the expectancy of your process in the long run.In the long run having a bad pricing model is not going to help you. That is what we mean when we call it "bad".
I don't think we are going to get anywhere. I don't really know what your overall point is except a vague idea that "having a good pricing model is not particularly advantageous".
I think the analogy is like this:
imagine u had a bow aiming at a target. If you were inaccurate but precise, say you always shot 10 feet to the left exactly, you would prefer to be less precise if all you cared about was whether you ever hit the target or not.
However, suggesting that projections are inaccurate in this way or that less precision is ever a net positive (other than occasionally randomly being good) seems silly to me.
I'm not arguing it's a net positive – only arguing that it's positive in some cases, and not obviously a net negative. Your projections are off. So why is it so important to convert them accurately into dollar values? Why can't you randomly be helped nearly as often as hurt by a conversion problem? Keep in mind the magnitude of random errors is reined in by the market, too.
Caroline J. Beck September 14, 2011 at 6:04 pm | Thanks for the great insight. It’s also inprotamt to realize that unlimited online shelf space can create a double-edged sword. Too many choices make it that much harder to choose!
If mistakes are not "obviously a net negative" I really don't see where there is room for discussion. This has reached the level of parody.
I agree with that part where you say that the market does help minimize the effect of some mistakes- besides that I don't really get what you are talking about.
Mistakes as to translation assuming erroneous projections, and limited by the market on the downside are not necessarily a net negative, and even if they are, the fact that in some significant percentage of the cases they're positive, then the edge for correct translation with wrong projections is even less. But nice touch with the dismissive snideness.
Also, let's assume the projections were done randomly – do you still think your translation model would confer an advantage? I don't. If you agree, then at what point does accuracy in translation not matter? In other words, how inaccurate must the projections be before the translation simply doesn't matter? Maybe our current projections are already there given the market limits of the errors in inaccurate translation models.
Eric's example of the archer who is consistently wrong versus the archer who is erratically wrong (and thusly sometimes inadvertantly right) is why there is room for discussion. I really think we need you guys to roll out a description of your model, or as Robert started to do in this thread, lay out the foundation for the basic problems that need to be solved.
Right now we're making our argument based on the idea that you guys have a model that is like all the other models we know for valuing baseball players for the roto game. Maybe that isn't so. Maybe you're onto something new that will turn everything upside down. You can tell by Chris's vigorous argument that he's skeptical. I am, too, but it is circular for each of us to argue against the other's misconceptions about what we're saying. So let's go back to the beginning.
We know $2600 is going to be spent on 10 teams of 23-28 players. We know that additional free players will be added to the system. Some number of players will have positive value in any of 1-5 categories, and some number will have negative value in 1-5 categories. We also know that we have the best projections in the world, the gold standard, and that they fail to describe accurately about 25 percent of the final results. About 10 to 15 percent of this inaccuracy is due to injuries that occur randomly. The rest is due to estimates of playing time that are wrong, stochasticity baked into the game, and the fact that a player is never the same player from one year to the next, so his baseline is ultimately unknowable. How do we turn those conditions into meaningful values that help us win the game?
What argument are you making? The argument about the pricing model being not worth much seems entirely built upon the fact that Phipps is using bad projections. So, if your point is "Phipps uses stinky projections", than I don't really have an opinion on that. I haven't seen his projections.
The point is that the best projections are slightly better than a weighted average. If you're pricing players based on projections, you have to either give a wide range of possible projections (the way PECOTA does it) or you have to pick one middle projecfion, which on average will be off by about 25 percent (and pretty much like all the other projections). If you're projecting someone to hit 24 homers and the range of possibilties for a projection as accurate as possible (if they don't fall off the map of entirely) is 18 to 30 homers, how much value does the precision of the pricing model have? That's the issue.
First, I don't think that using ranges really changes the output of a model. I know in my model there wouldn't be any real change in anyone's price if i started using probabilistic ranges for stats rather than just mean expectations. It would take a model with a really unusual approach to give a different value for a 15-25 HR range than it gave to a 20 HR projeciton.
As for your other point- the fact that everyone's projections will look bad at the end of the year doesn't invalidate precise mathematical methods. Fantasy baseball has a lot of variance- it isn't chess. This means that there are limits to how much edge a good player can have over a bad player. But the edge is still there. There is a lot of variance in poker, but the better players end up with the money in the long run, even though many of their single decisions don't work out. In games likes this you find a method that gives you a little bit of edge, and you apply that edge over and over and over. And in the end it adds up. Would you have more edge with a better model if there was less uncertainty? Probably, but that doesn't mean your model isn't helping you. The inherent variance of baseball player performance does not invalidate a good model. Math just doesn't support that conclusion. Variance does not change expectancy- it hides it.
"The inherent variance of baseball player performance does not invalidate a good model. Math just doesn't support that conclusion. Variance does not change expectancy- it hides it. "
My argument isn't about invalidating a model. If you have the same projections as everyone else and a better pricing model, you might have some grind em out advantage over those with less effective pricing models. The argument, however, is that that edge is small compared to other aspects of the game play.
I'm going to try and come up with some thought experiments to illustrate why the fact that individual player performances deviating from projections does not signficantly reduce importance of a pricing model. This might take a couple days as I don't want to waste people's time with hogwash.
The other issue is that just talking about how accurate projections are is a very complex issue. BaseballHQ projected 11 players in the AL this year to hit between 24 and 26 HRs. So if those guys combine for 275 HRs this year, would you call that projection accurate no matter how much the performances of the individual players deviate? What if they only hit 250, but that is because 10 of the guys hit 25 and one of them hits zero? Or is 11 too small or too big of a group to do this analysis on? These issues aren't clear to me.
No and proceeded to tell me all the hirorble things about the first six weeks, from the cramps and bleeding, the mood swings, the colic, the sleep deprivation and I was not happy to hear it.Then I had my baby and discovered that my life had turned upside down. It hit me like a ton of bricks, that no longer could I just live for myself but I was responsible for a little baby for 24 / 7. I was tired, sore, cranky, unsuccesful at breastfeeding (and beating myself up about it), worried about everything and when she called I was able to vent to her. I said I want my old life back. I want to take him back to the hospital. She knew I was venting. She laughed and I did too. But we could be HONEST with each other. I didn’t have to oooh and aaah and gush about how much I Loooved my baby. It was nice to be able to say this sucks! But I did and do love my baby, like I’ve never loved anything else in my life. I don’t want my old life back (well, not always
I love my children and if anything happened to them I probably would drive myself off a bridge because what would be the point of living.Motherhood presents us with strong emotions. Good and bad. Let’s not beat up this woman who expressed one of the bad ones.
I think the reason it's hard to make accurate projections even setting aside injuries and variance, i.e., luck, is that healthy players' skills don't remain the same. And anticipating skill growth and regression, apart from luck, is the key to the game. One can plot general growth curves for players (hitters peak and 27 and slowly decline), pitching, I think, at 31, but the general curve is just an average and does not apply to individual players whose growth and regression can be gradual or abrupt, or proceed in fits and starts. So whatever projections you use will be wrong unless they can anticipate growth or regression optimally for each player – apart from the variance that accompanies their growing or regressing skill set.
The problem with projections whether you use a range or a specific number is that by their nature they assume an average amount of growth or regression given the player's age or experience level. If I think Justin Upton will hit 45 homers this year (that that's really his new baseline given his park, his rapidly approaching peak, his growing experience, etc.), I cannot possibly put that number into the model because it's too far away from the normal growth curve of a player his age and with his history. But outlier careers exist! Not just as a matter of variance, but outlier baselines exist. If you were doing career projections you could not have given Pujols or ARod their actual baselines. It was too unlikely. But some player is going to be Pujols or ARod. So your model either has no outliers or random outliers. That's the trouble with projections – they're impossible to do right not only because of variance, but because they're either too timid, or too speculative.
So I've found it's better to just practice identifying the breakout players, and go the extra buck for them at auction (within reason). Just get really good at identifying them by being honest and rigorous at the end of the year about which ones you got right, wrong and why. Condition one's brain to synthesize the data well enough to get a sense of what collection of variables gives the best chance to find the breakout guy. If you're good at that, then you'll win your leagues more often than not. If you could build a model that synthesized those variables better than my brain, I'd be concerned. I don't know if that's possible, but if there were a projection-creating model that was more than as Peter says, weighted averages, that would be something.
As I've said many times before, I'm skeptical that a pricing model (as opposed to a breakout-finder model) would confer a significant advantage. And I've probably argued it more vigorously than I've needed to. The bottom line – in my experience playing against really strong players some of whom use a model of that kind and some who don't, it's not a difference maker. It's possible in the 30 or so high-level experts leagues I've played in, and the 50 more regular leagues, I've missed something, and there's more to it.
But apart from my experience the fact that most valuation models depend on projections, and projections, it seems to me, are necessarily timid or random makes it hard for me to think that any system that merely seeks to accurately translate projections is going to be a winner unless the other players aren't any good at finding the outliers, i.e., are as likely to leave the breakout players behind as they are to choose them. Then the vig might be enough to make the difference.
That's my essential point.
Now THAT's a pissing contest if I've ever seen one!