The Limits of Analytical Thinking in Fantasy Baseball
April 19th, 2010 by Chris Liss in UncategorizedI made a comment earlier that analytical models of the kind used by the quants here are great for solving the mysteries of the past but not very effective for guiding us as to what will happen in the future. Here's why:
The models discussed here are really nothing more than analytical tools. To the extent that one is analyzing data, they are useful.
But what does it mean to analyze something? From the online etymology dictionary:
analysis is the "resolution of anything complex into simple elements" (opposite of synthesis)".
This works well when you're looking backwards because the present (the starting point) is already formed, and you're trying to trace it back into its component parts. That's how time works – from myriad possibility into a single actuality. From the many to the one. Looking backwards, you cut up the result into its component parts – via analysis.
But when you want to go in the other direction – from past to future, you must assemble the component parts into a single whole. This would be the process of synthesis, the opposite of analysis. And analytical thinking is not all that helpful when your aim is to synthesize component parts into a whole. The models discussed above are just tools to amplify the power of one's analytical thinking. They confer no advantage if we reject the premise that analysis is the right thought mode through which to attack this problem.
You can also see that these models always proceed from present to past and never in the other direction. Consider how they work – by positing a fictional future (year-end projections for every player) and then working backwards to see what each was worth at auction.
Why posit a fictional future? Because that's the only way the model can work – backwards.
Moreover, the modelers do not acknowledge that what they're trying to do is get the best possible results in predicting the future by using a tool best designed for understanding the past, and in fact go further, insisting there is no other way to approach this problem. Even after you describe another way, they insist that you must really be doing it their way. And that if you could somehow combine your way with their model, you'd do even better.
This is the case even though they're well aware of the flaws in creating projections from the impossibility of knowing a player's statistical baseline, to the imprecision (and even arbitrariness) in calculating a player's breakout and failure scenarios. But because they believe analysis (positing a fake future to look back from) is the only way to proceed, they assume everyone has these same problems.
But I think what I've been doing is synthesis. It's another mode of thought – similar to how an NFL quarterback absorbs myriad information in a split second and knows where to deliver the ball. I'm assembling the component parts and synthesizing it into a bid. It's not without it's problems, and it's by no means a guarantee of anything (Even Tom Brady who is far better at his job than I am at mine throws interceptions). But at least it's designed for the task at hand.
Analytical thinking is important – Bill described its achievements in one of his first posts, and one cannot dispute their significance. But that does not make it optimal for solving all problems, pricing the future performance of fantasy baseball players among them.
Loading ...
I'm just getting caught up on the weekend's storm of posts and comments, so forgive me if my thoughts are a little disconnected.
Chris, it seems like you're essentially saying: "Wherever the market settles, if my mind tells me I should bid one more, I will."
I just can't fathom how that would be better than actually figuring out what you think a player is worth to begin with.
Whether you're cognizant of whether you think Heyward has a 15% or 20% chance of breaking out or whatever, you are making some sort of call on this – and it's being made as a split second decision instead of thinking it through as rationally as possible beforehand.
"Moreover, the modelers do not acknowledge that what they're trying to do is get the best possible results in predicting the future by using a tool best designed for understanding the past, and in fact go further, insisting there is no other way to approach this problem. Even after you describe another way, they insist that you must really be doing it their way."
False. Speaking for myself, I am the first person to admit that projections are flawed. Projections are flawed. All of them. They will never be perfect. Ever. All we are doing is trying to get the best possible result in an imprecise game. I'm not insisting that projections are the only way to approach the problem – I'm saying that unless you are randomly making bids, you necessarily have some sort of expectation for a player. And that's all a projection is – an expectation. It doesn't get any simpler than that. Whether you care to label your judgments and expectations of a player a 'projection' or not, they necessarily are.
Even if you reject the notion of a 'projection,' as long as you're coming up with expectations for a player and not simply making random bids, you're going to have a margin of error too. And what I'd really like to know, Chris, is whether or not you believe that your margin of error is smaller than that of a projection system. I don't believe I've heard you answer this question yet, and I'd very much like to hear your thoughts.
You say that our models are using past data to try and predict future outcomes (something I'll freely admit), but aren't you doing the same? All of your experiences and everything that you're researching is all past data as well – and it will never be any different (for any of us).
Finally, as to this Tom Brady analogy that keeps popping up, I'd contend that this is very different than bidding on players. Tom Brady has thrown a football tens, maybe hundreds of thousands of times in his life. He is repeating the exact same activity, over and over again, every day of his life, making large use of muscle memory in the process. In drafting fantasy baseball players, we are not performing the same exact activity time and again, and we are doing it once or twice (or six times) per year. We are making stabs in the dark on different players with different backgrounds, different skill sets, different aging and growth patterns, different everything – which seems to be one of the tenets of your philosophy – so how can it be the same? How many times have you bid on pre-2010 Albert Pujols? Pre-2006 Alex Rodriguez? Pre-2009 Nick Punto? Throwing a football is extremely repetitive, relies upon muscle memory, and provides instantaneous feedback. Fantasy baseball auctions do not.
False. Speaking for myself, I am the first person to admit that projections are flawed. Projections are flawed. All of them. They will never be perfect. Ever. All we are doing is trying to get the best possible result in an imprecise game. I'm not insisting that projections are the only way to approach the problem – I'm saying that unless you are randomly making bids, you necessarily have some sort of expectation for a player. And that's all a projection is – an expectation. It doesn't get any simpler than that. Whether you care to label your judgments and expectations of a player a 'projection' or not, they necessarily are.
I’m confused – what have I asserted that's false? It seems you're saying exactly what I said you would. I never said you didn't know projections were imprecise – I said you insist we're all using them and that it's the only way even though you know they're imprecise. And that's what you're doing!
as long as you're coming up with expectations for a player and not simply making random bids, you're going to have a margin of error too.
Which I acknowledge above.
Finally, as to this Tom Brady analogy that keeps popping up, I'd contend that this is very different than bidding on players. Tom Brady has thrown a football tens, maybe hundreds of thousands of times in his life. He is repeating the exact same activity, over and over again, every day of his life, making large use of muscle memory in the process.
This would be valid if Tom Brady were the champion "throw a football throw a tire 25 yards away” guy. He's not – he's a Hall of Fame NFL QB. No situation on the field is exactly the same, and it takes a lot more than muscle memory to be a great quarterback – one must read the situation and synthesize the data immediately. One can only do this with great study, practice, experience and aptitude. It's why rookie QBs are almost never good – even if they have plenty of reps and muscle memory or could compete in a "throw it through a tire" contest with Brady.
But Chris, how is what you do not a projection in some sense? All a projection is is an expectation of a player. And you have those.
I believe fantasy baseball is a combination of analysis and synthesis. These are complimentary ideas, not absolutely opposed…
If you choose this definition of analysis – a philosophical method of exhibiting complex concepts or propositions as compounds or functions of more basic ones -(from dictionary.com)- you can make very different agruments about what it does for you. It allows you to take a very complicated system (FBB) and break it down into smaller parts to evaluate. I am sure you consider the parts of the whole, steal and HRs and average or wins and era.
Using them together gives you the ability to generate an idea of what each of those items does to the whole, (analysis) and then bring that together to form a bid (synthesis). Whether you do this in advance of the auction (projections) or at the auction on the fly (gut) doesn't change the fact that it happens.
The modelling argument merely is stating that I am giving these items the same value for each and every player, while the gut argument can't make that claim.
The advantages to doing this in advance are numerous. You can make sure that all of your values add up to the total available budget. You can track the player pool. There is a point in any auction where half of the value of the pool has been bid for (regardless of how you value it), wouldnt it be useful to know that 60% of the money is spent? the remaining players are now worth 80% of their previous value (smaller amount of money to cover same player value) or that 40% of the money is spent? (players are worth 120% now…)
These are advantages that can't be quantified on the fly, you have to guess at them. Even if you have a very good feel for how it works, you error rate can't match that of a spread sheet.
I have a vague expectation of a player, but it's not the driving force behind my bid. I don't expect Romero to break out. I think he could break out. I think he could get destroyed, too. I don't know. I keep it loose, untranslated, but I decide he's a player I want to own for the right price, whatever that is. And when his name comes up I bid until the price is no longer right. I just try to keep an eye on the possibilities, both likely and unlikely. I don't have particular expectations of any player beyond the obvious common ones – Rajai Davis is steals guy, Carlos Pena is a power guy. Sure. But those are just facts about the players, not really expectations. I'm not trying to project their season. I'm just consider the possibilities loosely without reducing them to a specific expectation. There's no need. I trust myself to value the player without it.
But Rep, you're starting from the fictional future (projections) and working backwards and then reassembling them. That's analysis. Adding together the component parts that you got from analysis of your fictional future is not synthesis – it's more like trying to put Humpty Dumpty back together again. This is because you've lost all the facts (team context, health, history, etc.) that you'd need to synthesize – they've been converted into this false future and broken up into pieces of that false future (HR, RBI, runs). The facts are gone.
The process of synthesis must operate on the facts, not fictions.
You when you claim to just know what you want to pay for a player, what do you think that is a combination of? or are you asserting you come up with a singular value that has no components?
Do you deny that somewhere in your subconcious mind you are taking into account a variety of information (from analysis of past facts? what makes you say pena is a power guy?) and synthesizing them into a bid?
And on a different note, how would you recommend a new person to FBB go about learning how to be better at the game?
what makes you think Arod is worth more than 30 and jason barlett isn't?
I definitely take into account a wide variety of facts including past stats, injury history, team context, etc. To the extent I'm able, I try to eliminate all fictions – hype, bias, expectations – and synthesize only the facts into an impression of the player's many possibilities. It is that impression that I believe I am bidding on.
I think a new person should just play. Grab the RotoWire cheat sheet or whatever, do the best you can, and see how it plays out. After one season if you pay attention, you'll be ready to start figuring out what works. For an experienced player, I'd say – take a really close look at the player pool. Don't presume you know who Carlos Lee or Andy Pettitte is. Look at them again, their numbers, their skills, their team context, their health, the second half, etc. It can be tedious, but I don't see a way around it.
A projection system derived from Regression Analysis takes known outputs and backtracks to see what inputs were important from a broad range of data. Some component inputs are material, many don't make a difference. In the end, a formula that takes known inputs and converts them into a known output is created.
To some extent all projection systems work this same way, even if the originate as weighted averages or even someone's gut check call. They work from a known pattern and then, using inputs that don't have known outputs, create their projected outputs. This is the opposite of working backward from the future to create an analysis model that loses information. It saves all the relevant information it can.
Converting the information we know about a player into prices works the same way. The real question here is whether projections is the right way to derive a future pricing model for fantasy baseball. Chris says no. I think not. Ron Shandler doesn't think so. Alex Patton laughs at the idea. Todd Zola would giggle at the thought. Lawr Michaels has his own methods. Les Leopold doesn't work from projections, though he got me making them back in the day. Other then Chris (and maybe Lawr), all these guys–with scores of years of professional fantasy baseball experience between them–would readily agree that they convert their analytical expectations into something resembling expected prices, which guide their bidding. But that isn't the same thing. Isn't this a better place from which to move this discussion forward?
What do the pros actually do? How do the sharps think they can improve it, and can they?
Is the cheat sheet basically just a weighted average of the previous years stats with some considerations for career arch, team, and etc?
Peter, all the pros publish their auction/draft guides ranking the players at each position. When they claim someone is the #1 prospect at a position, they are PROJECTING the future. They are claiming the #1 guy is most likely to have the best season. Regardless of how they reach that decision, (assuming they aren't drawing names out of a hat) they are using the past as a predicitive tool of the futute.
Almost all the pros would agree that ARod will likely have a better fantasy season than Jose Lopez. How do you reach that conclusion? ARod will likely hit more HRs than Jose Lopez. Hard to argue with that. History says ARod has more power than Lopez. The interesting questions are: how many more HRs is ARod likely to hit than Jose Lopez? If you could bet on it, would you take ARod and lay 10 HRs (ARod hitting 10 more is a push, 11+ more you win, 9- more and you lose)? Would you lay 15 HRs? Certainly you wouldn't lay 50 HRs. But this type of thought process leads you to a value where you wouldn't take either side. That is your expected value. There is also a point where you would lay 2 to win 1 on ARod. And a point where you flip and take the other side. That is the 33%ile for you. (2 events above for every one event below)
Projections are by definition guesses. Effort is made to make the best guesses possible. It is even understood by most who project the future, that it is not absolute. A range exists for the possible outcomes. If the potential choices are all considered, you have what is called a probability density function. A pdf has many of the same properties as a distribution of known events (historical data). There is an expected value and a standard deviation (a measure of how quickly the results disperse from the expected value). If you take projections and try to turn them into absolutes, you are making a mistake. If you take projections and turn them into ranges for the future, they have predictive value.
Here is an interesting theoretical question for you. You divide up the pool into 10 teams with even value. (meaning at the end of a large number of simulations, the average value of the player preformance is 260 for all the teams) What other factor will determine which team wins the most often?
I am misunderstood, I guess. Or I misled, more likely.
My point, which I thought I had made explicitly, was that we all make projections and we all make prices. I've certainly said that a few times. I think even Chris has agreed to as much, at least in some meta Tom Brady kind of way, though he's mostly denied it because he resists the definitiveness of it, and it isn’t his draftday method. Whatever. A few days ago and a couple days ago and maybe even yesterday, I wrote that I make projections and I make prices. The question for me is how we use them.
When I wrote above, "Converting the information we know about a player into prices works the same way. The real question here is whether projections is the right way to derive a future pricing model for fantasy baseball." I was arguing against taking values derived from projected statlines as the basis for any pricing system. I meant real honest-to-god projections. Statlines. I think that's a mistake.
Of course I make bid projections. In fact I make statline projections, too, because they have their uses, and I make bid price projections, using a different method, because that's how I run my auction. When I listed the Touts above who don't use projections in their prices, I meant that they don't use projected statlines to create derived bid prices, which is the method Robert was describing of his model and which I thought was the basis for most of this debate, at least when we aren’t arguing about freestyling.
I thought it was obvious they the touts project future value. I said they "would readily agree that they convert their analytical expectations into something resembling expected prices, which guide their bidding." I meant that they project the future, they guess at it, but they don't use numerical projections to do it. Each has his own approach and values, but I can say for sure that none of the listed ones converts projected statlines into values and then uses them as bidding guidelines. Not at all.
I think there is a good reason for that, which is where I was trying to steer the discussion.
I'm sorry that I wasn't clearer.
Peter, I don't think I do projections even in my head. I have some ideas about players, but I keep them vague on purpose. I liked Ricky Romero, I wanted to get him. But I didn't know if he would break out of fail. I thought he had a chance to break out and a chance to fail. I didn't know what I thought the percentages were. If I were forced to put a percentage on it, I wouldn't trust it. It would be arbitrary. Just something that seemed plausible but certainly not meaningful or reflective of a deep feeling I had either way. But I liked Romero because I thought his chance for a good year was legitimate – the ground ball rate last year really jumped out at me. I wanted to own him, and I hoped to get him for a reasonable price. If the price became unreasonable, I would have moved on. That's about as specific as I get. Because I got him in LABR for 3, I suppose I had an idea of what reasonable was, but I'm not sure whether or not I would have gone 5 if someone had said 4.
"Because I got him in LABR for 3, I suppose I had an idea of what reasonable was, but I'm not sure whether or not I would have gone 5 if someone had said 4. "
You've said this kind of thing a couple times, Chris, and it strikes me as odd. If, even afterwards, even by putting yourself in that position again, you don't know what you would have done, how do you determine whether going to 5 would have been a good move? Removed from Romero, I'm sure there were other times in the auction where someone pushed you and you were forced to decide if going the extra dollar was warranted. But if all there is is your gut driving your decision at that point, how can you determine whether it was a net positive or net negative decision?
Put another way, if you don't know whether you'd go that extra dollar or not, there is some non-zero chance that you would, and some non-zero chance that you wouldn't. So assume, for a moment, that we have two parallel universes. In one, you push the extra buck. In the other, you don't. Are you making the right decision in either one? And which one? And how do you know?
So assume, for a moment, that we have two parallel universes. In one, you push the extra buck. In the other, you don't. Are you making the right decision in either one? And which one? And how do you know?
One never knows for sure, of course, and that's true whether you use a model or do what I do. But we do have some clues: how did that player do? How did other player remaining do? Did I still have enough money left to get other players I needed? How much did other players go for relative to that player, and how did they do?
The reason you never know for sure is it's hard to separate what's luck and what's skill. Both by the players, and also in your picking of them.
I got Alex Gonzalez for $1. I didn't set out to get him, but I did envision the possibility that he'd hit for decent power for a MI if healthy. So I wasn't miserable to settle for him. Let's say he hits 20 HR. Was that skill on my part or luck? Skill in that I was willing to take him. Luck that I won the bids on other players and needed a $1 player there.
So this whole "right decision" business is tricky but over time you have enough results to get a better sense of where you went right and wrong. I don't think this problem is dependent on one's method. Because you can get good value according to your projections, but the player does terrible. Or you can see me overpay by your projections for most of my team and win the league by 20 points.
Chris,
I'm not saying that you can't compete or be wildly successful by taking this approach. Not at all. As I've said before, you're an accomplished player, who I have a lot of respect for, and who has had success in the upper echelons of this game.
My point here is that, if you forcing yourself to make decision during an auction that, even in retrospect, you don't know which way you would have went, you are giving something up. You are giving up edge. If there is a non-zero chance that you will bid $1 more, and there is a non-zero chance that you will not bid $1 more, each time the bidding stops on say, 50 players during the course of the draft, if it can go either way, by definition you will be wrong on some of those players. Doesn't matter which ones they are or if you can even say at the time (or even afterwards) which they were. But you must be, given these parameters. Not as a matter of projection (another matter entirely), mind you, as a matter of function. And I'm not saying this is insurmountable (it certainly isn't). I'm just saying that it doesn't seem to be ideal. Doesn't seem, to me, to be the best we can do.