Before I start, I want to say the following: Bill and I feel we have analysis to share, and would like to do so regularly. We were surprised at the sheer volume of objections, and how far afield they seemed to run, when we tried to begin our presentation with some general points. Neither of us are particularly quick writers and we are both extremely busy with work and family. At this point we feel the most constructive way for us to proceed is to share details in a hopefully regular manner without delving into what appears to us to be an endless and somewhat premature debate. It's a long baseball season, and I'm sure by the time we are done we will have addressed many questions and at the least will be better situated to address anything that remains undiscussed.
I want to make sure everyone is on the same page about what type of model we are discussing. It is simply a model that takes a player's end of year stat line and converts that production into auction dollars. Also, I'm sure all the experts and poker players are familiar with the concept of Expected Value (EV), but for anyone who isn't please try wikipedia or some examples like this.
In my last article I asserted that the clear starting point in any auction is to examine the pool of players and divide the total budgets of participants among that pool of talent. Somehow people assumed I meant to take everyone's projected stats and compare them to the worst player at their respective position and divide the dollars exactly along the lines of this kind of VORP assumption. That is neither what I wrote nor what I meant. I was simply trying to take this in a one-step-at-a-time approach to see where we diverge, and then discuss those points of divergence when we get to them. For people who accept that as the first step, I hope this article makes an interesting second step.
There have been several questions in the comments threads wondering how you can possibly price young players who derive most of their value from their upside using a model. I will address that here, but since I do agree that is the trickiest type of player to price, I would like to build up to it by starting with the completely vanilla case, moving on to something slightly trickier, and finishing with the young upside guy.
When pricing a full-time healthy player with a secure job, the player's value is the conversion of his mean projected stat line. When you buy a player like Longoria or Weaver you are buying their stat line for the entire season. Whether you trade that away or not, those stats will be realized, so the expected value in Fantasy is going to be the expected value of production. This is the mean projection, not the median, as has been asserted several times in the comments.
When pricing a player who is starting the season injured, you can no longer use just the mean projection. This will undervalue the player. Take Bedard this year. If you think he is returning 1/3 of the way through the year and so create a stat line projecting the 2/3 of a season you think he will produce upon returning you will be missing an important point. Until his return, you have that pitching slot and get some other production out of it. Fortunately, this is a number you will already have addressed in making your model, as it is simply your baseline pitcher production. So what you need for valuing Bedard is to use 1/3 of Baseline + your mean projection for Bedard when he returns. This same method is how you would value players you are hoping will be called up like Carlos Santana. For a player you hope will be traded over from the NL (Adrian Gonzalez) it is very similar, but you also need to include a factor for the likelihood that he is traded over as well as for the timing.
Now let's examine a 'young player with upside'. The variety here is much wider but this is the general approach that can help you estimate the auction value of a player like this. Let's say that we are looking at a player who is starting the year with a job, but is far from locked in to keep it. Here we need to think about the path his production can take and how that will translate into fantasy value. Let's break it down into three groups of outcomes and assign each a probability and a mean projection within that outcome:
Outcome 1: He struggles early and loses the job.
Outcome 2: He plays well enough to keep the job, but simply provides reasonable production and not the breakout upside you were dreaming of.
Outcome 3: Your upside ship comes in.
If this is a hitter, his value in O1 can not be anything wildly negative so we just call it a zero. At this point you will be finding a replacement and would project the remaining 80% of the year as baseline for that position. If this is a pitcher, he most likely produced something with clear negative Fantasy value during his losing-the-job period and we need to put a statline in there for that. In 2000 I was quite happy to pick up a $1 Roy Halladay and a $1 Kip Wells. The numbers they put in the first month while losing their jobs were bad enough to knock me completely out of contention. Too bad it wasn't a keeper league.
O2 and O3 both provide results like the vanilla player we started with and in both cases are just worth the mean line you assigned that range. For example, if this is a hitter who you think has a 20% chance of losing his job, a 70% chance of producing stats in a range where the average projects to be worth $5, and a 10% chance of breaking out and having a type of season with an average value of $25 you would call this 0.2*0 + 0.7*$5 + 0.1*$25 for a value of $6.
I foresee two pretty strong objections to this type of analysis. The first is going to be that the model is only useful to the degree that your projections are accurate, and once you get to projecting a player under two or three different scenarios, those projections are going to be less accurate. I completely agree. But I think this methodology is much better than the alternative of 'going with my gut' and trying to synthesize all this in one big step. The gut approach opens you up to pushing too hard for guys you love, because there is no check on your own bias. If you force yourself to turn things into statlines, you are forced to be honest with yourself. When you call a guy a breakout, and go to buy him, but one or two other people at the auction also think he is going to breakout, too, you will be glad you have taken the time to calculate what you really think he is worth. There is a lot going on at a fantasy auction, and I for one am skeptical that a person could crank through all that, for player after player, in real time as well as accurately convert those stats to values. Whether you are dealing with weather, stock market, or Fantasy Baseball, all models will only be as good as the inputs. By using an approach like this we can refine things better over time and see more clearly what we are messing up and what we are getting right.
The second objection I think will come up from that last example is that when you buy a player for 6 and he comes in at 25 you win much more than 19. That this is the kind of realized return you need to win a league. That, at least, is implied in a lot of comments about needing to give yourself upside so you can win. I think this warrants discussion, but I am setting it aside for now because in the near future there will be a "what it takes to win a league" article and I think it makes much more sense to address it then.
Tags: Model, Robert Dixon
You can follow any responses to this entry through the RSS 2.0 feed.
You can skip to the end and leave a response. Pinging is currently not allowed.
Loading ...
Thanks Robert. That's a good and uncontroversial start. Are you saying that you create actual $0, $5 and $25 projections for your player? Or do you simply assign the probabilities of variance off whatever basic projection you're working from?
Peter, I am glad we can get on the same page for a starting point.
I am not 100% sure I understand your question, but what Bill and I try to do on the non-vanilla players is put their outcomes in clumps and assign those clumps probabilities. Admittedly, estimating those outcomes on young players is not at all easy and I think that is an area where a lot of experts do very well, but my overall point is whether you are an expert or not it helps to quantify what you are buying. I think Carty has made a very similar point several times in the discussion threads.
The players we tend to give the largest number of different projections to look at are actually the ones in group A, but this is slightly different and more a matter of personal taste. Since we trade for a living we view most things in terms of bid and ask. What price would I pay for something and where would I sell it. We like to run a variety of projections stretching from the pessimistic to the optimistic to see what kind of range the players have. Similarly we will play around with the stat converter and stretch those, too. In the end we assign an exact fair value, because I think it is key in an auction to track the amount people have net overpaid or underpaid, we look at our own buys both as the static good by x or bad by x, but also in a range where we know the minimum stat line we just bought and have on almost all of our buys a lot of confidence that the player projects to at least that minimum.
Thanks Robert. My specific question was whether when you "give the largest number of different projections" to a player, do you give actual statlines (for Jason Heyward to fail he would need to hit .210 in 180 AB with a .340 SLG, while a Heyward topline would be hitting .300 in 525 AB with a .540 SLG, with the middle view somewhere inbetween) or dollar values (you estimate that a failed Heyward would be worth $0, a topline Heyward might push $30 and the middlin' version would land around $15).
I think the previous issue about projections and pricing them revolved around their actual utility and accuracy. I make a set of projections based on a player's past performance, and I use those values as a check against my bid values, which I derive by looking at historical earnings and cost records for that player, and compare them to similar players in similar situations in the past.
This doesn't involve as much math as your method, but it is based on bid and ask, too. What I'm interested is in what it means to "run a variety of projections stretching from the pessimistic to the optimistic." What is the process there? And do you think it is more valid than my approach.
We do put actual stats on players over various outcomes and run values on that to price out players. Our goal is to come up with a reasonable price. We want to figure out what level we are clearly interested in owning a player, what level we clearly pass, and how we feel about the in between. We also want to assign an exact value to the player for purposes of tracking the pool during the auction. I think the tool helps a little on the tricky upside guys because it allows for a better conversion from outcomes to values. I doubt I have edge on the young players because projections are going to be so much rougher that a model is going to do a lot less for me, although it is still good to know what you are paying for when you buy a guy.
I think for players with more established production levels the tool provides an enormous boost and allows for a fair degree of precision in pricing.
We will be happy to discuss projections eventually, but whether you intuit or model you still have to deal with projections on some level so I don't think it really needs to be addressed for validity of this approach.
What I think is confounding is how you get an exact value in the context of such a noisy system? For instance, if you use my Heyward example earlier, not only does your Heyward variations effect Heyward, but also the PT of other Braves. Which Braves? I had ideas before the season, but I sure as heck wouldn't put much faith in them. Yet the difference, Heyward fails, Matt Diaz or Jordan Schafer earn 12 or maybe Melky earns an extra 6 or, well, you see where I'm going with this.
It seems to me that we're talking about some broad probabilities in each scenario that effect the projected stats of the other players. If you're getting an exact value from each variation in projections, how does the tool account for all that?
First off, Robert – thanks for posting this and describing it in a fair amount of detail.
A couple comments/questions:
But I think this methodology is much better than the alternative of 'going with my gut' and trying to synthesize all this in one big step.
One question I have is how you come up with your percentages for O1, O2 and O3 for a given player. Not just what the specific projections are in each scenario (which is another question), but what the percentages that each player breaks out, fails or just does okay. And how these scenarios vary between two players in relatively similar situations, e.g., Heyward vs. Travis Snider.
Are the percentages and projections done on feel, or is there a more scientific basis for assigning 10 percent to his $25 season rather than say 14 percent on a given player?
I guess what I'm not sure about is whether you're operating on hunches/guesses at *some point* to inform the model. And if you are, why you think using hunches in the initial phases of producing your inputs is less perilous than using hunches at the auction itself.
The gut approach opens you up to pushing too hard for guys you love, because there is no check on your own bias. If you force yourself to turn things into statlines, you are forced to be honest with yourself.
It certainly "opens" you to it, that's true. But you must exercise discipline when the player you wanted gets bid up to a level where you're no longer as enamored with him. If you don't do projections and dollar values, how do you know where that line is? I know it when I see it is the best answer I can give you.
And to the extent that discipline is imprecise or prone to bias, I can over or underbid. But I don't see how that's different than saying to the extent your projection is too high or too low, you'll over or underbid. At some point, we both have to make a call.
Chris, I think you are taking this from the wrong angle. The question is not whether my inputs arrive at something "good enough". I maintain that my inputs, representing the best understanding I have of a player and converting those correctly to value represent the best answer I can come up with.
When you buy a player you buy his projected stats whether you want to think of it that or not. You might be buying his mean, you might be buying his top 15%, or you might be buying a combination of him being traded over then producing a set of stats. We all are doing that whether we want to view it that way or not.
So my question to you is why you seem to think that taking the time to attach numbers to all of your hunches and correctly model them could not help you be even better at FBB?
I don't think the French analogy holds up. FBB players have values derived from their production. You can choose to estimate this conversion using your intuition or based on comparing prices of players to comparable players, but there is still a conversion happening. Your inputs might be so good that a weaker conversion system doesn't stop you from being a very good FBB player, but I don't understand why you are so married to the idea that a better conversion system would not help you.
A better conversion system of facts to action would help me, no doubt. And that's why I'm constantly forced to revise my understanding of what's relevant and in what proportion. And also how the pool is affecting the value of steals vs. power, and what types of teams and strategies are more likely to win. There's no way my brain algorithm is at maximum efficiency in converting facts to bids or the absence thereof. So I agree.
Where we disagree is the way in which my conversion system will improve. You seem to view it as axiomatic that one must convert the facts (actual, truths that we know about the players which includes past performance) into speculative "best understanding" numerical estimates. Once one were to do that, then (to the extent they really do represent our best understanding), I don't think I could argue that conversion accuracy wouldn't help.
But I'm unconvinced of the premise that your projections represent your best understanding of a player as all your doing is taking a wide variety of facts, some of which are easily quantifiable, some not and reducing them to a number.
Clearly, there is much nuance and also information lost in this reduction. Even if you use three numbers to account for volatility (which is a good thing), even if you wisely account for time missed by adding replacement stats to Erik Bedard, you're still reducing a human being to a particular line early in the process.
So my question was about that process – in your case, projecting a high/low/middle line and assigning percentages to them. I wanted to see how much of your input numbers is really due to your best understanding of the facts, and how much is arbitrary assignment of probabilities to different outcomes. Because to the extent that Heyward's breakout clump of possible seasons should be 20 percent given his once in a decade pedigree, your 10 percent is going to undervalue him. And that's even assuming perfect projections for O1, O2 and O3, not including variance,
When I go to bid on Heyward, I know he's an uncertain commodity, but the factor that will sway me to go the extra buck within reason (got him for $14 in NL Tout Wars) might be his once in a decade prospect status – which to me means that a rare true breakout for a 20-year old rookie is more likely in his case than it would be for a player like Snider, for example – a very good prospect, but not at that level. If I had reduced him to a stat line, I might lose the 20 percent for his $25 season and never get it back.
You could argue – fine – put 20 percent in for Heyward in your model then. But in that case, I'm just fudging my model to generate the result I want – so why bother? You might say because at least I'd know what he's worth even given an optimistic view of his chances to be great this year. But I'd argue that my 20 percent estimate really isn't that accurate. I'm not really sure what the relative percentages should be for his breakout or failure. In fact, my brain just isn't that practiced in looking at players in that way. And to the extent I can look at them that way, I'm not sure how to test those probability assignments to improve.
Whereas my brain is very practiced at bidding on players and seeing how my teams (and others) fared as a result. So giving Heyward a dollar value based on the information I have flows much more naturally than given him a breakout/average/failure percentage. And actually I don't even give him a dollar value, I just know that when someone says 13, and I say 14, I hope no one says 15. If someone did, would I have said 16? I honestly don't know. I would have had to hear it, process it and take stock in that 10 second window.
So for putting down a player's specific projection ahead of time to be better than keeping it vague and trusting my gut, I'd have to believe that I had a very sound basis for that player's probability distributions as well as the numbers assigned to each. But I don't.
Given that all your examples presuppose you are better at picking the valuable players, your argument still comes down to "I think I have a better read on these players than you do so your model doesn't save you."
I happily concede an argument that if there is another Robert Dixon with the identical model and understanding of how to use it but with better projections he would be better at this game than I am. I don't understand why you won't concede that if there were another Chris Liss with all your understanding of players but a better model and understanding of how to use it he wouldn't be better at FBB than you.
I am going to write a fresh post for responding to your process on how you bid for Heyward.
It's not merely that I have a "better read" than you. I just don't believe either of us has the kind of read that allows us to say there's a 10 or a 15 or a 20 percent chance Heyward breaks out. (Unless there's some basis you have for picking the number, which I'm not aware of). It's a false precision, so as good as your model might be, it's working with arbitrary inputs.
My brain can deal with true inputs but they're just harder to convert. So there's more room for error on the front end. But at least it's getting accurate information to work with.
And Robert – how do you know you're better with your model? Let's suppose you didn't have the model, don't you think you could get a pretty good sense of what to bid on some of these guys? I mean after a few years of doing it, it's not rocket science. You won't be perfect, but there's no reason to think you'd be further off by doing it that way than by assuming 10 percent for Heyward's breakout. I just don't see how a model where I'm forced to make a call early in the process that I'm not comfortable making is going to be better than my making a call at auction that I'm very comfortable making.
You might argue – how can I possibly aggregate the data and make a bid/pass call if I don't know what Heyward's breakout percentage is? The same way one might understand a sentence or expression in a language he doesn't speak, but doesn't know which word corresponds to what. He just knows the meaning of the whole.
I think I'm good at translating the whole of the data into a number. And I don't know exactly how much weight I apportion to each factor for each player. Certain facts or numbers jump out at me as relevant in certain cases, other ones in other cases. The biggest skill I think – and I could be wrong – is not in knowing every possible thing, but in being able to hone in on what's important in a certain situation – what jumps out.
So I'm not sure I'd want to sacrifice that for a model which forced me to specify every step of the process. I don't know every step my brain makes. I just look at the facts, and form impressions which I translate to a loosely ordered list and trust myself to get the job one at the auction. I'm not sure why you're convinced that making these calls ahead of time is necessarily helping you.