Ok. This was going to be a comment but I decided to make it’s own post
Let me try to summarize my position:
Wins Produced measures the value of each players contribution on the court when their on it.
The coefficients line up perfectly to Point Margin on a game to game basis
In general the value of the wins produced can be assumed to be linear but edge conditions have been shown to affect it significantly (i.e. its harder to accumulate marginal wins for really good teams , its easier to accumulate wins on really bad teams this is the Diminishing returns and increasing returns effect). Prof. Berri has a simple model for this (see his blog and the Al Jefferson post) that we (he and I) agree needs to be refined.
With me so far? Ok let’s talk Rbs. The claim is that we don’t see the full effect of a great rebounder on a team because he takes away rebounds from his team (even though I’ve shown repeated examples of great rebounders on great rebounding terams). So rebounds are overvalued in the model. While true, the problem with this claim is that the same happens with pretty much every other statistic in basketball. When Kobe shoots the ball someone else didn’t. At the end of the day we can only credit the player who undertakes the action for the effect. The other big problem is that the effect of any player statistic on any team statistic is driven by the minutes he plays and who his replacement player is on that particular team. So above average performance on any statistic is muted by these factors. Here’s the thing, the actual value of the event itself and the correlation to winning cannot be disputed.The year to year correlation of player statistics accounting for age and now for the quality of teammates cannot also be disputed. So we know the value of the event, the probability of the event repeating and we can use this to project team wins. Which we do. And we do so fairly well. When we see discrepancies (see Heat) we work on improving the model.
I want everyone to be clear on this: I want reasoned critiques. This is how we learn and advance the discussion. Some of the best work I’ve done has been driven by criticism. Dialogue drives growth. What I hope is that the dialogue is not one sided. All of us have to accept the fact that the likelihood of any one of us holding the ultimate truth is remote but if we’re all willing to pool our efforts and give a little we might have something.
At the end of the day, the contents of this blog are just my opinion and my wife can make clear and concise arguments against my infallibility.
Just remember to think positively.
Image courtesy of xkcd
Arturo's Silly Little Stats v2.0 
Alex
12/08/2010
Hey Arturo – In my post on the subject, I compared (or at least I meant to) the rebounding/diminishing returns issue to a logistic regression. It’s hard to get close to 100% (or 0%), and moving from 50% to 55% is much easier than moving from 90% to 95%, whether it be in win percentage, rebound percentage, or whatever. You wouldn’t use a logistic regression to predict season wins (or win percentage), but how hard would it be to move the WP framework from a regular linear regression to some manner of non-linear regression? I’m mostly having trouble thinking of what the proper distribution would be.
arturogalletti
12/08/2010
Alex,
I read and love your post.
The really strange thing is that I expected WP to Point margin to be non-linear and instead it was perfectly linear.
What people get wrong is that it’s Diminishing and Increasing returns at the extremes. I’ve done some work on this issue. If you look at wins since the merger and exclude the strike season in 99, we find that wins are normally distributed with a std deviation for wins of 12.6 wins total. This means that we can expect >95% of all seasons to fall between 16 and 66 wins and it also helps the theory that incremental wins/loses outside of these parameters require incrementally bad or good performances.
So I would expect that between +/- 1.5 std devs from the mean ( so from 22 to 60 wins) the linear approximation is extremely good and it get’s progressively worse. I need to fully develop this at some point.
Statement
12/08/2010
You phoned in this post, that’s for sure. By that I mean it’s hard to write.
Arturo, well done with the WP.
Rashad
12/08/2010
I think the biggest issue with predictions, as I stated at the beginning, is with predictions at the extremes. Basically, my intuition said that the wizards would be terrible, but better than your 9-11 game prediction, and blazers would be good, but worse than your super team prediction. But maybe the extremes are just harder to predict since they are less-likely events? Who knows.
To be fair to the blazers, they’ve had a ton of injuries, but even when everyone (besides Oden) was healthy they didn’t have anything like the efficiency differential of a super team. I am still amazed that Blatche’s rebounding hasn’t improved significantly. The dude is just allergic to the ball I guess.
some dude
12/08/2010
Arturo, I have a problem with your claim of ” At the end of the day we can only credit the player who undertakes the action for the effect.” Reason being is that something like shooting can be controlled. Look at Josh Smith in Atlanta. He used to chuck a ton of 3s but last year consciously stopped doing it. Shooting and Rebounding cannot be equated the same as “events.” Players don’t always have to shoot. It’s a choice they make. Rebounding is a skill that you don’t really choose to use, you just use it. And then there’s the fact that the majority of rebounds will be had just by being in a certain spot. Shooting doesn’t work like this (or passing or drawing fouls, etc).
I love using the 06 Ben Wallace problem. When he left, they replaced him with 3 worse rebounders and yet the team rebounding was exactly the same because guys like Prince, Billups, and Hamilton all raised their rebounding. So why is Ben Wallace getting so much credit for his teams wins because of rebounds? Just because he got them? The example demonstrates that the majority of the rebounds will happen regardless, so why credit players so highly because of it? And while shots will also happen regardless, shot makes will not and that’s what is important most of all.
The funny thing about that switch was that the team wasn’t hurt on their defensive efg%, either, blocks, or forced turnovers. Almost nothing changed, but their defensive efficiency got worse. The reason was Ben’s replacements fouled at such a higher rate that the opposition got a ton of more free-throws. Ben’s worth to the Pistons was never so much in his rebounding abilities (though, I’m not saying it wasn’t worth something), but rather his ability to play awesome man to man defense without fouling.
See, when I look at teams efficiency differentials, I don’t see the direct effect of adding an above average rebounder over an average rebounder being anywhere near adding an above average scorer or overall defender or distributor. Sure, if you can add a catch-all guy like Pau or Odom, it’s superb, but when you add a catch-all guy like David Lee you won’t see the expected returns that individual WP48 would expect at the team level.
I believe there’s a very specific reason why guys like Camby and Lee and others like them have had to little team success in their NBA careers. Camby in particular since becoming a starter in this league hasn’t been able to get what, above a 7 seed in this league? If he truly has been one of the best players in the game, I don’t see how that is possible over 10 years.
StL Reflections
12/08/2010
This seems spot on to me. We can measure how much better a shooter someone is, because we know what percentage they shot. We don’t know how many additional rebounds someone actually added to their team, but the evidence suggests it is a LOT less than a one to one ratio between rebounds and value.
bduran
12/09/2010
Of course using specific examples is a poor way to evaluate a statistic. Especially when you’re trying to critique it. Obviously no stat is a perfect model and so it will overvalue or undervalue some players and we should be able to find exmaples of these. However, in general, it should be accurate.
How long did Garnett play at Minnesota or Pau at Memphis before moving on to championships teams?
StL Reflections
12/11/2010
True,
but if you find consistent pattern of overvaluing, then its worth noting, and seeing if there are ways to improve results. WoW seems to overvalue big men who don’t shoot (Camby, Wallace, Rodman, Evans, Lee, Love to a certain extent) people of that nature, which doesn’t mean that WoW isn’t the best stat out there-I think Hollinger at least definitely over values volume shooting-but that its worth knowing where its likely to overvalue players and teams (like Golden State this year).
Man of Steele
12/08/2010
Arturo,
I’d like to comment on the same phrase, but from a different angle.
“At the end of the day we can only credit the player who undertakes the action for the effect.”
I am in agreement with this statement on the face of it. The problem that Dr. Berri’s article does not address, but that yours brings up, is that WP actually credit player for more than the action they take. Ever since Wages of Wins, rebounds have been counted as a “possession” statistic. Unfortunately, it is inaccurate to say that the player who retrieves a defensive rebound is solely responsible for giving his team possession of the ball. Other players may have defended the shooter, and other players may have made a key box-out on the very same shot. On other shots, two to four players from the defending team may have been able to retrieve the defensive rebound. Crediting the rebounder for creating a possession in these instances is crediting him for something more than the action he undertook.
some dude
12/08/2010
exactly. Pau and Odom are the lakers 2 best rebounders last season, yet the Bynum-Pau and Bynum-Odom combo were both better rebounding combos (with the same starting 3 of Fish, Kobe, Artest) than the Pau-Odom combo.
Not only that, but Pau and Odom had the two highest WP48 on the team (and much higher than Bynum), yet again the Bynum-Pau combo again was the best and it wasn’t close.
Pau-Bynum combo +8 eff diff
Odom-Bynum combo + 5 eff diff
Odom-Pau combo +4 eff diff
According to WP48, odom should be starting and playing more than Bynum, but the team stats are all better with bynum playing over Odom with the main 3 last season.
Farmar in place of Fisher also yielded better results with Odom-Bynum than Odom-Gasol. I wish I could comment on Farmar with Gasol-Bynum but between my recollection and the statistics, it appears Phil never played Farmar a single minute with those 2 in at the same time, something that always perplexed me.
Guy
12/08/2010
SD: Have you been banned over at WOW? What was your offense?
some dude
12/09/2010
I will leave that answer out for the time being, in fairness to Mr. Berri, but I want to make it clear that I did not curse, insult anyone directly or indirectly, violate any sort of rule that may exist, troll, or spam, and I only addressed the data in the blog post and posed issues/questions.
Westy
12/08/2010
Good post, Arturo. But, I would note, I agree with Man of Steele in observing that a DR (and a possession gained), for instance, could [is] be caused by multiple parties.
some dude
12/09/2010
to make a sum of what I had posted there, I’ll put it here quickly.
If the elasticity of rebounds in WP48 at the individual level is 3.2%, what does that actually translate to at the team level?
If we have 2 seasons of the same 10 players where the PF in year 2 increases by 1% in rebounding and everything else besides rebounding is held constant (efg%, TO%, etc), that PF has a 3.2% increase in WP48 but some of that comes at the expense of WP48 of his teammates. So how much is the total effect on the team WP numbers? If we all agree there are diminishing returns, then an increase of WP48 of 3.2% leads to a team WP increase of….?
Guy
12/09/2010
sd: The answer, according to Berri’s research, is that for every Win Produced a player adds, the team loses .3 wins from other players. Berri’s study finds that for every extra win produced by a player’s teammates, he loses .3 wins. Let’s say a .300 player joins a team of average players. Each of the other four guys now has teammates who deliver .600 WP, a gain of .200. So they decline by .200 * .3 = .o6. So the team thought it was adding .200 by bringing in this .300 players, but it only got .200 – .060 = .140 after paying the 30% Diminishing Returns tax.
Dr. Berri describes this 30% effect (a loss of 5 wins in this example) as being “small.” One wonders what a large diminishing return effect might look like. (This is all based on Berri’s research — the actual DR rate is likely higher, especially for big rebounders.)
bduran
12/09/2010
Is that actually how it works? Suppose this player moved from a team where is team mates had also averaged .1 Now we know great player make their team mates less producitve so those gusy are actually better than .1, which means better than his new teammates. So you would expect a small increase in the .300 player when he moves to his new team, right?
Xavier Q
12/09/2010
My understanding was that it was 3%, not .3 (or 30%). So the team in the example was actually losing .006 for bringing on a .300 player, which is “small”.
Guy
12/09/2010
Dr. Berri: “This study – across 30 years of data – indicated that teammate WP48 had a statistically significant and negative impact on player performance. The coefficient on this factor was -0.300. ” (But again, this is likely to be a conservative estimate.)
EntityAbyss
12/09/2010
So yea, everybody guess what I did? Well, I don’t have microsoft excel or the know-how to do the stuff arturo does, but I decided to take what I know and can do to find out in a single example, the effects of diminishing returns.
As I thought about which example I should use, I figured I’d use the 93-94 spurs. Word around town was that Dennis Rodman took away rebounds, but Arturo and Dberri claim that it’s effect is real but appears small. So although, it is only 1 example, it took a lot of time for me, and I found the results rather interesting.
A couple of things. The spurs had a rebound differential of -9 in the 92-93 season and then a rebound differential of +544 in the 93-94 season. Hmm…
So what I decided to do was find out the effects of diminishing returns on all the players from the year before to that year. First I had to find out who was on the team the year before that wasn’t on the team in the 93-94 season (williams bedford, sean elliot, sidney green, avery johnson, sam mack, larry smith, matt othick, and david wood). This information wasn’t important to what I was doing, but at the time, I didn’t realize that. Then I found out who wasn’t on the team in the previous year that was on the team the following year (sleepy floyd, jack haley, negele knight, chuck levitt, chris whitney (rookie) and everyone’s favorite player dennis rodman).
Now, the key thing that I was concerned about was pace. I wanted to make sure I pace-adjusted. Why? It’s because when you do not pace-adjust, numbers can come out telling a different story. That was the key. adjusting for pace. The San Antonio Spurs had 95.4 possessions per game in the 92-93 season (19th fastest), but had a very slow 90.1 possessions per game in the 93-94 season (last in pace).
Next I found out every players pace adjusted (per 100 possessions) rebounds per 36 minutes the previous year (chris whitney was a rookie so I left him alone). For the returning spurs players i adjusted their 95.4 possessions to 100 and for the new players (besides whitney), I just adjusted for pace of the previous teams that they played for. Also, Chuck Nevitt only played 1 minute, so I didn’t include him. So here were my results.
Player –pace-adjusted per 36 — pace adjusted season totals
New Players
sleepy floyd — 3.8 — 91
jack haley — 9.4 — 103
negele knight — 2.6 — 164
dennis rodman — 18.1 — 1215
chris whitney — 3.4 — 32
Returning Players
willie anderson — 3.9 — 60
antoine carr — 7.5 — 407
terry cummings — 9.4 — 20
lloyd daniels — 5.1 — 226
vinny del negro — 4.0 — 171
dale ellis — 4.3 — 327
jr reid — 9.1 — 405
david robinson — 11.2 — 1002
Those look pretty. Anyways, Afterwards I took the per 36 rebound and entire season totals from the previous year and using the minute allocation, I checked to see how many rebounds each player would be expected to produce and then compared it to what they actually did. The numbers above were pace adjusted to 100 possessions per 48 minutes, so also adjusted the 93-94 rebound totals to 100 possessions per 48 minutes. The spurs had played 90.1 possessions per 48 minutes. I also didn’t include Antoine Carr since he only played Antoine Carr since he only played 34 games and that could be a major reason why his production wasn’t where it usually was. here are my results.
PA36 = pace-adjusted rebounds per 36 minutes
PE = Player expected pace adjusted rebound totals based on minutes, pace, and maintaining production from the previous year
ACT = Player actual pace adjusted rebound totals
Player –92-93 PA36 –93-94 PA36 –93-94PE –93-94ACT
sleepy floyd — 3.8 — 3.8 — 70 — 70
jack haley — 9.4 — 10.2 — 22 — 24
negele knight — 2.6 — 2.9 — 93 — 103
dennis rodman — 18.1 — 18.3 — 1354 — 1367
chris whitney — 3.4 — 3.4 — 29 — 29
willie anderson — 3.9 — 3.9 — 243 — 242
terry cummings — 9.4 — 10.4 –267 — 297
lloyd daniels — 5.1 — 4.6 — 125 — 111
vinny del negro — 4.0 — 3.3 — 195 — 161
dale ellis — 4.3 — 3.9 — 279 — 255
jr reid — 9.1 — 6.5 — 306 — 220
david robinson — 11.2 — 10.5 — 908 — 855
So there you have. The biggest surprise for me at first was david robinson. When I first saw the big difference in his rebounding numbers from 92-93 to 93-94, I thought maybe diminishing returns had a huge effect on him until I adjusted for pace. Some players rebounded better and some rebounded worse with the only big difference from the previous year being jr reid. The next year however, he averaged 9 rebounds per 36 minutes (rodman was still on the spurs), and continued to keep it up for his career. In my opinion, It’s most likely just a bad year, which on occasion does happen.
Using the previous years production (also adjusting for pace), the average difference in players’ rebounding numbers from the previous year was .22 and all of that could be explained by the big decrease in jr reid’s numbers. Not including him, the average difference is 0.
Also, the totals numbers of rebounds (pace adjusted) expected from their previous year would lead to 3891 and they got 3734. That’s a difference of 157 rebounds. Once again, a major part of that is jr reid. Not including him their expected would be 3585 and the team, not including reid, got 3514. That’s a difference of 71 rebounds.
Looking at the numbers, you can make the argument that the diminishing returns are in fact (in this example) very small, and they don’t seem to be triggered by rodman. Other factors could be involved (age, or other stuff). Anyways though, If I had more time, I do other people (maybe camby), but at least in this example, it is seen, that diminishing returns do occur, but they are in fact very small.
If there was a comment of the week (or month), I’d deserve it.
EvanZ
12/09/2010
Look at Robinson’s DRB% the year that Rodman arrived. It went down 5 points from the previous season from 24.8 to 20.3. The next season when Rodman played half as many minutes, Robinson’s DREB% went back up to 22.6 (magically recovering half the previous dip). The next season when Rodman was not there anymore, guess what happened to Robinson’s DRB%? It went back up to 24.2%.
Guy
12/09/2010
That’s a lot of work. Unfortunately, with all the player changes, it’s hard to know what you’ve discovered. Fortunately, there’s a very simple way to find out if Rodman was afflicted by DR. Here’s what you do: find out how many rebounds Rodman added for his teams according to WP: (Reb48-PositionAverageReb48) * MP/48. For PF, average is 11.4. For Rodman, WP’s estimate will be several thousand. The question then is: how many rebounds above average were his teams? For each season, simply take Reb – .5 * (Reb + OppReb), and add them up.
Now compare your two numbers. If the two numbers are in the same general ballpark, then you can say that Rodman delivered the rebounds estimated by WP. After all, over a long career with many different teams and countless teammates, Rodman probably played alongside players who were average rebounders overall (for their positions, of course). But you won’t find that. You will find that the total rebound advantage for Rodman’s teams is less than half of what Rodman ostensibly contributed. So then you have to decide which to believe: 1) Rodman did what WP says, but had the misfortune to to play with very bad rebounders every year of his career, in many cities, or 2) WP overestimates players’ rebound contribution. Those are the only two possibilities — one of them must be true.
And then repeat this for any great rebounder you want. You will find the same thing. Which is how we know, beyond any doubt, that there are large diminishing returns on rebounds.
Gabe
12/09/2010
Guy,
I think I’m a pretty good basketball player. If I play at my local park or YMCA, with an average group of teammates and opponents, I’m probably the equivalent of a .150 to .200 player. Now let’s say instead of regular teammates I was playing alongside four NBA players, but the other team was still just regular guys off the street. My team would win, probably 21-0. And I most likely would not have scored a single point or recorded a single rebound.
WP would have described my production as having had little effect on my team winning, I would receive some credit for the team defensive performance, but because my NBA player teammates scored all the points and got all the rebounds they would appropriately be given the vast bulk of the credit.
WP can’t make a determination on what would have happened if I’d been playing on a team with just regular players, because that is not what happened. WP is not making a determination on how “good” I am, it is simply giving credit to the players who produced the win.
Guy
12/09/2010
Gabe: in what way is your comment responsive to what I said? Or did you mean to reply to someone else?
Gabe
12/09/2010
“…you have to decide which to believe: 1) Rodman did what WP says, but had the misfortune to to play with very *bad* rebounders every year of his career, in many cities, or 2) WP overestimates players’ rebound contribution. Those are the only two possibilities — one of them must be true.”
Guy, I hoped that my little story was a simple way to illustrate how WP is limited by what actually happens on the court. You believe WP should intuit that certain players rebound contributions can be relatively easily matched/replaced by the other players on their team all rebounding a little more. But how can this be applied logically?
To go back to my story–> On a team with four NBA players, let’s say I had zero rebounds, zero assists, zero steals, and was 0-1 on fga’s, and my team still won 21-0. My WP would say that I didn’t contribute much of anything. Now if I was playing on a regular team instead, *I* would produce significantly more, but that does not automatically mean that *someone else* would produce more. What if the four NBA players were playing with a person who had never played basketball before? It’s possible that player would produce the exact same stat line as me, and the team with NBA players would still probably win 21-0.
How can I be given an extra share of the credit (I’m not a bad player, I’m just having all the rebounds and points be stolen by my NBA teammates), while withholding that same credit from the total novice who would not improve at all if playing on a team with non-NBA players?
Back to the NBA, yes diminishing returns clearly exists, maybe more so with a statistical outlier like Rodman, but what you appear to propose be done to improve WP seems like it is necessarily somewhat arbitrary.
Guy
12/09/2010
Fixing Wins Produced would be a substantial undertaking, for sure. I don’t think it’s that hard to deal with rebounds, as you basically credit players for rebounds above/below average using something like the Hollinger coefficients (.3/.7). You’d need to make a small team adjustment to make everything “add up,” but that wouldn’t be a big deal. (I couldn’t follow why Alex thinks this would be so enormously challenging.) But then you also have to properly value efficient shooting, which WP doesn’t do well at all. (I know that’s supposed to be a strength of WP, but in fact even PER has a better correlation with eFG% than WP48 does.) Related to that is giving proper credit to high-usage players, who help elevate the eFG% of their teammates. And I’m not sure about the value WP places on assists — that’s one of the most difficult conceptual issues I think.
So, it’s a huge project. Probably easier to just use Win Shares, or Statistical Plus Minus. :>)
Zach
12/09/2010
Guy, I’m confused as to why you think that just adding in a random team adjustment would somehow be better than giving the credit to the rebounder? It seems to me like this would make the problem worse, not better.
Guy
12/09/2010
Zach: it wouldn’t be a very large adjustment in most cases. Using something like the Hollinger coefficients will probably get you within +/-100 rebounds in most cases. You could allocate those based on each player’s share of the team’s rebounds, or something like that. I’d guess whatever approach you took would have a minimal impact on player rankings.
Zach
12/09/2010
Right, so if it has a minimal impact then why should it be done? Also, what evidence do we have that assigning those 100 rebounds in any other fashion would somehow make the individual player rankings more, rather than less accurate?
It seems to me like the point has been reached where everyone, including Dr. Berri, acknowledges some diminishing returns in rebounds, as well as in almost all other statistics. I also think he has written that it is important for someone building a team to understand how the pieces all fit together, and not just blindly get a bunch of higher WP guys who don’t necessarily complement one another. So, unless someone has developed a way to better assign those remaining 100 team rebounds, and can actually prove that it is better through the use of evidence, I don’t really see what the disagreement is still about?
Guy
12/09/2010
Zach: you’re mixing up two things. Changing the coefficients has a very large impact on the rating of many players. Players can gain or lose as much as 30% of their currently-estimated WP48. If you also want your player ratings to add to the right team total, that would require a fairly small additional allocation of rebounds that aren’t accounted for by the new coefficients. It’ s the latter I was talking about.
Zach
12/09/2010
I get what you are saying. I’ll try to explain my point more clearly. Let ‘s say we agree that the coefficients should be changed and now we are left with 100 extra rebounds that must be assigned out to players. What evidence do you have that assigning them out evenly to all the players would be a better way of giving credit for them as opposed to just giving it to the player who actually got the rebound?
I understand that WP is not perfect, and it would be nice to know exactly who to give credit for those 100 rebounds. However, I think you are making a big mistake by just assuming that we can assign them equally and that would somehow be better. Until we have a new statistic to work with, or some evidence based way of treating those extra 100 rebounds, I don’t get why you think it’s better to randomly assign value to players who may or may not have had anything to do with the stop and/or the rebound.
Guy
12/09/2010
Zach: How do you know which 100 rebounds to give to the players who “got them?” I don’t see how that works in practice. My suggestion is to allocate them based on players’ share of rebounds. But again, it won’t really matter. If you believe the coefficients should change, then the accuracy gains from that change will dwarf any inaccuracy caused by this final allocation; if you think the current coefficient (1) is correct, well then obviously you shouldn’t change it.
The underlying point is that in situations where we don’t know definitively who contributed productivity, it is in fact better to distribute it evenly than to assign it overwhelmingly to the wrong player. The first choice will — on average — result in more accuracy. It’s better to be a little wrong on lots of players on a team than massively wrong on one (Camby, Evans, etc.).
And look, no method will meet the standard of “proof” that each player got exactly the right credit. Productivity in the NBA is generated by teams — the only true statistical “facts” are what the team does. Every player metric represents only an ESTIMATE of players’ individual contribution to that total performance — that’s the best we can ever do.
bduran
12/09/2010
My understanding is that he doesn’t think that it would be challenging, just not desirable. It seems like it might make sense, do to the diminishing returns factor for every box score stat, to have a larger team component in general. Not sure.
Also, is it surprising that a metric that so overvalues points scored correlates better with a scoring metric?
I’m not a huge fan of stat +-. Do you know what the correlation between win shares and WP is?
Devin
12/11/2010
I just want to say that those were amazing posts Gabe.
EvanZ
12/09/2010
Your statement implies that WP is not useful as a player metric, but rather serves simply as a box score summary.
Man of Steele
12/09/2010
Guy, those are not the only two options. A third, and I would argue more likely option, is that Rodman has received credit for retrieving defensive rebounds, and thus creating possessions, which were not solely due to his play.
By the way, it is not entirely accurate to call this “stealing” rebounds from teammates. Having Dennis Rodman allows the Spurs guards to get back on defense or get out on the fast break more quickly because they have less responsibility to retrieve rebounds. The big rebounder is not “stealing” rebounds, he is merely beig assigned credit by WP for actions for which he is not responsible. Let me be clear: it is only accurate to say Rodman steals rebounds from his teammates from the perspective of WP; in reality his teammates give him more marginal rebounds, which is a positive for the team (due in large part to Rodman’s rebounding prowess, although not entirely since the decisions by Rodman’s teammates to “give” Rodman rebounds are subjective).
Guy
12/09/2010
MOS: I wouldn’t say you identified a third option, I’d say you restated my option #2 (WP overestimates players’ rebound contribution) but said it better than I did. Obviously, Rodman did physically obtain the rebounds that Reb48 records. The question is how much credit to give Rodman vs. his teammates.
The bottom line question we want to answer is this: if SAS had an average PF instead of Rodman, how many fewer rebounds would the team have? The answer from WP is 660 rebounds. That means that Rodman’s teammates were -388 rebounds below average as a group, almost 5 rebounds per game. Does anyone find that plausible? And if you do, are you also ready to believe that Rodman’s teammates were hundreds of rebounds below average EVERY year of his career, no matter where he played? So I’m surprised EA chose this example, because it’s really a fantastic illustration of DR. WP thinks that Rodman contributed 660 rebounds above average, but as EA noted SAS improved by only 277 rebounds when Rodman arrived (from -5 to +272) . Rodman delivered about 40% of what WP estimates, which is about what he did over his entire career. Where are the missing 383 rebound? If it’s not DR, where did they go?
Jon
12/09/2010
Actually, Arturo, WP seems to do a pretty terrible job at predicting wins on a team level. When i first started reading about WP this offseason, it seemed to me highly flawed for many of the commonly-cited reasons. However, i was somewhat apprehensive about being wrong, given the conviction that obviously intelligent people like you and Dr Berri have about it. I’m completely unconvinced of its worth when looking backwards (sure the team wins add up correctly but in my opinion they are badly misallocated among the players). So i decided to track NBA win predictions from a variety of sources this year. WoW blogs like your own, basketball reference, basketball prospectus, Las Vegas, ESPN, Hollinger, Simmons, etc. And although its still fairly early, the WoW predictions have been horrendous. In fact they fare worse than simply repeating last year’s win totals as predictions on most measures!! Wouldn’t you know the top 2 spots right now in terms of both correlation between predicted and actual, and sum of absolute value error, are Bill Simmons and the ESPN aggregate predictions. And 3rd or 4th depending on the stat is the Las Vegas lines. In reality, if you’re model can’t outperform Las Vegas, its not of much value. But if it can’t outperform repeating last year’s records??? Time to throw it out the window i would say.
arturogalletti
12/09/2010
Jon,
Fair criticism. Again, small samples. I feel fairly confident that if we did aggregate from the smackdown vs aggreagate from ESPN we’d be ok. I’ll break it out as a post.
EvanZ
12/09/2010
Arturo, in all fairness you had a post a few weeks ago that argued 20 games into the season we should be fairly confident that results are meaningful.
bduran
12/09/2010
EvanZ, does this situation change much if you compare the prediction to differential instead of W-L? Seems like there may be a fair amount of variation in W-L ratio compared to efficiency differential this early in the season.
bduran
12/09/2010
NM, read your blog 🙂
Guy
12/09/2010
Arturo: Sample size doesn’t need to be a limitation for you with the great data sets you and nerdnumbers have assembled. You can go back and “retrodict” past seasons, and tell us whether WP48 does a better job than PER, prior season team wins, in predicting future team wins. It would be especially valuable to group teams by the level of personnel continuity they have, so you could look at those teams with the most player turnover — those are the most important. I’d wager you can predict future team wins as well as WP48 does just by using something as simple as Points per minute.
EvanZ
12/09/2010
Guy, I’m pretty sure they did this at some point, and the correlation from year to year for a player switching teams was something like 0.7 instead of 0.8. Somebody can correct me if I have that wrong.
arturogalletti
12/09/2010
I have done the retrodiction for a couple of seasons on a game to game basis. Just simple rolling WP & MP averages for last x games let me call about 80% of all game results (W-L)
bduran
12/09/2010
If you use the last x games and those games are all within the same season, isn’t that essentially the same as using data for the whole team, which is essentially efficiency differential?
arturogalletti
12/10/2010
Not really. There’s a tremendous amount of variation in minute allocation on a game to game basis over the course of the season due to coaching and injuries. If the value allotment is correct then your predictive value holds ( and if not it goes in the crapper). So if your player value allocation is on point looking at projected player productivity times actual value delivering opportunity (minutes played) should be very accurate at predicting results. So if the difference in team performance as a result of Player A missing time and his minutes going to player B is predictable based on current performance levels (I’m taking season to season prediction out as a variable, one problem at a time 🙂 then we have something in terms of spreading value around. This is the focus of my current research.
If the win values and std deviation for all players are right then historical data will confirm it.
Guy
12/10/2010
Arturo: This is just backwards. Using games from the past few weeks to predict today’s game absolutely minimizes the amount of personnel variation. I literally can’t think of an analytic approach that would do less to produce the variation you need. And of course all the players are on the same team, so share the team defensive adjustment. It’s success in predicting performance one or two years in the future that will answer the important questions.
arturogalletti
12/10/2010
Guy,
This is not the only thing I’m looking at but game to game gives me a lot of data to look at. Injuries and roster shakeups are key for this (value allocation and what affects it). So if Carmelo gets hurt and is replaced, I can study what the predicted and actual effect was.
Predictive over time is something I feel good about at this point (i.e, I have something that does a fairly decent job it just needs improvement). I’m more concerned about factors that affect player performance (teammates for example and Homecourt) in the short term and how to account for those.
Guy
12/09/2010
Interesting. But it’s hard to evaluate 80% success. How often does the favorite win in the NBA? (Just picking the home team gets you to 60%.) So I think you have to compare it to other metrics to know if 80% is good, bad, or in between.
Predicting wins for teams with significant roster changes is also a good test. Intact teams will tend to perform the same, and so doesn’t necessarily tell us a lot about how well metrics are dividing up credit among players.
EvanZ
12/09/2010
What does the retrodiction get you?
Guy
12/10/2010
Predicting futue wins on teams with personnel changes will, I think, tell us how well metrics are measuring current player production. Obviously, players’ individual production will vary by season, but all metrics will face that same challenge. The metrics with the most “signal” (measure of player’s real productivity) should do the best predicting.
Evanz
12/10/2010
lol
Guy, my question was literal. I wanted to know what the result of his retrodiction was.
Guy
12/10/2010
Funny.
I do agree with bduran: there’s no point in predicting games based on last 10 games. The whole point of the exercise is to maximize the change in personnel, and Arturo’s method ensures almost no change. And we already know that team point differential (aka WP48) is good predictor of wins/losses.
EvanZ
12/10/2010
I think I didn’t fully understand what Arturo was saying. I thought his first statement was that he did retrodiction for a couple of seasons – i.e. predicting the next season with the previous season’s data. Apparently, that is not what he did. But those are the data we all want to see. Right?
Jon
12/09/2010
Arturo,
I’ve collected most of the predictions from the smackdown and they are almost all similarly bad (below the performance of last year’s records). There is one doing a bit better than the rest (RoBlog) but still well below Vegas, ESPN, Hollinger, and Simmons. Most of the WoW blogs tended to be higher on GS than others, higher on NJ than others, higher on Port than others, higher on Sac than others, etc.
EvanZ
12/09/2010
yeah, I reported similar findings a few days ago atmy blog.
Jon
12/09/2010
If it makes you feel better the one model/prediction set doing worse than everyone (even WoW) is Kevin Pelton’s model SCHOENE. It had Atlanta with a losing record, Boston near .500, Cleveland at .500, the Lakers at .550, and NJ better than Utah! I’m impressed that he actually published those results rather than just scrapping it on the spot.
arturogalletti
12/10/2010
Jon,
I have no problem with the fact that we may have not gotten it right the first time. There were a huge number of variables to consider. That said, I kinda want to hold my evaluation until the full year is in.
entityabyss
12/09/2010
Wait, I’m a little bothered. I took a long time to do what I wrote in my comment way above. So for Guy or anyone else, I did find out if rodman increased their rebounds by the amount WP would suggest. The answer is yes.
As you can see above I adjusted for pace and minutes and I added up each player’s rebounding totals from the year before (what they’d be expected to do) and added it together and compared to the actual result. I don’t know what you’re not seeing. If they were a “bad” rebounding team, rodman made a HUGE difference. As the numbers I wrote show, he had a VERY SMALL impact on his teammates’ rebounding numbers. READ it. It answers the question. In the example, diminishing returns does EXIST, but it is in fact VERY SMALL. I don’t know what you’re not getting. Please let me know what I didn’t write.
Also, that’s the comment of the month. I might do one for camby. Also, arturo, doesn’t my last comment prove your point? Why is it going unnoticed?
entityabyss
12/09/2010
Also (please READ my post on diminishing returns), Guy, you confuse me. The year before rodman came, the spurs had a -9 rebound margin for the year. That’s around -0.1 per game. The next year, they had a rebound margin of +544 for the year, and at the SLOWEST PACE IN THE LEAGUE. They went up to +6.6 a game. Average rebound differential is 0, so they were well above average and went from below average.
I am unbelievable frustrated. I adjusted for EVERYTHING. Getting rodman added a rebound DIFFERENTIAL (more important than just rebound totals) of +553. They had the SLOWEST PACE. I adjusted for all the players’ previous season pace, and I adjusted for the minutes they would play. I put the numbers together and compared it to the actuals.
The result – A VERY SMALL DIMINISHING RETURN. Did you read what I wrote or just skim through. I really think people either didn’t read what I wrote, or read it not very carefully. It’s so annoying. LOOK at the numbers. READ the comment. THEY DID WHAT WP SUGGESTED THEY WOULD. Yea there were some players whose numbers went down, but there some payers that went up. In the end, the effect to the team expected rebound totals was VERY SMALL.
How is it, the numbers are in your face and you ignore them? They did what they were supposed to do.
Guy
12/09/2010
EA: Four players on SAS played at least 1000 minutes both seasons, giving us a reasonably reliable read on their performance: Robinson, Ellis, DelNegro, and Reid. You could have saved youself a lot of work by doing what EvanZ suggests and using Reb% (the percentage of all available rebounds a player obtains) — that adjusts for pace, number of missed shots, everything. Now, every one of these four players saw his reb% fall significantly, by an average of 22%. Now, I would never claim that one anecdote like this proves anything by itself, especially since you have a lot of other players leaving and arriving to complicate the story. But if you want to use this one example, the story seems to be that Rodman reduces other players’ reb% by about one-fifth. And that happens to be consistent with my estimate that Rodman adds about 40% as many rebounds as WP estimates (which is better than most high-rebound players, because Rodman got a ton of Oreb).
Also, the WP estimate is that Rodman should add 660 rebounds to a team (assuming he replaced an average PF). Since SAS only improved by 277 rebounds, that’s a further strong indication of diminishing returns.
Camby, BTW, is the poster child for diminishing returns. Perhaps the player most overrated by WP.
Benjamin
12/09/2010
Arturo,
I haven’t posted before but I’ve read a lot on this site leading up to this season. I want to start by saying thanks not just for putting all the time into thinking about quantifying basketball, which is obviously a labor of love, but also sharing those thoughts with the world which takes another big helping of hard work and can be frustrating at times.
I hope there is some value with my sharing my perspective on this debate.
When I first Dr. Berri’s site his wins produced model appealed to me because it does seem that most quantitative analysis of basketball overemphasized scoring to this point. I found Dr. Berri’s writing compelling and intutive. So when Dr. Berri linked over to your site and I was able to follow day in and day out you work putting the model to work to make predictions and find intriguing nuggets of info, I was thrilled. I didn’t do a lot of thinking about the ins and outs of what you were saying because like Dr. Berri’s I found your writing authorative and intuitive.
When the season started and you started comparing the predictive success of the various models a concern creeped into my thinking. This simmons guy (who I read and listen to frequently) was doing pretty good. This made me think, if simmons, a guy who spends a considerable amount of time analyzing other sports, could do this well, it’s not inconcievable that someone who is paid to analyze basketball could do as well or perhaps better than the model without all the quant stuff. Maybe it wasn’t time to fire all the GM’s yet 😉
I began to think about what I did and didn’t like about the model.
* On the offensive side of the ball I love the emphasis on effectiveness over volume and in particular how much turnovers are penalized.
* I’m not entirely thrilled about the value of assists. I don’t like ’em because their recording is subjective so if you play more games for a generous stat crew you do better. And as a basketball player I’ve been in plenty of situations where the shooter does more of the work (i.e. choosing when and where to cut, coming of a screen correctly, etc.) that leads to the basket then an assister and it seems odd to get the same credit for that play as one where there is some real passing magic.
* On the other side of the ball I had some reservations about defensive rebounds not taking into account the defensive effort that lead to the rebound
When guy and some dude started beating the rebounding drum I thought, great we’re going to get somewhere on one of the areas I though the model might need som refinement. Unfortunately it doesn’t seem that way. In all fairness I haven’t read either of Dr. Berri’s books. But it seems to me that rather than address Guy/Some dude’s specific criticisms you have both tended to point back to previous work or generate new data that looks synonmous with old data. In other words I don’t feel like you’ve said, what if we were wrong, how we would go about proving that based on these criticisms. Instead you’ve said look at how self consistent our data is, and, here is more data that demonstrates consistency. As an engineer I know how easy it is to generate self consistent data and still miss something huge so I haven’t found this especially convincing. On the other hand, as this has dragged on I think guy has taken the less useful role of being a critic rather than a member of the community. As a critic I don’t think someone has to offer an alternative, only demonstrate flaws. But as a member of a community, I think you have a responsibility to follow up flaws with at least attempts at solutions.
entityabyss
12/09/2010
Guy, I don’t see this 1/5 decrease in rebounds. Of the four players you included, the big reduction in rebounds was mostly due to the decrease in reid’s rebounding rather than the rest. Also, you fail to include the rest of players, who the majority of saw no decrease or rather an increase in rebound numbers.
The team as a whole when you factor in pace did not see much less rebounds than would be expected if you just added their rebound numbers from the previous. Also, reid’s rebounding numbers increased to the norm the following year.
Also, your 277 rebound increase number is misleading. The reason being that they played a 5 possessions slower game and also, you didn’t include the decrease in rebounds of the opposing team. They went from a team with a rebounding differential of -9 playing 95.4 possessions a game to a team that had a rebound differential of +544. That’s an increase of +553 before you account for pace. After pace adjusting, you see around a 10% increase since they played the slowest pace. You must’ve looked at the increase in rebounds rather than the increase in rebound differential.
Guy
12/09/2010
EA: As for the one-fifth reduction for returning players, that is the average decline in the four players’ reb%. Check the numbers at BBR if you don’t believe me. These players all obtained a significantly smaller share of the available rebounds after Rodman arrived. If you check Rodman’s other transitions (there are a lot) you’ll find the same general pattern (with some exceptions, I’m sure). I’ll do the Bulls ’96 for you (change in reb% from ’95 — change in rate, not a % change):
Longley -4.2%
Pippen -2.3%
Wennington -2.2%
Kukoc -1.1
Kerr -.5
Harper -.3
Six players, and every one saw their reb% fall. Average decline: again about 1/5.
We’ve already discussed the differential issue at great length, and I’m afraid you are simply mistaken about it. The currency here is rebounds, and each one is worth 1 point or .033 wins. You don’t need to also count the rebounds denied to the opponent, because every rebound obtained is a rebound denied to the other team BY DEFINITION and the .033 win value already incorporates that. There just isn’t any debate about that, and I don’t know how to explain it any better. Maybe someone else can do better. So it is not misleading to say the Spurs improved by 277 rebounds. That’s a very impressive gain, by the way — I would guess one of the largest of the past 20 years — worth about 9 wins. And I’m sure Rodman deserves most of the credit.
entityabyss
12/09/2010
Benjamin, this is for you. I was thinking that maybe the rebounding thing was a problem. I looked into it, and arturo and dberri are right. Diminishing returns don’t have a big effect. The studies that suggested otherwise tend to include changes in a team’s rebounding rather than differential.
When a team gets a rebound, the other team doesn’t. So if there’s a 100 misses, and the average teams gets 50 rebounds, the opposing teams gets 50 as well. However, if a team would get 55 rebounds, the change is +5, but that doesn’t account for the decrease in the opponent’s rebounds. The opponent gets a -5. So when your team gets 55 out of 100 rebounds, it’s +10 because the opponent will get 45. What I noticed is, the people that argue for huge diminishing returns don’t seem to account for this.
Now, instead of arguing that, I decided to see for myself. If you look waaay above at my first comment on this post, I talk about the 93-94 spurs and the effects of diminishing returns. I adjusted for pace (something that isn’t often done) and the results show that at the team level, there weren’t much diminishing returns. Please read it, and if you’re confused about anything, I’ll explain.
As for Dberri, he promotes his work, because he believes that they do answer the questions. I’m trying to do this here.
Also, you could also, get calculator, and over to basketballreference.com, check a team that just got a big rebounder and see for yourself the effects of diminishing returns. Don’t forget pace.
EvanZ
12/09/2010
EA, how do you explain the effect of Rodman’s arrival on his teammates decrease in DREB%?
EvanZ
12/09/2010
“What I noticed is, the people that argue for huge diminishing returns don’t seem to account for this.”
Guy, myself, and others have repeatedly “accounted” for your newly discovered fact by telling you to focus on REB%, which magically accounts for the thing you are apparently accusing us of not accounting.
EntityAbyss
12/10/2010
Hey Evanz, I decided to focus on REB% and I have something coming. 8)
EvanZ
12/10/2010
BTW, that 93-94 SAS team must rank as one of the best rebounding teams ever, diminishing returns or not.
0.80 DREB% is incredible. The previous year it was 0.7. So, Rodman’s marginal value was very impressive. That was also a very stellar defensive team.
some dude
12/10/2010
I am getting a 70% DREB% for that team. 2597 DREB against 1089 OREB on B-R.com.
Where are you getting 80%? Different season or are my eyes lying to me?
Evanz
12/10/2010
lol, fat fingers I guess!
Well, that’s diminishing returms on my brain late at night.
some dude
12/10/2010
in fact, it looks like Rodman’s contributions were on offensive rebounding, not really defensive. The DREB rate before and with Rodman was largely unchanged, but the OREB increased quite a bit.
Evanz
12/10/2010
which makes sense, as diminishing returns are less for OR.
some dude
12/10/2010
true, but curiously enough even though almost all the gains were in offensive rebounding (260+ more!), their offensive efficiency only rose by 0.8. An increase of nearly 4 Oreb per game only added just under a point of offensive efficiency. hmm..
The culprit seems to be the drop in FG%. Which would go with the argument that a poor offensive player like Rodman makes it tougher on the other players to score as efficiently. Then again, with most of the better players unaffected, perhaps not. Lloyd Daniels sure dropped randomly (that is more than any possible Rodman effect, that’s for sure).
The other thing to note is that since Rodman didn’t add any defensive rebounding, really, what did he add? The team improved on defense and some of it was on FG% improving. But like Ben Wallace, you see a drastic reduction ins fouls and FTs with Rodman now on the team. Again, Rodman’s defensive value seems to not be as much in rebounding but rather his ability to play solid defense without fouling, which is difficult to perform.
Just my observations of the data.
Benjamin
12/10/2010
I’ve been thinking all day about what the value of rebounds by thinking about where the value of rebounds comes from. My background is in comp sci so I prefer to think inductively.
Defensive rebounds are increased in one of two ways.
– causing the opponent to miss more shots
– preventing the opponent from getting offensive rebounds
Reduce the analysis to one possession at a time and you’ll see the only way the defense gets a rebound is if the offense, shoots, misses, and does not collect and offensive rebound (which actually just extends the possession and may still lead to a defensive rebound, but I’m going to ignore that for a second)
Anyone who has played basketball on a team based level (i.e. a little more cohesion than your standard pick up environment) can tell you that both those requirements take a team effort. Reducing opponent field goal % takes good team defense, if your team cannot work well together they will get picked a part by adding any kind of structure to the opposing offense. After a miss, no matter how great rebounder you are you will have some defensive rebounds taken from you by the offense if your teammates do not box out. These two things are especially obvious if you’ve ever coached basketball.
Now the wins produced metric doesn’t include defense explicitly but I think it does a great job of including it implicitly. By valuing defensive rebounds, which are generated by good defensive technique (lowering opp fg% and boxing out, the defense half of basketball is quantified. This is extremely attractive because it would be difficult to objectively quantify defense otherwise. I don’t disagree, and I don’t think guy and some dude do either, that WP nails this at a team level. When you look at team defensive rebounding it really is telling the story about team defense.
Where I get stuck is the distribution of value of defensive rebounds amongst players on the team. Taller players who operate closer to the basket tend to get more rebounds. So the team defense work when distributed along the lines of def. reb. will tend to get distributed to the taller players who tend to operate near the basket. The crazy thing is that sometimes this works out more or less correct. The size, and defensive proficiency of a player like dwight howard has a tremendous impact even when he doesn’t actively impact a play. The guy jameer nelson is guarding doesn’t just have to think about getting by him, but how he’s going to score if he get’s to the lane with dwight lurking down there. This makes nelsons guy take less effective shots, which lowers the fg% of the team. Thus more defensive rebounds for the magic which tend to go to dwight, the player who is certainly having more than a 20% impact on defense when he is on the floor. But if you look at the heat, are things turning out ok? LBJ looks like a terrific help defender, but statistically he is getting a lot less of the defensive production than Z if you look at dreb/48. LBJ gets 6.9, Z gets 8.9.
I realize that WP actually accounts for this to an extent by using a position adjustment. I have two issues with this. The first is that using the same adjustment for all players at a position seems to regress the value of defensive rebounds to the mean. Meaning I think that Dwights defensive rebounds come closer to representing his defensive impact for that team than Z’s. When we use both players to figure the positional adjustment for centers we make Dwights defensive rebounds worth less relative to other positions than they should and we make Z’s defensive rebounds worth more relative to other positions than they should. My second issue may be from my background in comp sci, but when I come across a solution that feels inelegant, I tend to think their is a problem with the model and prefer to fix it rather than find the ‘right’ fudge.
Ultimately I think a more accurate way to reflect the individual defensive efforts of players is to look at the fg% and offensive rebound rate of the players they are guarding. Unless this is done WP for is a valuable tool that still requires some subjective unpacking to be used effectively. I think this is intuitive if you go back to the discussions on this board who would make the best 5-of team. People were generally picking all around players like barkley over players who’s production mostly came from d. rebounds.
Westy
12/13/2010
Good posts, Benjamin.