Why should I refuse a good dinner simply because I don’t understand the digestive processes involved?-Oliver Heaviside (1850-1925), when criticized for his daring use of operators before they could be justified formally
Yesterday, I introduced one of my regular season weekly features, The NBA Now Rankings Week #1. Today we introduce another new feature: Bobo strikes back. This series will track offseason NBA predicions by WOW analysts and others across the web as the season goes along.
Before we get to that, for those who can’t get enough , the weekly podcast is up. Counseling is also available.
As you may or may not know the members of the WOW network (see sidebar) have engaged in a friendly contest (Wages of Wins Network 2011 NBA Super Stat Geek Smackdown). In it each of us who entered picked the win totals for every team as well as predicted postseason awards from the media and Wins Produced (let’s get this out of the way go here for the Basics behind the math). In general, We did this by attempting to model the teams using Wins Produced to come up with a reasonable prediction for the season. At least some of us did 🙂
The reward?
We heard there might be an opening.
As part of the contest, we decided to track some well known models and personalities across the web who made offseason picks as well. We felt it was our duty to our audience and science to dispassionately analyze and report the numbers.
Potshots me? I'm just a cute kitten doing my part for science!
Before we get to it let’s provide some background.
The Contenders
Image courtesy of xkcd.com
From the WOW network of bloggers (go take a look):
- Hickory High:Predictions (Beware the Franken-Win Model!)
- Nerd Numbers (a nice recap series by Andres Alvarez on the concensus picks)
- The City: 2010-11 NBA Final Predictions: Wages of Wins Network Smackdown (Evanz with just the facts)
- NBeh?:Updates: Season and Team Projections (Junior Probationary Assistant GM of this blog lays down his picks)
- Roblog: NBA Season Preview (the architect of the smackdown, Rob than can be your wrestling name)
- Sport’s Skeptic:NBA Stats Smackdown 2011: Team Predictions and Summary (an interesting approach using the wisdom of crowds)
- Miami Heat Index (this is really good other than the insane 75 wins for the Heat):
- My own models :
- The Build (my effort to model every player and every team). There will be two versions of this one evaluated. One that accounts for strength of schedule and one that doesn’t.
- Bobo the Monkey (my take at the dumbest possible model, this is just a team’s win total for last year)
If Bobo wins, we all lose.
From across the web:
- From Basketball Reference’s set of projections. Three models:
- One based on statistical plus/minus
- One based on Win Shares (using the Simple Projection System method)
- One based on a heavily regressed-to-the-mean version of last year’s Simple Ratings (not a serious model just an experiment to see how the dumbest possible model does think Bobo the monkey)
- Basketball Prospectus SCHOENE model.
- Bill Simmons‘ picks (he put them out on his twitter feed)
- John Hollinger picks. These are paid (require ESPN insider access) so they will not be reproduced here. However, I am a paying customer of ESPN and I dutifully wrote these down and will enter them into my model evaluator.
Let’s look at the picks.
The Picks
Of note is that a few of the analyst (including a pro) did not bother to make sure the win totals added up properly. Shame on you 🙂
The Method for Evaluation
To evaluate how the predictions are doing, I take everyone’s raw wins predictions for each team and combine it with the equation for home team winning I came up with for a single game (see here for detail). To put it simply:
Probability of Home team winning a game = (Projected Wins Home Team-Projected Wins Road Team)/82 +.606
Then I worked it out for every game so far as follows:
- If W% is greater than 60% call it a strong Win
for the home team - If W% is greater than 50% but less than 60% call it a weak Win (WW) for the home team
- If W% is greater than 40% but less than 50% call it a weak Loss (WL) for the home team
- If W% is less than 40% call it a strong Loss (L) for the home team
- I then look at everyone’s hit rate for strong predictions and for all predictions.
- I rank each model/analyst for both
- I assign points based on ranking (double points for strong predictions since I value those more)
To me this is a fair test since the closer your predictions are to the actual shape of the NBA (in terms of how good team are) the better the win predictions will be
Let’s take a look at results thru 11/01/10
The Results so Far
Right now the WOW analysts are leading the pack. My two models are slugging it out with Evanz for the Lead. The bigger surprise for my models is at the bottom though were much like the Clippers making a run at the Wizards for the most lottery ball, our friends from ESPN are making a run at Bobo. I’d say more but I’d still like to get linked by ESPN in the future.
Arturo's Silly Little Stats v2.0 
nerdnumbers
11/02/2010
Man Robbie,
As the two people that worked to put this on we’re getting worked by the rest of the network! We need to at least get in front of BR and BP!
Chicago Tim
11/02/2010
You had a few typos. Most importantly, it’s SCHOENE, not SHOENE.
Fascinating results so far. I’m not surprised about Simmons, but Hollinger worse than Bobo? Shocking. But we are a short way into a long season.
arturogalletti
11/02/2010
Fixed!
nerdnumbers
11/02/2010
Hey Tim,
So Arturo is pretty much the NBA Guru today. Check out the Automated Wins Produced site and then look at Arturo’s prediction for Derrick Rose. Freaky scary!
Robbie O'Malley
11/02/2010
Yeah Andres, we’re doing terrible right now. However, it is a long long season. A few things my crystal ball predicted would happen have not happened yet. It will all fall into place. As long as I beat Hollinger, Simmons, and Bobo I feel okay though. At least I got the number of total wins right!
Guy
11/02/2010
With everyone between 70% and 76% so far, seems like not much difference yet.
You have a problem with your strong/weak classification: you call all predictions over 60% a “strong win,” but 50% of all games should be in that category since HFA is about .600 in the NBA. In contrast, a home team below 40% is a total mismatch. These categories need to be centered around .600 , not around .500, if you’re going to use them. (Though I’m not sure they add much — once your sample is bigger, the overall hit% should give you what you need.)
arturogalletti
11/02/2010
Guy,
Remember that I’m embedding the HFA in there. So a Strong Win or Loss is better than just picking the home team all the time. So far the results are very promising.
Guy
11/02/2010
Right. So two equally matched teams will yield a home win prediction of .606 — right? How is that a “strong win?” Won’t half of all games be in this category?
And why give twice as much credit for correcting calling a heavy favorite? Wouldn’t it make more sense to give extra credit for calling the tough ones? Or maybe I don’t understand what you’re doing here…..
arturogalletti
11/02/2010
Guy,
I’m looking at predictive power. Anything in the 40% to 60% range is a toss-up. 60% is typically the margin you need to beat the house consistently.
reservoirgod
11/02/2010
Hollinger & his pseudo-science has been a joke for years. I can only hope to understand how he’s managed to hold onto that job w/ ESPN.
Devin Dignam
11/02/2010
Hey, my updated picks were adjusted to add up to 1230! The predictions I submitted into the contest were “wins produced wins”, so I was happy to get them so close to 1230 (seriously, it sounds like I’m making an excuse, but…I’m in the top 4 right now, so just trust me).
Cool to see that I’m near the top. I wonder how the adjustment changes my results so far.
And of course it’s hilarious that Hollinger is so low right now.
arturogalletti
11/02/2010
I’ll get them in the next go around.
Evanz
11/02/2010
It will be interesting if I actually end up doing this well, since I’m simply averaging WP and WS (plus using a totally ad hoc aging model). WS alone isn’t doing too well.
Also, I should point out that if I had arbitrarily assigned those last 13 wins needed to get to 1230 for my picks, I know I wouldn’t be in 2nd right now. 😉
Evanz
11/02/2010
Oh, I forgot to ask. Arturo, can you do a straight up correlation, too?
arturogalletti
11/02/2010
I could but it wouldn’t tell us very much (low sample size)
Guy
11/02/2010
Wait, is “Hit Ratio (strong)” the percentage of each system’s strong picks that are correct? If so, that just rewards a system for making fewer strong picks. If a system only assigns an over 60% win probability to teams that are truly 75% or higher favorites, then it will have a terrific hit ratio but actually be doing a terrible job. (Think about it: a perfect prediction system should probably only get about 75% of the “strong” games correct, or whatever the mean % is.) What you seem to want is a ranking that rewards a system for getting a game right, and gives an extra reward if it strongly predicted that result. But now you are rewarding a system for being correct when it chooses to make a strong prediction, which is very different.
arturogalletti
11/02/2010
Guy,
I’m just trying to measure the success rate of strong buys based on projected win probability. I care more about “sure things” than marginal plays.
some dude
11/02/2010
this is cool and all, bu I want to know which model to use vs the spread!
come on, throw us gamblers a bone, here. 😀
Neal Frazier
11/03/2010
Will you be asking any blog that looses to Bobo the Monkey to leave the WOW network in shame?
arturogalletti
11/03/2010
I don’t think we’ll have to ask 🙂
Devin Dignam
11/03/2010
Oh, and by the way: where you called me “assistant GM of this blog”, you spelled assistant incorrectly.
🙂
arturogalletti
11/03/2010
I’ll fix it but just for that it’s going to be junior probationary assistant gm from now on.
🙂
Guy
11/03/2010
Arturo: can you please explain what a “hit rate” means? It sounds like you are letting each system choose its own “strong buys” and then measuring the success rate. I’m not trying to be a pain here, but that is just very wrong. Imagine you had a perfect system that predicted every team’s win% exactly right. Now, take all those predictions and regress them 50% toward the mean (a .650 team becomes a .575 team, etc.) Under your system, this new set of predictions will do MUCH better — although by definition it is really worse — because it will only issue a “strong buy” when one team is actually vastly better than the other. The trick is that the new system will issue many fewer strong buys, but will almost always be right. So at a minimum, you need to tell us how many strong buys each system generates.
Much better would be something like rewarding each system with one point for every game it calls right, with a one point bonus if it was a strong prediction. Or measure everyone against the Vegas line as common benchmark. But you can’t let the system decide how many strong buys it issues, and then just rate by success rate.
arturogalletti
11/03/2010
Guy,
Win %: (Proj. Home Team Win% – Proj. Road Team Win%) +HCA(.606)
Hit Ratio (All): % of all games picked correctly by Win % (so for all 52 games this year)
Hit Ratio (Strong): % of all games were the win % is >60% or less than 40% picked correctly (on average about 33 games per model)
I’m doing it by rank but I can do a point per game called correctly with 1 point for a strong call and minus 1 for a miss.
bduran
11/03/2010
Guy,
There are definitely other ways to rank, but I don’t have a problem with his. From a betting stand point, he’s rating recommendations. When the model says you should bet (>60% or <40%), you want it to be accurate because you have money on the line. If the model isn't very inaccurate between 40% and 60% it doesn't hurt you because you didn't risk your money anyway since the model didnt' have a very strong recommendation.
This early in the season this seems like an excellent way to rate, because there's not enough data yet to really understand how good these teams are based on current season data. Maybe once we've hit the halfway point you could start comparing all the models to expected win% given by efficiency or point differential, but right now that's not a viable option.
can'tseethechart
11/03/2010
even blown up on its own page i can’t read the first chart. would be nice if charts were spreadsheet links or at least much bigger and readable without hassle.
can'tseethechart
11/03/2010
on second try i got the 1st chart readable. well half of it at a time that is. if the chart is going to be that wide i’d suggest put the team names on the right side too and maybe even in the middle. as it is I have to scroll left to right to read the names and try to track the row to the column. and then do it time and time again for each team.
arturogalletti
11/04/2010
nosee,
Looks pretty good on mine. I have the limitation that I don’t want to share Hollinger’s #’s since they’re behind the wall so to speak at ESPN. I’ll see what I can do in upcoming weeks to improve readability.