Hey EA, eventhough it's like 2 months late, can you make that spreadsheet avail? It'd be fun to punch in current RAPTOR ratings below, and do season long projections, then do adjustments for playing time.
That was great to get a peek behind the stats curtain that I'm too lazy to hunt down. I would be interested in another compare/contrast at the half way mark of the season.
It's worth remembering Raptor is primarily a player rating and projecting tool. They do bundle player ratings together to produce team forecasts, but AIUI that isn't the primary purpose or focus, and that's why the team component is just a simplistic summing of the individual ratings, as others have noted I tend to evaluate it with that lens more than the team prediction one, personally, although I can understand not doing so.
For those who are curious, the current team raptor stats to compare to the ones in the video are:
My own before-season predictions are looking pretty good right now (a coworker even commented on how spookily accurate they were). I only predicted the Western conference (like hell I know what's going on out east). I used 538's predictions as one source to inform my own ideas, so I can't discredit them too hard, but I was a lot closer than their model was at the time.
Even so, some things surprised me.
- The Warriors came out strong from the beginning. Bjelica and Porter surpassed my expectations, to say nothing of low-minutes-MVP candidate GP2 (I was all about him in Summer League, but I thought he'd be way more marginal in the regular season, lulz.) Plus the defense is really organized! I put us at 3rd overall assuming the new guys would take a while to fit in and we'd need Klay to put us over the top.
- The Lakers are even more dysfunctional than I thought. I had them at 6th.
- The Thunder have stolen more games than expected. At least 1/3 of them are the Lakers' fault.
A day or two I had basically every other team ranked in the exact order they were in the standings (I think the Mavs were up a spot or two compared to my guess?) but since then there's been more movement in the standings, like Denver dropping from 5th to 8th because Jokic is out. Just goes to show, it's not only what you predict but when you follow up that matters.
Wow this was super fun, and really shows how many moving parts go into the win prediction — Ie the errors have to really add up!
Also makes me wonder how the numbers are obtained for the Pythagorean win ratio. Technically it also accounts for pace (implicitly via total points scored/allowed), so it’s more informative than just the point differential.
Haha, how is this not a travel on PG? And what's the rule on the refs reviewing this? I imagine they went back to replay to check if he got the shot off on time, but are they not allowed to rule it a travel after the fact? It feels like this is exactly what replay would be good for...
Thanks Eric! This is a great breakdown. Super interesting. I really enjoyed how you explained the limitations of the model and how realistic adjustments can make significant impacts to final results. Very cool
Whoaaa... I realllly liked this video. This is a much more practical application of stats and modeling that I think a lot of people miss out on. I doubt anyone at 538 thinks their model is a crystal ball to predict the future... it's more like a gathering a measurements and a rough estimate.
A good analogy would be like hiring a contractor to replace your fence... you can get a rough ballpark figure of the cost using price-per-linear foot and type of materials for the fence, but until the guy shows up the property and can assess all the special cases, you can only take that original estimate as a ballpark figure.
Nov 24, 2021Liked by Eric Apricot, punk basketball, Daniel Hardee
When a data-driven, numbers-oriented, scientific website is so wrong so often, it's time to ask what's up. I think I looked up at one point how off 538 has been in previous years and it's something like ten games per team per year, roughly. Which is a large margin for error. And I think you mentioned possibly the most important problem with their model. It's the idea that if you add up some simple value for each player on a team, you get the total wins. But that's not how the game works. It's actually the 2021 Laker Theory of Value: take some players known to have racked up numbers and put them together, and you'll get the sum of those numbers. Well... so far that looks like a mess. Because if you add Westbrook's assists to Carmelo's shooting, you might get something good; or you might get Westbrook passing it to where he isn't -- and neither of them play defense. And similarly, the Warriors sum is so much greater than its parts. Passing, spacing, gravity, screens, communication on defense, etc.: these are all team behaviors that increase in value depending on the fit and familiarity. If you put Draymond on certain teams you may not get that much offensive advantage, which is what the 538 model in fact indicates. But on the Warriors he's a crucial point forward. He makes Steph alone a good deal better, as well as everyone else. The recent Raptor game is another piece of evidence against the 538 model. In just about every measurement, Steph had a mediocre game. But if you watched the game or even better the Explain One Play, Steph by his gravity, movement and screens got Wiggins and Poole great open looks. If every game were like that, the model would have a much higher number for Poole than for Steph, and we would all laugh haha.
If I were going to Vegas and had to bet on basketball games to save my life and was given a choice between basing bets on the 538 model or on the wisdom of the DNHQ chat, I'd go with the latter, easy. Human eye test has its flaws, but so do many metrics.
PS: A lot of us including management liked GPII over Avery Bradley for the 15th spot even though over the years his NBA box stats haven't been better, so that's another win for the eye test.
As it turns out you were probably pessimistic though we won't know for sure until Thompson and Wiseman show up. In the meantime, specifically in reference to Wiseman, that means that other players are getting those minutes at center and they are all better than what you would expect Wiseman to be. In addition, the end of the bench, GPII and Chiozza have been far better so far than what anybody could have reasonably expected.Between the 2 of them they fill the vacuum created by Livingston's retirement. Also there ought to be some sort of fudge factor for having the greatest shoot of all time healthy and on your roster. We know that makes a huge difference that doesn't show up if his team<cough>Oubre</cough>mates aren't capable of capitalizing on it.
I love how this video was so carefully reasonable, sensible, and realistic, and then the team just blew the doors off of it.
Hey EA, eventhough it's like 2 months late, can you make that spreadsheet avail? It'd be fun to punch in current RAPTOR ratings below, and do season long projections, then do adjustments for playing time.
That was great to get a peek behind the stats curtain that I'm too lazy to hunt down. I would be interested in another compare/contrast at the half way mark of the season.
It's worth remembering Raptor is primarily a player rating and projecting tool. They do bundle player ratings together to produce team forecasts, but AIUI that isn't the primary purpose or focus, and that's why the team component is just a simplistic summing of the individual ratings, as others have noted I tend to evaluate it with that lens more than the team prediction one, personally, although I can understand not doing so.
For those who are curious, the current team raptor stats to compare to the ones in the video are:
Team Raptor 11/24/2021
Player Min Off Def
~~~~~~~~~~~~~~~~~
Steph: 542 +9.2 +2.0
GP2: 225 +4.8 +9.3
Dre: 250 +0.6 +6.9
Bjeli: 257 +0.9 +3.6
Dray: 477 -0.8 +1.5
OPJ: 260 +1.6 +1.8
Loon: 311 0.0 +1.6
Wiggs: 500 -0.1 -0.5
Lee: 301 -0.6 +1.2
Poole: 500 +0.4 -2.4
JKu: 94 -4.3 +2.7
Chioz: 66 -3.8 +0.8
Dowtin: 6 -8.4 -5.4
JTA: 255 -1.7 -2.2
Moody: 85 -6.3 -8.4
My own before-season predictions are looking pretty good right now (a coworker even commented on how spookily accurate they were). I only predicted the Western conference (like hell I know what's going on out east). I used 538's predictions as one source to inform my own ideas, so I can't discredit them too hard, but I was a lot closer than their model was at the time.
Even so, some things surprised me.
- The Warriors came out strong from the beginning. Bjelica and Porter surpassed my expectations, to say nothing of low-minutes-MVP candidate GP2 (I was all about him in Summer League, but I thought he'd be way more marginal in the regular season, lulz.) Plus the defense is really organized! I put us at 3rd overall assuming the new guys would take a while to fit in and we'd need Klay to put us over the top.
- The Lakers are even more dysfunctional than I thought. I had them at 6th.
- The Thunder have stolen more games than expected. At least 1/3 of them are the Lakers' fault.
A day or two I had basically every other team ranked in the exact order they were in the standings (I think the Mavs were up a spot or two compared to my guess?) but since then there's been more movement in the standings, like Denver dropping from 5th to 8th because Jokic is out. Just goes to show, it's not only what you predict but when you follow up that matters.
Wow this was super fun, and really shows how many moving parts go into the win prediction — Ie the errors have to really add up!
Also makes me wonder how the numbers are obtained for the Pythagorean win ratio. Technically it also accounts for pace (implicitly via total points scored/allowed), so it’s more informative than just the point differential.
OT: https://youtu.be/o9Lx1JqaDLg?t=115
Haha, how is this not a travel on PG? And what's the rule on the refs reviewing this? I imagine they went back to replay to check if he got the shot off on time, but are they not allowed to rule it a travel after the fact? It feels like this is exactly what replay would be good for...
Thanks Eric! This is a great breakdown. Super interesting. I really enjoyed how you explained the limitations of the model and how realistic adjustments can make significant impacts to final results. Very cool
Whoaaa... I realllly liked this video. This is a much more practical application of stats and modeling that I think a lot of people miss out on. I doubt anyone at 538 thinks their model is a crystal ball to predict the future... it's more like a gathering a measurements and a rough estimate.
A good analogy would be like hiring a contractor to replace your fence... you can get a rough ballpark figure of the cost using price-per-linear foot and type of materials for the fence, but until the guy shows up the property and can assess all the special cases, you can only take that original estimate as a ballpark figure.
When a data-driven, numbers-oriented, scientific website is so wrong so often, it's time to ask what's up. I think I looked up at one point how off 538 has been in previous years and it's something like ten games per team per year, roughly. Which is a large margin for error. And I think you mentioned possibly the most important problem with their model. It's the idea that if you add up some simple value for each player on a team, you get the total wins. But that's not how the game works. It's actually the 2021 Laker Theory of Value: take some players known to have racked up numbers and put them together, and you'll get the sum of those numbers. Well... so far that looks like a mess. Because if you add Westbrook's assists to Carmelo's shooting, you might get something good; or you might get Westbrook passing it to where he isn't -- and neither of them play defense. And similarly, the Warriors sum is so much greater than its parts. Passing, spacing, gravity, screens, communication on defense, etc.: these are all team behaviors that increase in value depending on the fit and familiarity. If you put Draymond on certain teams you may not get that much offensive advantage, which is what the 538 model in fact indicates. But on the Warriors he's a crucial point forward. He makes Steph alone a good deal better, as well as everyone else. The recent Raptor game is another piece of evidence against the 538 model. In just about every measurement, Steph had a mediocre game. But if you watched the game or even better the Explain One Play, Steph by his gravity, movement and screens got Wiggins and Poole great open looks. If every game were like that, the model would have a much higher number for Poole than for Steph, and we would all laugh haha.
If I were going to Vegas and had to bet on basketball games to save my life and was given a choice between basing bets on the 538 model or on the wisdom of the DNHQ chat, I'd go with the latter, easy. Human eye test has its flaws, but so do many metrics.
PS: A lot of us including management liked GPII over Avery Bradley for the 15th spot even though over the years his NBA box stats haven't been better, so that's another win for the eye test.
As it turns out you were probably pessimistic though we won't know for sure until Thompson and Wiseman show up. In the meantime, specifically in reference to Wiseman, that means that other players are getting those minutes at center and they are all better than what you would expect Wiseman to be. In addition, the end of the bench, GPII and Chiozza have been far better so far than what anybody could have reasonably expected.Between the 2 of them they fill the vacuum created by Livingston's retirement. Also there ought to be some sort of fudge factor for having the greatest shoot of all time healthy and on your roster. We know that makes a huge difference that doesn't show up if his team<cough>Oubre</cough>mates aren't capable of capitalizing on it.