There’s inertia in the output because there’s a ton of total input and we can only update a little bit of that per week. Basically this is the scientific method and we’re still in the earliest stages of the experimentation phase. Our working hypothesis at this point is still more induction and deduction than anything else. Well, Ken’s is.
Ken’s model starts with an educated guess and sets the bar rather high for new developments to prove the data wrong. That can serve to eliminate noise (such as early-season jitters, the process of gelling teams together, freshmen boneheadedness, pending coaching decisions about how to utilize rosters, and blowouts of people who were just too short for you when you actually played like hot garbage yourself) from the signal of what a basketball team really is and will be.
But it lacks the advantages of other systems which use much less frontloading. Systems which can fluctuate wildly in their predictions, but at least can claim to a much larger extent that their predictions are based on “real” data. In other words, at least as they try to represent the whole league top to bottom, systems weighted like that have more analytical power as opposed to KenPom but less predictive power. They’re less blind but more fickle.
There is immense value in both approaches.
And also in Jeff Sagarin’s approach, which is to play dead every year until he finally feels he has enough “real” data to make real predictions and then his system pops out of a bowl of rice.
At some point they’ll all tend to converge anyway. But we’re a long way from that at this point.
One more thing about KenPom in this context. Ken does love to tweak it, both to hunt for untapped predictive markers and to adjust for trends he sees in the game. But those tweaks tend to be pretty small. In general from a bird’s-eye view it’s the same system every year, with the same inherent strengths and weaknesses, the same early shakiness, which gradually attenuates at about the same rate. People are used to using it, and to a large extent they have its downsides figured in so they can still use it intuitively, and still have an intuitive idea what they’re getting from it at a given point in a season.