I don't think it's ridiculous or silly at all. He probably was borderline, he didn't perform under the lights, and a borderline starter got relegated. The only question was how he'd do in live reps and it got answered.
And your examples aren't really good ones. Snell rushed for over 1,000 yards with Boom (doing the same and also rushing for 1,000 yards). Bledsoe was All-Pro and Brady probably would have started for other teams if he wasn't a Patriot.
I stand by the coaches know more than we keyboard warriors.