I'll be the voice of the middle ground, in that I have a six month old. My wife and I have decided that we leave the baby home for sporting events, unless we have suite or premier seating. I live in DC, and we go to Nats games in the Emerald Club. Lots of great space for a baby. Good AC, fancy changing room, cushioned seats that are shaded. Also, the fans aren't really noisy in those areas of the stadium. It's a nice family experience that way.
On the other hand we went in normal seats to DC United. We left less than half way through the game and decided we would not be taking our baby back to a game unless we had fancy seats. We'd rather get a sitter and get the full experience in the normal seats anyway.