I can only speak from my son's own personal experience.
He will be 9 years old in October and has been playing organized baseball since he was 5. Currently, his age group places him in the Rookie Ball level (coach pitch); however, this fall, he will be transitioning to the Majors (kids pitch to kids).
If my son had not played "All-Star/Select" baseball during each of the last three summers, I don't know if he would even still want to play baseball anymore.
Unfortunately, until you reach the Major level (9-12 years old), the league where he plays does not do a good job of sifting through the talent based upon their pre-season "assessments." Therefore, my son has played on some REALLY, REALLY, REALLY, bad teams both of the last two years.
As a result of being surrounded by sub-par teammates during the spring--my son has very much anticipated playing with a higher caliber teammate (and opponent) during those 20-30 All-Star games every July.
In short, he develops more as a player each July than he does from March-June while playing with his non-AllStar/Select peers.