I am imaging that you have never actually seen Billy Hamilton play before. If you had, there is no way that you believe he would outplay Harris or Stubbs at this point in his career. There is not a chance.
Most guys stop at each level, but not all of them. Some guys go from rookie ball to A+. Some guys start out in A ball the year they are drafted. A handful of guys have skipped from Low A to AA (Cozart being one example).
But, the biggest reason I think we see guys making stops at every level these days is simple.... money. There is more incentive now to take your time developing these guys than to rush them because they are getting paid more and leaving when they get the chance. A guy gets up at 20, he is gone before his prime. Move a guy a tad bit slower and he gets up at 23 and he is better prepared (most likely) to step in on day one and you get him through his prime at a cost controlled price before he walks into free agency.
Also, baseball is the toughest sport around. That is why you don't see guys jumping straight to the pros from high school or college like other sports. And every single year, the professionals get even better. These guys are better athletes than ever before and the amount of detail they can get with scouting reports and simulation (pitching machines that have video projections of the pitchers wind ups but then throw their pitches with similar movement at the same speeds) make these guys more prepared than ever as well. This isn't your fathers baseball. There is more to take on than ever before to be a successful Major League player and I think the gap is wide from the minors to the Majors than ever before because of the technology that you get in the Majors compared to the minors that didn't used to be there.