Until you got through the next 5 batters.
Over the course of 300 PA, having Phillips and Frazier(at their current OBP) over Stubbs and Cozart(at their current OBP) would have produced roughly 25 more times on base.
Do you really think we'd notice a tremendous difference in getting an extra base runner in front of Votto every 3 games?
That's not to say it shouldn't be done, but I don't think we'd really see a noticeable difference.