I'm taking Brisco's side on this. It's simple human behavior. Stats can be looked at in a vacuum, but the human element cannot be discounted. It's like the shut down closer argument. A team breathes a sigh of relief when Hell's Bells blasted in SD, because they "knew" the game was over. But, theoretically, any good pitcher on the roster can get those outs.
A bad defense can weigh on a pitcher, just like in any profession. If i am working with idiots, i put more pressure on myself to do well, for the project, company, whatever. Athletes are not immune to the human condition.