They're not supposed to consider offensive performance at all, but they have historically. Essentially, many writers seemed to want to the award to be "best player at X position" rather than just "best defender". They just couldn't bring themselves to give an award to a guy who couldn't hit. And because defensive performance has been terribly subjective historically, with the only real "data" being errors, a stat extremely biased to good hands guys with no respect to range, it was more narrative-based than anything else.