I think defensive performance varies in the sense that, based on fluctuations in so-called true skill and the variability of a season's worth of batted-ball opportunities, particularly for outfielders, a defender's actual measured production can be quite different from season to season.
But "actual measured production" is not exactly the same thing as "true skill," and I think what M2 was saying in the original post was, the current metrics are sensitive to the opportunities factor to the point where the numbers can swing markedly from year to year even if the true skill does not budge an inch. As with many other cases with baseball stats, estimating true skill and projecting future performance and figuring out what really happened all require a different lens.
(Of course, it also begs the question of how much people actually care about such things in the context of things like MVP voting. It's kind of like noting a guy was really BABIP-unlucky -- sure, it hurt his numbers, but no one's going to vote based on hypothetical value.)
But I do agree with you, RMR, about the fact that we have to use something, even if it's just UZR + gut. When comparing two players, one of whom hits better and the other defends better, we can't make a choice without at least implicitly assigning a value to the difference in defense. Even ignoring defense is making a value judgment, albeit a lazy/wrong one.