Re: "You're making us nervous."
You have every right to be there. They can't make you leave.
They have every to be nervous about it. You can't stop them from being protective of their kids.
Games are won on run differential -- scoring more than your opponent. Runs are runs, scored or prevented they all count the same. Worry about scoring more and allowing fewer, not which positions contribute to which side of the equation or how "consistent" you are at your current level of performance.