"League average" for starters quoted above is the average production from pitchers in games they started in the major leagues. It's an appropriate baseline that includes those "6th starter" innings.
Suggesting that based upon his peripherals, Arroyo is aging into a slightly below average starter using such a baseline is pretty informative.
If one doesn't like average, then use replacement level. Concerning his value, it will get one to the same place though.