Because you're still a part of the community that you serve, you just have special privileges that let you do what the community can't. Remember, almost every other kind of vote that you can cast is immediately binding - but when it comes to up and down votes, you're still a mere mortal, along with employees of the company.
Down voting needs to cost you something because the concept of negative reputation within reputation systems can be downright horrifying if left unchecked. Any time people can lose points, you have to be careful. Otherwise, even moderators aren't immune to casting them spuriously in lieu of doing other things like editing. It's good that you seem to favor voting and commenting instead of removing content, that's definitely more healthy when it comes to retaining users. But, we can't leave the ability unchecked.
The exception we made was for questions, because people just weren't voting on questions very much. However, you're still limited to your maximum votes per day (with the extra votes thrown in just for questions).
Also, try to avoid casting a down vote in place of the down vote that a flagging user did not cast. That's putting yourself in the position of being a proxy, which is something you ideally avoid.