If the question doesn't belong (i.e. it gets closed), then the reputation penalty can be removed when the question eventually gets deleted via recalc. So in terms of reputation impact, on new and old users, that's not really an issue.
Negative voting is necessary on questions because it provides signals to both the system and the community. To the community, it helps indicate poor quality questions, and this is an important utility. To the system, it does much the same, but the system capitalizes on this by monitoring users who consistently provide low quality questions.
A user that provides many poor contributions may eventually be prevented from posting further questions. Whether it's because they're consistently posting just plain junk, or if they're posting something that's off-topic every time, it's basically the same signal - they're contributing noise.
Closing isn't used as "punishment" on users. It's administered to the question, on the merits of the question as presented, not on the behavior of the user. As such, closers in the community aren't what's "punishing" the user. It's their own questions that's doing so.