So here is a classic case of what I mean.
I noticed that I was down-voted once and then again so two down-votes. So I decided to put up a cocky comment like, "Just love it when my questions get downvoted for no apparent reason! :)"
Now one of the people who noticed this, seem to take pity and probably upvoted my question, just so that it doesn't send out the wrong message to people who posts such questions. Now I am pretty thick skinned and even if I am downvote to oblivion I shall not stop asking question morally or ethically questionable subjects. But does it not send out the wrong idea to new adopters of SO and prevent them from posting questions here?
When I say newcomers, I mean guys who have not read the wiki or faq on SO etiquette and then they move on to some other forum where probably people are more welcoming. We cannot argue that they should read the wiki. Flag if they think they are being serially down-voted or other reasons. They will just leave without going through the hassle.
So how do we tackle such situations.. I definitely don't have enough rep to do anything about it, but people with some rep should take notice of such things and act as mediators I guess. :)