In your title you ask how many people have left, which probably wouldn't be measurable even if you specified the constraints of the query more clearly. Instead, I'll address the question in your body.
What can be done to discourage this behavior and encourage genuinely friendly behavior?
If you see something that violates site policies, including the code of conduct, flag it. Comment flags were recently overhauled to make it easier to see how to flag unkind or abusive comments. Use your flags.
If you see a systematic problem on a site (not just a stray comment), bring it up with specific examples on that site's meta and ask the community how to improve things. Helpful tip: these kinds of meta discussions go better if you approach with a question rather than a vague assertion.
Another thing you can do is to model good behavior. Make sure your own comments and posts are constructive, welcoming, and free of passive-aggressiveness, snark, or ranting. If you see somebody who needs a helping hand -- a pointer to the help, an edit that you can provide, etc -- then offer it if you can. Be the change you want to see.
You might be labeling as "passive-aggressive" things that are not, like downvotes and close votes. Those are important site functions that are explained in the help.