(This answer is primarily driven by my experience with Stack Overflow, and my understanding of the Stack Overflow tour page; but I have confidence that the same principles should apply more or less network-wide.)
Whoo boy. I have so many complaints about the reputation system generally (and even more about how voting impacts on the Q&A, as opposed to users). I actually have a blog post planned about it for the somewhat near term, but I stumbled upon this Meta post so I might as well take the opportunity to gather some thoughts.
Perverse incentives
Getting your question or answer upvoted awards 10 reputation; getting it downvoted only takes away 2. That incentivizes people to be controversial rather than boring. Boring is usually higher-quality and more encyclopedic.
Getting your answer accepted awards an additional 15 reputation on top of the +10 from OP's upvote. This incentivizes tailoring answers to OP's specific circumstances, rather than trying to write in a broadly-applicable way that will be helpful to people who find the Q with a search engine. It also incentivizes tolerating questions that are clearly too broad (if you help OP with multiple issues, and make code with multiple issues work, that increases the likelihood of OP being positively overwhelmed with the helpfulness). Also, anything with that effect, has the ripple effect of discouraging editing (since Q&A are now more tightly coupled, the minimum unit of coherent editing increases).
Accepting the answer also gives 2 reputation to OP. This incentivizes OP to accept an answer even if none of the provided answers actually get it right.
Reputation is awarded the same way for answers even if the question is downvoted and/or closed. This incentivizes answering questions that are clear duplicates, because of the likelihood that OP will upvote and accept that answer. Such answers interfere with the deletion of low-quality signposts (although I am personally of the opinion that keeping around duplicate questions is useful because it helps establish the importance of their canonicals). It also incentivizes trying to mind-read OPs who don't express themselves clearly.
There's an especially obnoxious cumulative effect here: the first upvote on a "fastest gun in the west" answer, along with the accept vote, nets a total of +25. It takes thirteen downvotes from concerned curators to make it unprofitable for unscrupulous reputation-seekers to answer questions that they ought to know shouldn't be answered. I've interacted with users like that many times. They often have 10+ user accounts and hundreds of thousands of rep; and their response to comments explaining the damage they're doing to the site ranges from a shrug to extreme indignation.
There's a lock-in for votes on Q&A that weren't edited, which doesn't respect the fact that sometimes people genuinely change their mind. (One thing that's happened to me a few times is that I upvoted something because I thought it was the best canonical available, then found something much better and actually discovered serious issues with the first option.) This incentivizes 2k+ users to make dummy edits in order to reflect that changed viewpoint. (I try to make my edits substantial, but there have been times where I improved a post while also changing my upvote to a downvote... because I really don't know what I was thinking with the original upvote. Sometimes the upvote was literally years ago, and I still had to edit because of the lock-in.)
Getting an edit approved awards reputation, but making unilateral edits does not. This means that people who reach the 2k threshold have a sudden shift from extrinsic to intrinsic motivation for improving the site. (This makes me want to launch into a whole separate diatribe about the edit queue, but I don't think it's on topic here.)
Downvoting costs a point. I doubt that many people are seriously discouraged by this, but it bears noticing that the stats show a heavy upvote bias among logged-in users. Last time I checked the statistics, votes from users were more than 90% up, while "anonymous feedback" is close to 60%. (This is, admittedly, skewed by the fact that users between 15 and 125 reputation can upvote normally but have attempted downvotes logged as feedback.)
Meanwhile, there is no reputation incentive for most curation activities. As above, that includes unilateral edits (2k+ users), and also:
closing duplicates
casting close votes on things that should be closed
casting delete votes on things that should be deleted
using the review queues (in any capacity whatsoever) - the only thing one can get from using the review queues is a ban from the review queues
(There are finer points I could make about the curation experience, but again I think they drift off topic.)
Comments can be upvoted, but not downvoted. While this doesn't award reputation, it has psychological implications. I've NLN-flagged quite a few old comments now that had a significant vote tally on them - typically, comments on an answer complaining that the problem shouldn't exist or should be simpler to address, and typically ones that show a serious (but common) lack of insight or understanding.
I'm not entirely sure how this works, but it's clear from my experience that moderators are disincentivized to delete duplicate answers on old questions if they're contemporary with the post. I don't get why, because this apparently would not significantly impact on the answerer's reputation - but I do think it bears mention in this section.
Currently, Q&A get rewarded the same regardless of the topic matter (or the inherent difficulty of the problem). This incentivizes people to stay in already popular tags and allows less popular tags (where answers are often more needed) to languish in obscurity. While the "no sense of urgency" culture is a net positive, good questions really should get answered eventually. As it stands, our only workaround for this is an extremely clumsy "bounty" system. Which segues nicely into...
Linearity
In my mind, it makes very little sense to count each upvote and downvote on a Q or A the same, and it makes barely more sense to tally per-post reputation to give an overall score to a user.
IMX, both Q and A aren't really any higher quality past a score of +10 or so; they're just older. Often they're lower quality because a) the information has become deprecated or b) it's an A for a Q that used to meet site standards but now doesn't. There are any number of psychological barriers to downvoting things that already have a lot of upvotes, and even with "trending sort" it can be quite hard to lower them on the page. Personally, my highest-voted answer is at +346/-2 on a Q that's +444/-5 - and I've thought more than once about deleting it, because the question is still blatantly two unrelated questions, even after extensive editing to make sense of a barely coherent mess.
IMX, Qs below -3 aren't really any lower quality than ones at -3 (except for spam/vandalism/trolling, which we do an excellent job of cleaning up). Even from -1 to -3 is a rather shallow gradient. A -3 score makes posts eligible for manual deletion; there's little point in going further than that.
On the other hand, IMX, As at +2 are much higher quality than ones at +1 on average - because the +1 category is swamped by things that were upvoted by OP and nobody else. A +2 typically entails some kind of confirmation from someone with relevant knowledge and understanding. Most people asking don't have that (or they would have self-answered).
Summing reputation gained across multiple questions doesn't make a lot of sense. For people who didn't get in early (and thus have the ability to gain "dividends" on old questions that come up a lot in search engines), the easiest way to game the system is quantity over quality. The first upvote on a question ought to count less (per the previous point), but it counts more (because of the accept system). But being able to write a lot of answers each day isn't evidence of expertise; it's evidence of addition to a particular mode of using the site. (I shudder to think what would happen if the daily reputation cap had not been implemented.)
One-dimensionality
Even if the system worked as intended in the above regards, and incentivized people to write high-quality Q&A (and put A only on Q that should have them), and take positive steps towards curating the site:
The ability to be helpful or insightful has little to do with the ability to be responsible - but we use the reputation system to award moderation-like privileges. (This has a knock-on effect that off-site critics mistakenly describe non-diamonds as "moderators".)
Clear communication is totally separate from expertise, but we reward both on the same scale. A reputation score can't tell you who's an X guru vs. who is obsessed with X and wants to find a lot of things to do with it vs. who is skilled at creating simple demonstrations for common X gotchas.
The site already has a separate system for recognizing expertise: tag badges. Why is, for example, editing of a tag wiki tied to overall reputation, and not to experience with that tag? Which leads me to...
Quirks of the privilege tower
It takes 15 reputation to upvote, but 125 to downvote. I mentioned earlier that this contributes to a bias in the voting/feedback stats. It's also inconsistent with the widely-held view (at least on Stack Overflow meta) that downvoting is a vital curation tool. In particular, that makes it that much harder to sink old content that is deprecated (assuming it can't be salvaged; but trying to add up-to-date information to old As that aren't CW often gets you yelled at) or which no longer meets site standards (which changed since the content was posted).
The site association bonus is enough to allow people to create chat rooms and edit CW posts, but not enough to downvote. This strikes me as very backwards. It shouldn't take subject matter expertise to recognize that a question or answer is unclear or unhelpful (or, say, politically inflammatory - and it's usually very hard to get site moderators to agree with R/A flags for that); but it might well require SME to recognize a technical inaccuracy or an opportunity to rephrase something in proper domain language, and it definitely requires SME to chat about the subject matter of that SE site.
It takes 50 reputation to comment on other users' questions. This means that a lot of new users who know a little and want to be helpful, end up trying to use the answer section to give feedback on questions (that should be closed). Even flagging takes 15 reputation, and doesn't let these new users explain specifically how they think the question should be improved. (Speaking of which, the flagging system is also a mess, insofar as low-rep users have additional flagging reasons that copy the close vote reasons, but work differently and independently from actual close votes, and are hidden in that interface.)
It takes 3000 reputation to vote to close questions. That's absurd. It's saying that "I think this question has issues that need to be fixed and shouldn't be answered in its current state", and getting two other people to agree with you before anything is actually done about it anyway, is somehow a greater and more dangerous responsibility than unilaterally attempting to fix it yourself (2000 reputation). (It's especially absurd in an environment like Stack Overflow, where the large majority of incoming questions are terrible and the site would be much better off if new questions started in a closed state and had to petition to get opened.)
It takes only 1500 reputation to create new tags, but 2500 to synonimize them. This is another bias towards creating junk and away from curation.
It takes only ten reputation to answer protected questions. This is definitely not enough, at least for Stack Overflow. Lots of old, famous questions have way more answers than they need - and we don't close them because they meet site standards and closure is seen as a black mark, but we still let almost anyone contribute additional redundant garbage to them. There have been times when I had something useful to contribute to a question that already had dozens of answers - but it's accompanied by a sincere wish that half those existing answers would just disappear.