Should we have some sort of indication that an answer is highly disputed?

Question

When there is significant disagreement on an answer, it is often the case that it will have a net score of 0, or 1, or -1. This is a bit misleading, because the actual votes on the answer might be -6/+5, but only the -1 score is visible. Once you have enough rep you can click and see the +/- votes (that's actually my favorite feature!) but not everyone has this ability, and not everyone who has it uses it all the time... I only tend to use it when I already think there is probably disagreement on a post (either from comments or because I think it's wrong) and I'm curious how others voted.

So, would it be worthwhile to have an indicator of some sort that an answer is highly disputed? The actual threshold for "highly disputed" would be up for debate, but when an answer has a significant enough number of votes to be judged as voted on by a "sample" of the userbase and when the upvotes and downvotes are really close to each other (as in my -6/+5 example) it's pretty clear at that point that the voters do not agree that the answer is correct. But this isn't obvious without seeing the vote breakdowns, and we might have a case of -5/+7, where the answer has net +2 votes and so the casual viewer thinks the answer is correct, even though it's highly disputed.

I don't know exactly what this indicator would be; that can be a discussion for once we've decided if we need this or not. But I'm starting to think we kind of do, because the net vote count can be a bit misleading. Thoughts?

To be clear, I am not suggesting that we display the +/- vote breakdown by default on disputed questions. I'm just wondering if we need some kind of visual indication that there is a lot of disagreement among voters. (Perhaps we could also allow the score to be displayed on-click if the answer is disputed. But the actual logistics of how we'd do this aren't the point; we can iron that out before the feature-request :) I'm just curious if others agree that this is something we might need.)

Answer 1

The actual threshold for "highly disputed" would be up for debate

Yeah, well... It's kinda hard to determine whether we need this or not without having actual examples of posts that might benefit from it. My gut feeling is that "controversial" is a pretty bad sign; if folks are that divided as to the worth of a post, there's probably something about it that could stand to be fixed.

But hey, let's throw something together and see what pops up:

declare @dumpDate as DateTime = (select max(CreationDate) from Votes);

select top 100 PostId as [Post Link], count(*) TotalVotes,
  Round(count(case when VoteTypeId=2 then 1 else null end)*100.0 / count(*),1) UpvotePct
from Posts p
join Votes v on v.PostId=p.Id
where VoteTypeId in (2,3)
and p.CreationDate > @dumpDate-30
and PostTypeId=##postType##
group by PostId
having count(*) >= 8
  and count(case when VoteTypeId=2 then 1 else null end)*1.0
    / count(*) between 0.33 and 0.67

order by count(*) desc

That'll give you some "controversial" questions or answers (PostTypeId=1 or 2) from the last 30 days of the data dump. Turns out, there aren't all that many controversial answers; here's what I got¹ for the last 30 days²:

• Different rounding with println and printf
• https://stackoverflow.com/questions/20043406/php-find-similar-substring-between-two-string/20043570#20043570
• Is it possible to include C++ libraries in C programs?
• What is the difference between dynamic dispatch and late binding in C++?
• C# huge performance drop assigning float value
• https://stackoverflow.com/questions/20242798/how-to-create-mini-versions-of-my-code/20242921#20242921
• Insert in middle of list
• Random.Next() always returns 0
• Why is there not a constructor for List<T> that takes a params argument?
• Can VideoView be detach and reattached without stopping the stream?
• Is separating MySQL code from PHP code a standard, common practice?
• Refactoring minimum method

Now, are those really worth calling out as controversial in some fashion?

---

Just realized you're thinking about smaller sites; here are all of the answers meeting this criteria on ELL:

• What's the accepted way to use "criterion", "criteria", "criterions"?
• Why is "a Japanese" offensive?
• My mom or our mom?
• Why is 'what will you do tomorrow evening' incorrect?
• It is expected that . . .?

Meh. I guess I could see this as perhaps an interesting 10K tool.

---

^{1_{Data.se is in kind of a bad way right now, so I just ran these internally; YMMV.}
2_{There are 5270 answers matching these criteria for all time on Stack Overflow}}

Answer 2

This makes sense especially in cases where a now-incorrect answer is accepted. Such an answer can have amassed dozens or hundreds of upvotes over time, and it can take a long time for it to get voted "back down".

On the other hand, suddenly showing the up-/downvote ratio would likely be confusing for many users.

You can also make the case that it isn't really needed: on highly disputed answers, comments will usually show what's up. Also sometimes, people will take the law in their own hands, and apply an edit.

Answer 3

Yes, this is an unfortunate artifact of using "upvotes minus downvotes" as your score.

There are methods that are arguably better -- see e.g., Sorting answers, given overvotes and undervotes, which shows how to take into account the statistical confidence in the overall rating, given the votes -- but my impression is that the StackExchange folks have decided the current method is "good enough". That's understandable.

I agree with you that this does come up occasionally and it would be nice to have a way to enable people to notice that the answer is disputed before using it. I wonder if a better way to deal with it might be by improving which comments are shown. Usually a highly-disputed answer will have comments indicating the problems with the answer, so if we can have some comment sorting algorithm that maximizes the likelihood that those comments are noticeable and get noticed, that might be an alternative way to deal with highly disputed answers.