101

Internally, we've been discussing how we can deal with obsolete or out-of-date answers. A few weeks ago, I requested some feedback on how to encourage edits to these types of answers, while I'm still wading through everything on that post but now I'm focusing my attention on the problem of negatively scored accepted answers.

There are three sorting options to display answers on a question:

  • active – based on last activity/edit
  • oldest – based on date posted
  • votes – score based

In each of these options the accepted answer is pinned to the top of the sorting order, unless it’s a self-answer by the OP. The "pinning" also applies to negatively scored answers. No matter which sorting option is selected by a user the accepted answer appears at the top. If you have the same question and the accepted answer is negatively scored, you’ll end up with a view like this:

enter image description here

Over the years, there have been many requests to change the behavior of these types of answers:

Plus many, many others.

Well, I think it might be time to implement a change to some of these negatively-scored accepted answers.

Proposed Solution

  • An accepted answer will no longer be pinned to the top spot if it is negatively scored and has hit the threshold to get the lightened greyish color (e.g., ≤ −3 on Stack Overflow)
  • When the accepted answer is unpinned, it will be sorted naturally based on the sort order selected by the user (active, oldest, votes)
  • If an accepted answer accumulates upvotes and gets ungreyed, it will take the top, “pinned” spot again

Here are a few numbers of the questions that would be impacted by this change.

+----------------+------------------------------+------------------------------------+
|   Site Name    | Questions w/ Accepted Answer | Questions w/ <=-3 & Another Higher |
|                | <=- 3                        | Scoring Other Answer               |
+----------------+------------------------------+------------------------------------+
| Stack Overflow | 1097                         | 897                                |
| Mathematics    | 17                           | 12                                 |
| Super User     | 14                           | 9                                  |
| ELU            | 20                           | 13                                 |
| Server Fault   | 14                           | 11                                 |
| SO in Russian  | 11                           | 7                                  |
| Physics        | 14                           | 11                                 |
| Gaming         | 10                           | 8                                  |
+----------------+------------------------------+------------------------------------+

Currently on Stack Overflow, there are 1097 questions that have an answer that is ≤ −3. Of these 897 questions have another answer that is scored higher, so these accepted answers would no longer be pinned to the top spot in the default view (votes). Additionally, they would not appear before newly- posted or updated answers in the "active" sort, which might aid somewhat in giving attention to answers that have been created or edited specifically to address problems with the accepted answer.

While this doesn't handle all negatively scored accepted answers, it's a start to get the eyesore of heavily downvoted stuff out of the way for other answers.

  • 3
    Is this RFC or just an announcement? – Shadow Wizard Aug 21 '15 at 18:05
  • 22
    Yes, @ShadowWizard - yes it is. – Shog9 Aug 21 '15 at 18:05
  • 3
    I think this is a great idea. – Andy Aug 21 '15 at 18:07
  • 23
    Have you considered simply never "pinning" an accepted answer? The default view should be votes. If an accept was just treated like 1 up vote (or maybe weighted more - 2, 3, or even 5 up votes), you just naturally filter answers. The biggest advantage of the up vote is a special recognition in the form of an extra 15 Internet points to the solution that best helped the asker. That doesn't mean it's the best answer or the most helpful to the next person who finds the question. As a bonus, accepting could also apply a standard up vote to a question for sorting purposes. – Thomas Owens Aug 21 '15 at 18:19
  • 2
    Give us some data! Of the questions whose accepted answer would no longer be at the top, what proportion have an answer that's bad and deservingly downvoted vs an answer that was unpopular but is actually good and was accepted because it works? – Gilles Aug 21 '15 at 18:23
  • 17
    @ThomasOwens Yes, I have considered it and I'd actually prefer that. We've discussed this extensively internally and the general consensus is that the accepted answer is the signal that it's a tested answer which worked to solve the specific question. That being said, I'm taking some baby steps here, with focusing solely on negatively scored accepted answers...it's entirely possible I'll continue to pester everyone here to eventually sway their opinions. – Taryn Aug 21 '15 at 18:24
  • 2
    My opinion means approximately nothing, but I like this proposed solution. – Scott Arciszewski Aug 21 '15 at 18:26
  • 4
    Interesting. If you want, I can expand on this in an answer as to why it's better. <1000 characters isn't enough for me to formulate the full thought. This seems like a good first step, but unless it's followed by second and third steps, I don't think this idea alone would be that helpful in accomplishing what it looks like you want to accomplish. I can even put it into context of previous SE posts. – Thomas Owens Aug 21 '15 at 18:26
  • 6
    @ThomasOwens Feel free to expand in an answer. We're seriously discussing "obsolete" answers and are working to figure out how to deal with them. I'm taking small steps, and this is the first step. I'm open to other suggestions about it. – Taryn Aug 21 '15 at 18:29
  • 3
    @Gilles Feel free to browse through this query to check out some of the answers on SO. – Taryn Aug 21 '15 at 18:46
  • 4
    Why aren't you sharing your internal analysis? – Gilles Aug 21 '15 at 18:54
  • 3
    Is there any reason you don't just put up a "disputed" banner if an accepted answer has more than 5 downvotes (regardless of +10/-5 or +0/-5) or some similar number with an explanation that users of the site found it controversial? Moving answers around is really annoying in general for me (both other forums I've used for Q/A do this and it's particularly annoying when it can be done not by the asker).. This seems easy and straightforward unless I'm missing something. – enderland Aug 21 '15 at 19:46
  • 2
    @enderland What happens when you have a highly upvoted answer that receives DVs and is accepted, do you expect it to get this disputed banner? Example 1 and 2 both received DVs and are highly upvoted. – Taryn Aug 21 '15 at 20:03
  • 2
    @enderland Now it's starting to get confusing to explain when we unpin answers. Some weird formula that new users would never understand or grasp, that's part of what we're worried about. – Taryn Aug 21 '15 at 20:07
  • 1
    @CRABOLO As I've mentioned this is a first step to moving possible obsolete answers out of the way. We're targeting negatively scored accepted answers now, those other answers will be looked at later – Taryn Aug 22 '15 at 2:36
39

I feel this does too much (move accepted answers way down, where an unpopular but valuable dissenting advice may be lost), and simultaneously not enough (still keeping lackluster fastest-typist accepted answer on top).

Jeff Atwood wrote way back in the days:

in the default sort order (votes), the answer the community likes best will be either:

  • Directly under the question
  • Directly under the accepted answer, if there is an accepted answer

I suggest just flipping the order here:

in the default sort order (votes), the answer the OP likes best will be either:

  • Directly under the question
  • Directly under the top voted answer, if it isn't top voted itself.

This could be applied in all cases (no need for an arbitrary -3 threshold), without risk of burying unpopular dissenting voices.


As for the active and oldest sorts, I never understood the logic of pinning accepted answers there: the user explicitly asks for a chronological list, not for what's the best solution. Accepted answers should not have any special treatment in those sorts.

  • 1
    logical and elegant. One (relatively minor) point in favor of keeping accepted answer on top in active tab: asker is notified about new answers; at that point they are expected to reevaluate newest answer against accepted and decide whether to accept a new one. One can argue that when asker decides to keep accepted answer, this re-evaluation may count as sort of a "bump", making it stay on top in active tab (I know that's sort of idealistic, in real life many (most?) askers just don't give a sh!t) – gnat Aug 21 '15 at 20:08
  • I get what your suggestion is trying to do but we'd have some strange cases like this where the accepted answer is straightforward and answers the question but would be bumped by what's basically a blog post that you'd need to wade through to find a solution. – Taryn Aug 21 '15 at 20:09
  • 6
    I think that while your rule is well phrased, it won't be easy for people to intuit the rule from looking that the page. Arrived at inductively, the rule might be: "Sort all answers by score except the accepted answer which can fall no further than second". From a practical point of view, it will also be identical to vote-order sort in many cases since most questions have fewer than 3 answers. – Jon Ericson Aug 21 '15 at 20:21
  • 1
    @Jon I suggest restricting that query to the questions where accepted and top-voted answers are different. – user259867 Aug 21 '15 at 20:25
  • 1
    @JonEricson it probably would be rather easy for readers to figure that swapped order means asker failed to pick the best answer – gnat Aug 21 '15 at 20:30
  • 1
    @gnat: Right. And they will assume (without years of history) that being accepted has no influence on the order. And they will be correct in general. What this suggestion does is break that assumption in the rare cases when there is > 2 answers and the accepted answer's rank by score < 2. So why not just forget pinning the accepted answer altogether? Do you have some examples of "unpopular but valuable dissenting advice" that might be preserved by this proposal? – Jon Ericson Aug 21 '15 at 20:35
  • @bluefeet The long answer there has a direct link on top for those who don't want an explanation. I think that in general, people able to write excellent explanatory posts also understand the importance of including a concise summary at the beginning. And if they don't consider it, someone can fix that with an edit. – user259867 Aug 21 '15 at 20:49
  • 1
    @JonEricson I think that average visitor from web search (folks at top of SE food chain) sees too few of the questions to make a habit / assumption that would matter. As for active voters (next in the "food chain"), they see too many questions to have difficulty avoiding incorrect assumptions – gnat Aug 21 '15 at 20:58
  • @JonEricson If my query doesn't lie, 60% of questions where accepted ≠ top-voted (which are the only questions under discussion here) have more than 2 answers. – user259867 Aug 22 '15 at 23:51
  • Just noting my +1 is not for the last paragraph, as it is at least: I believe keeping the accepted answer first always make sense, but when it loses that in the by votes order, then it makes sense to order it normally for all sort types. – Mark Hurd Dec 18 '15 at 10:49
15

I don't have too much of a problem with this proposal. It mitigates any problems with the OP being careless, but does it in a way that doesn't really take away the OP's ability to pin a tested answer.

I don't, however, want to remove the OP's power to pin an answer too much. The OP is the one who has to actually implement the solution. Other voters might just read the answer over, say "that looks all right" and upvote without truly testing the answer.


Counterpoint:

The above applies to Stack Exchanges like Super User and Stack Overflow, but there are other SEs, such as Skeptics where answers tend to be more informational.

On these SEs, The OP is not actually expected to implement the answer in the way they would on SO. The OP can only evaluate the answer's correctness academically, like everyone else does.

On these kinds of Stack Exchange sites, it actually would make more sense to take some power away from the OP and give it to the community.

  • 5
    That's an argument for doing away with Accept entirely on those sites, not altering the sort order. – Shog9 Aug 21 '15 at 20:14
  • 2
    @Shog9 Site specific setting? – Taryn Aug 21 '15 at 20:19
  • 5
    @Shog9 Is that a valid option? No one said that was a valid option. I'm pretty sure that's been status-declined before, too. – Thomas Owens Aug 21 '15 at 20:21
  • 1
    @ThomasOwens if it's dropped entirely, we better figure some other way to differentiate as Q&A site from discussion (discourse:) forums. To avoid misunderstanding, not that I am for keeping it, just pointing that it's a matter where we'd rather tread carefully – gnat Aug 21 '15 at 21:20
  • 3
    Didn't say it was valid, @Thomas... As gnat suggests, taking this argument to its conclusion suggests that topics so afflicted may not be viable here at all. As always, it is worth thinking twice when dealing with questions that do not reflect real problems someone must solve. – Shog9 Aug 22 '15 at 1:30
14

I like this idea, and I especially like it as a first step toward possibly revamping how we handle accepted answers more broadly. Ideally, answers would be sorted by score, period, like they are when there's a self-acceptance. The usual objection to that is "but the accepted answer needs to be easy to find" (this argument doesn't have a good response for self-answers).

To address that and move us closer to better sorting overall, maybe an idea I proposed on an earlier question would help here: Add a link under the question to jump to the accepted answer, wherever it is on the page:

add link under question

Perhaps it would be a different shade of green (or different color), and maybe it should be bigger. And maybe it has a graphical component, like the check mark, and isn't just text. This is a rough mockup, not a precise proposal. Consult actual designers for implementation.

This allows the reader to jump straight to the accepted answer -- whether it was a lower-scoring self-answer, a significantly-downvoted answer, or whatever other criteria come into play later.

For consistency I would do this on all questions that have accepted answers, even where the accepted answer is first -- you might still want to skip past 23 comments and a post notice, after all.

If you make this small change (in advance of larger changes), the metrics gathered from it might inform the larger discussion of whether to unpin all accepted answers. Knowing how many people use the link to jump straight to it seems helpful. Granted, there's no way to know how many people scroll/page for it, so it's an imperfect metric -- the absence of clicks doesn't tell us as much as we'd like, but the presence of them would.

  • This is ... confusing. The links before that are action links related to the post directly above them. An "accepted" link does not do anything but jerk the scroller to another place in the page. – Deer Hunter Oct 30 '15 at 3:49
  • 1
    @DeerHunter the action is "jump to accepted answer". Suggestions welcome on concise alternatives for wording. Would you find it less confusing if it were below the other links instead of next to them? – Monica Cellio Oct 30 '15 at 3:51
  • Yes! Another improvement would be adding an arrow (↱, for instance), and another navigation link (like ↸ return to question) so that the navigation row wouldn't feel empty. – Deer Hunter Oct 30 '15 at 3:57
  • 2
    @DeerHunter perhaps you missed this in my answer: "And maybe it has a graphical component and isn't just text. Actual designers should be part of that discussion." I'm not trying to propose a final design; that would be premature optimization. The key idea here is "click on something right under the question to jump to the accepted answer"; this is a rough sketch of one way that could look. – Monica Cellio Oct 30 '15 at 13:05
  • worth a separate featreq, IMO. – Deer Hunter Oct 30 '15 at 13:09
  • Better to locate the link on “Answers” header line, next to tabs active/oldest/votes – Michael Freidgeim Jan 27 at 0:17
  • @MichaelFreidgeim that works too. More specifically, that tab should contain the accepted answer first (regardless of score or author) and otherwise sort by score -- mostly today's behavior, aside from the special treatment for self-answers. I don't know how that tab should be labeled, though. – Monica Cellio Jan 27 at 1:01
  • My suggestion was about location, but keep as a link, not as an extra tab. It’s actually the same as James suggestion below meta.stackexchange.com/questions/264045/… – Michael Freidgeim Jan 27 at 1:37
  • @MichaelFreidgeim oh, I see. Yes, I like that too (and already upvoted). – Monica Cellio Jan 27 at 1:38
6

The problem that you're looking at is that in any sort order, the accepted answer is pinned to the top, unless the accepted answer is a self answer, in which case it is sorted normally. The default sort order for a user is "votes", which orders so that the highest scoring answer is on top, but the accepted will override the rest of the votes and stick it to the top. Other options for sorting are "active" (most recent posts or edits are on top) and "oldest" (first post is on top, newest post on the bottom).

I think the right solution is simply to just eliminate the pinning nature of accepted answers entirely. This will allow the answer ordering methods to be true answer order methods and an accepted answer is just special recognition in the form of an extra 15 Internet points to the solution that best helped the asker.

The first thing that this will do is make the vote orderings true orderings based on some characteristic of the posts - the net score, the time edited, or the time posted. The default view for unregistered visitors and new accounts would be the same - net score.

I don't think that you can totally ignore the fact that an answer is accepted, especially when sorting by net score. As an asker, I have three options for scoring an answer - up vote, down vote, or accept. When I click the up vote, I'm thanking someone for their contribution and indicating that they helped me. But when I click on accept, I'm essentially saying that a particular answer was the solution to my problem. Ideally, if I don't accept an answer that someone else provides, I would write my own answer and accept that while up voting the answers that got me there.

So, how do you count an accepted answer? Right now, accepting an answer grants you +15 reputation. That's good. But let's also somehow count that toward the score of a post. I think the best method of doing this would be to have the accepted answer count as n up votes. To me, I would consider an accepted answer being worth 4 up votes. Why 4? To grant the asker the ability to give one answer the equivalent of 5 points in sorting questions by score (1 up vote + 1 accept = 5 votes). An open issue would be to determine if accepting an answer should force the up vote on that answer from the asker or not.

I think that the biggest advantage of this is that it puts more power in the hands of the community when it comes to what visitors see. I think this is more in line with the blog post and comments when Stack Overflow went live - share the correct knowledge by letting the community (and not just the asker of a question) decide what the correct knowledge is.


I asked for some help in the Programmers chat room about a query to help with some data. If you count an accepted mark as 4 up votes, there are 62,078 questions where another answer has a score of more than 4 greater than the accepted answer. That means, in my schema, instead of affecting about 900 questions, you'd affect over 62000 questions. If you're looking for impact on views, especially from drive-by searching users, this appears to do it more.


Some people have mentioned concerns about finding the accepted answer. I do think that it should be made obvious that the person asking the question was able to identify something that solved their problem. I also do think that it should be easy for someone to jump to that answer and read it. However, I don't think that one person should be able to pin an answer to the top of the list, since that particular answer may not be "the best" by some measure of the community and we want to curate and make "the best" answers available to the world easily.

  • 5
    My thoughts exactly. Make the accept vote a tiebreaker or a slightly higher weighted vote, but don't let it override the community, even for upvoted posts. Don't restrict this to negatively-voted accepted answers, have it apply to all accepted answers that are X number of votes below other answers. The green checkmark is a clear enough signal as to what the asker thought was best, no matter where that answer is in the list. – Brad Larson Aug 21 '15 at 20:15
  • 1
    may not be "the best"... -- given position of SE founders, this is somewhat an understatement. "I feel accepted answers are a fine social contract, but not a good data point for... answer quality" (Jeff Atwood) – gnat Aug 21 '15 at 20:23
  • 4
    Accepted answer gives 1.5 times the rep of an upvote - how about we take that as our measure of its value and weight it as 1.5 upvotes? – psr Aug 21 '15 at 20:47
  • 2
    @psr: That's been my thought in the past; it also has the advantage of guaranteeing no ties, ever. – Nathan Tuggy Aug 21 '15 at 21:45
6
+100

First of all, I agree that dropping -3 or worse answers is an unambiguously good thing to do; don't have much else to say about that besides what's said in the original question. However:

I don't think only addressing answers with a negative score properly addresses the problem.

Here are many examples where the accepted answer has quite a high score, but another answer has an even higher score:

  1. Difference: 2611, Accepted: 128/-41: Clone a specific Git branch
  2. Difference: 463, Accepted: 924/-10: How do I remove a Git submodule?
    • Original asker is still active and changed the checkmark
  3. Difference: 1249, Accepted: 955/-42: How do you discard unstaged changes in Git?
  4. Difference: 1121, Accepted: 413/-5: Case insensitive 'Contains(string)'
    • Original asker is still active and changed the checkmark
  5. Difference: 1106, Accepted: 432/-24: How to get current branch name in Git?
  6. Difference: 998, Accepted: 890/-8: Removing multiple files from a Git repo that have already been deleted from disk
    • Original asker is still active and changed the checkmark
  7. Difference: 977, Accepted: 937/-77: Create GUID / UUID in JavaScript?
  8. Difference: 866, Accepted: 333/-1: How can I merge two Python dictionaries in a single expression?
    • Original asker is still active and changed the checkmark
  9. Difference: 862, Accepted: 1537/-23: Iterate through a HashMap [duplicate]
  10. Difference: 826, Accepted: 57/-4: How to create REST URLs without verbs?

Going through these exercise made me realize two things:

1. It happens reasonably often that the question asker is still active and changes the checkmark.

In these cases, if a better answer is offered much later, and the asker is still maintaining the post, they may swap it out to give more attention to a better, later, more up to date answer. We don't want to bury this answer just because it only lags behind the original answer by 4 votes. We want the asker, in these cases, to be able to promote newer answers if they're still around.

2. There are plenty of obsolete answers that attract some downvotes but not enough.

Only handling the -3 or worse cases will only handle a tiny fraction of the problem answers. There are still many cases as #7 or #10 above where the answer has significant score but is not the best answer.


My first instinct was to promote answers that are above a certain percentage score, or something like that, but then I realized that it always comes back to one thing: the Wilson lower bound confidence interval, once again. My last post on this issue

Here's a new idea:

After 60 days of the creation of the question, if there are any answers that have a higher Wilson confidence score than the accepted answer, where the accepted checkmark counts for, say, 4 votes as in @ThomasOwens answer, they will get sorted past the accepted answer. In other words, after 60 days, there will three groups:

  1. Answers that have a higher Wilson confidence score than the accepted answer
    • Accepted answer is scored with +4 bonus for the checkmark
    • These answers will be sorted by total score; this just checks to see if they can be displayed above the checkmark at all.
  2. The accepted answer
  3. Remaining answers scored by total score as before.
  • 1
    Handling <= -3 cases is only a small part of the problem - yes that's correct I'm tackling a small piece of a larger issue...i.e. baby steps. I've looked at some of these other answers you are pointing to and there are cases (like this) where the accepted answer has fewer votes but directly answers the problem, so we wouldn't want to bump that from the top spot. I've still working through suggestions from my initial MSE post, this is the start of it. – Taryn Aug 21 '15 at 20:36
  • 1
    @bluefeet Sounds good. I've edited to clarify that I agree with your original idea. – durron597 Aug 21 '15 at 20:37
4

Additional Suggestion

What about when the accepted answer is unpinned there is a link at the top of all answers (or somewhere nearby) linking to the accepted answer?

e.g.:

accepted-answer-link

  • 3
    This seems similar to this suggestion. – Taryn Aug 22 '15 at 2:54
  • 1
    @bluefeet I didn't read that answer tbh, even so there is a difference in that I suggest only showing the link based on if unpinned :) so coincides with the suggested changes. – James Aug 22 '15 at 3:35
2

Always have it ordered by votes only. An "accepted" tick will still appear next to accepted answer and grant +15 to Asker, but will have no bearing on order of answers. Asker still only has power of +1/-1 vote just like everyone else.

I think giving the Asker the ability to "accept and pin" an answer is basically giving one person more power than the entire rest of the community.

The tickmark itself is useful, it's helped me diagnose some problems more easily in the past to know what specifically helped the Asker's exact situation.

0

Originally posted here.

Rather than forcing answers to the top, an accept could have the same effect as +3 and then +10% of the absolute value. This adapted score isn't shown, but it is used in sorting.

The +3 means that any post with an accepted answer at 0 and another at 1 still keeps the accepted one on top. Instead of the 0 score, it would be sorted as if it had 3.3 - i.e. above the answer with 1.

However, an accepted, up voted but wrong answer at 20 would be taken over by a post at 30 - 20 + 3 + 10% = 25.3, which is less than 30.

The 10% is of the absolute value because otherwise a post at -4 would become -7.7, or -8 when sorted. The accept always boosts the score - in this case to -0.9 or -1 . Workings: -4 + 3 = -1 + (0.1 * abs(-1)) = -0.9, which rounds to -1.

  • Problem with this is it is not at all intuitive to new users. Having some strange formula to determine positioning is difficult to explain to anyone. – Taryn Aug 30 '15 at 0:50
  • @bluefeet it is, but if it goes into the FAQ / help centre... There are already a few things like this (not that I can think of them right now...). I'm still wondering if it could be simplified in some way? – Tim Aug 30 '15 at 8:04
-2

Is this question still current?

If so, I'd like to say that you can't always judge an answer by the number of upvotes or downvotes. Particularly for certain subjects where posters might be misinformed by inaccurate popscience or propaganda. A system that works for coding where answers are testable doesn't necessarily work for everything. Sometimes the most popular answer is flat out wrong, sometimes the least popular answer is absolutely right.

Not only that, but sometimes people "game the system". If there are two answers, one from experienced high-rep poster A and one from some new-guy expert, poster A knows full well that if he can get his chatroom friends to quickly upvote his answer, other posters are more likely to upvote it. Then you get a cascade effect, which also applies to downvotes.

I like the way the OP gets to say which answer he thinks was best, whereafter it appears at the top. Sure, that isn't necessarily perfect either, but I think it's a second opinion that can be a useful counter. It's also a nice little reward to the OP for taking the trouble to ask a question. Without guys like him, stack exchange would wither on the vine. All in all, if it ain't broke, don't fix it.

  • Things could be done better. "if it isn't broken, don't fix it" means for me: "never miss an option to kill an initiative which could make things better". Btw, how about this? – peterh May 7 at 18:55
  • @peterh : I'll get us both into bother if I use comments to reply to chat. Please ask me about that elsewhere. – John Duffield May 7 at 19:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .