The Wikipedia of Long Tail Programming Questions

Question

So I have just read the newest blog entry and it seems to be addressing something I had brought up here before, namely a way to deal with the constant stream of duplicated questions. It's interesting, and the way I read it it is encouraging what I wanted: Assembling a set of FAQs for a specific subject, mostly by FAQifying existing questions.

However, after several months of doing exactly that, I can say that there are several obstacles in the way of doing it. Here's my list, initially ordered by what I consider increasing severity:

1. Searching for dupes is so badly supported technically, I need to spell it out: It's a pain in the ass, and I'm not referring to a member of the Equidae family here.

2. The existing system encourages answering duplicates, instead of taking the time to find out whether the question was already asked. Even if I know there must be dozens of questions this is a dupe to, if I sneak in an answer as the first, I might earn some rep on it before it gets closed.

3. If an old question exists, but whoever asked it accepted an inferior answer, this answer will always be on the top, even if it got down-voted into smithereens and a much superior answer exists, that got up-voted often enough to earn the answerer a gold badge. For a site aiming to deliver answers to questions people paste into Google, this is really bad, since those people might not know the site well enough to know to scroll past the first answer.

4. There's no good place to put together a list of FAQs so that they are easy to be found. For example, for SO's c++ tag there's a built-in list of FAQs, but this is not a list of frequently asked questions, but a list of frequently linked-to questions, which is not the same at all. The most linked-to C++ question is The Definite C++ Book Guide and List, which is rarely ever asked about, but often hinted at. Stack Exchange itself solves the problem for its FAQ by having an FAQ index here on Meta, but I see next to no chance such a pseudo-question for, say, the c++ tag would survive even 5 mins before getting closed according to SO's rules.

   So we set out to create a c++-faq instead, where we can add/remove questions manually, with the tag's main page listing the most up-voted of these questions. While this is much more close to a real FAQ, it is like a red-haired stepchild of the c++ tag: the latter offers no way (link) to discover the FAQ tag and its entries.

   That tag achieved to make it easier for the regulars to find duplicate in order to close them. This was a very good first step, but only the first one of a long way still to go.

5. Once you start to generalize questions to FAQs, they often require an elaborated answer to be answered comprehensively, which is very hard to fit into one answer and still be readable. Posting several, related answers, however, has the problem that they might need to be read in a specific order, in order to make sense. The only way to solve this is to fix the order by links in the question, which is very clumsy to do.
   (Note: I don't think this is the most severe issue, but I needed to put this at the end nonetheless, since in comments and answers others have referred to the problems by their original number.)

All the things I listed here (feel free to add to the list, BTW) could be solved, and there have been ideas proposed to solve most of them. (For example, reputation awarded to questions that are closed as duplicates could be limited, while closing questions as duplicates could be awarded with reputation.) For others, solutions would have to be set up and tried. For example, a tag page's FAQ tab could list questions that are frequently referred to when closing duplicates, or there could be some magic around [faq] tags so that they can be discovered and searched.

The problem I see is that this isn't wanted, to the point very valuable community members leave SO because they consider futile any attempts to alleviate the situation.

Answer 1

Duplication is a Hard Problem, and not one that is going to go away by sheer force of will. We can all bloviate all we like about how bad the situation is but what's lacking is a comprehensive solution. So far all we have are scattered suggestions that attack tiny fragments of the problem and may have unintended negative side-effects.

I've noted in the past that the core team often has dissenting views, and there is no clearer evidence of this than on the issue of mass duplication:

• Joel takes an optimistic and idealistic approach, believing that making the internet a better place is a venerable goal and that communities should have an intrinsic motivation to do it. It works for Wikipedia, right?

• Jeff is more of a populist, stating essentially that given the right tools, the people will eventually do the right thing, at least most of the time, and user experience is king (no penalties for asking/answering duplicates).

• Robert, whom you might be less familiar with unless you participate on other Stack Exchanges, is keener on crowd psychology and social proof. If you follow his blog posts you'll see that his message is that new members of a community are heavily influenced by the behaviour of the existing community, and that quality standards must be established early on and periodically reinforced in order to avoid an Eternal September situation. How duplicates are treated factors heavily into a community's quality standard in my opinion.

This is more than just an academic exercise or a criticism of the team. The point is that all of these viewpoints are correct - for a certain segment of the community.

Some people really do want to make the internet - or at least their community - a better place. But others are either chronic helpers or reputation hoarders and are unable to see the bigger picture.

Crowdsourcing is an effective tool - as long as the goal is strictly informational and personal motives are abstracted away.

And most people do aspire to live up to the community's standard, but there will always be a certain number of leeches and help vampires (which contribute heavily toward duplication).

It's easy to solve problems for the general case. It's the long-term accumulation of edge cases that erode communities. What we need to do, but have not even come close to doing, is gating off the edge cases without damaging the mainstream cases. It's abundantly clear that Jeff will never accept a solution that discourages participation, and it's hard to argue against his logic, considering that its precisely that participation that made Stack Overflow so attractive/useful in the first place.

As infuriating as it is for many of us to see other people piling on to obvious octuplicate questions for easy rep, levying penalties on them is not the answer. What's needed is a way to provide an incentive for minimizing answer sprawl that is stronger than the incentive to add to it, without lessening the incentive to answer in the first place.

---

It's usually at this point that everybody jumps headlong into a discussion of methods, without actually stopping to think about what the incentives are. There are several!

• Getting the problem solved (asker);
• Helping the asker get his problem solved (answerers);
• Teaching the asker how to solve his own problems (answerers);
• Earning reputation (answerers);
• Keeping the site free of noise and clutter (moderators, closers);
• Helping future readers to solve similar problems (closers, editors);
• Improving the overall quality of content (editors).

---

Probably the most interesting thing about incentives is that the asker almost universally has only one incentive, and that is to solve his immediate problem. Sometimes people may post "learning" questions but those are generally not the people who post low-quality or duplicate questions.

So the first question we should be asking is: Could we help the asker get his problem solved faster and more easily by never posting the question at all?

It's been stated that the in-site search needs to be improved, and it does - but that only helps the people who are willing to search. Primarily we're concerned with the people who don't search - those proverbial help vampires. Can we provide an incentive for them?

If you're anything like me then you've succumbed to the rubber ducking phenomenon at least once or twice. Nothing is more likely to get somebody to abandon their half-written question than having their problem solved while they're writing it. Can we do that?

We have the "related questions" section that pops up under the title box. It's actually not too bad at finding duplicates. But it suffers from a fundamental problem: People have to click on those links and navigate to different pages. They have to interrupt their work flow to investigate possibilities that only might be useful. For non-power-users who don't know how to instantly fire open all those links in different tabs, that's a major disincentive. Easier and faster to just fire off the question, right?

A more powerful system could stop these questions mid-stream. Refine the duplicate matches while the post is being written, and if any start to look "exact" enough (based on some yet-to-be-discussed scoring mechanism) then display that question and answer inline. If there's an issue of screen real estate then display a highly-visible indicator that a strong match was found and provide some AJAX-ified ability to display the content without ever leaving the current question.

How many times have you started asking a question verbally and had somebody cut you off in mid-sentence with "I know what you're going to say, and the answer is..."? I've had it done to me and done it myself plenty of times. This is the equivalent of that, except less intrusive.

Below is a conceptual example of what I'm talking about. I suppose the tooltip preview would be optional, but would be a major help here. For space reasons it would only show the top (accepted or highest voted) answer and might have to cut off the answer and/or question if it were too long.

---

That's one major incentive taken care of, but let's say it fails. We now need to take into account the answerer's desire to help as well as his desire to earn reputation - both very powerful incentives on their own - and try to instill an even greater desire to improve the community. To "make the internet a better place", as it were.

This is not going to be easy using any technical means. For one thing, it's not even possible to measure the success or failure of the latter objective, whereas the former objectives are easy (upvotes + accepted). I propose an alternative: Allow people to do both at the same time.

Closing the question as a duplicate does not contribute as much toward helping the asker solve his problem as actually answering the question. But could it? Quoting from Jeff:

Put yourself in their shoes. Instead of finding …

Duplicate Question
—
Duplicate Answer

They have to deal with finding:

Duplicate Question
—
[closed as duplicate of Question] click here to see answers

Now, what other site requires users to do some sort of weird scroll-down, click-here-first to see the answer nonsense on the search results before they will reveal the answer? Oh yes, our old hyphenated pals. Do we really want our site to work like theirs?

And of course the answer is no, we don't. But so far, I don't think anybody's pointed out that those aren't the only options! What about this instead?

Duplicate Question
—
Original Question
—
Original Answers
—
Duplicate Answers

I can almost already hear some snarky person saying "hah, that's just merging, we already have that! Just auto-merge them!" Except it's not:

• The asker's question still exists in its original form and "owns" its own answers;
• Merging might require additional cleanup, i.e. removal of duplicated answers;
• If it turns out not to be as exact a duplicate as people thought, the action can be undone;
• It helps moderators immensely in determining whether or not the question is actually a viable candidate for merging (if not, then is it really a dupe?)
• It ranks duplicated answers at the bottom regardless of score, at least until an actual merge is done. So authors of duplicate answers get to keep their rep, but once a question has been closed as a dupe, they'll no longer be "featured" on top until a true merge and the original answer authors will get most of the credit. (I'm assuming that the "merged view" would support upvoting and editing of original answers as well).

---

So unless I've overlooked something huge, we've now wrapped up the answering incentive problem as well, without providing any disincentives whatsoever to askers or answerers. All that's left is an incentive to get people to edit the question into canonical form.

Using the above UI, we already have a head start - the ability to compare the duplicate and original side-by-side and quickly see which parts would need to be generalized. The ability to actually vote to merge the questions would, IMO, make for an excellent 20k privilege (or maybe even higher).

The idea would be that members with editing privileges would edit one of the questions (probably the original) until it completely encompasses the duplicate question, then vote to merge. The knee-jerk "oh my god, we can't let anybody but moderators do that!" reaction could probably be softened a lot by actually keeping a record of the merge and providing the ability to undo the merge if necessary (even if only a moderator can perform the undo).

What I particularly like about this is that not only does it make it easier to merge, it gives the people who contributed good answers a strong incentive to participate in the merging process. Why? Because once it's merged, then their great answers will get ranked normally instead of below all the originals, attract a bunch of upvotes and earn the authors more reputation.

And guess what other problem this solves? For heavily-duplicated questions, it solves the barrier to entry problem that several people have complained about. That's because after five or six merges, there will be so many edits to the question/answers that they'll naturally get bumped into Community Wiki mode, and (almost) anyone will be able to edit them then.

---

So - did I miss anything?

Answer 2

Searching for dupes is badly supported technically

This is the one thing where technology can be used to drastically reduce the amount of duplicates. If the "similar questions" popup searched as comprehensively as google does (or even simply better than it does now) then we'd eliminate a ton of questions right off the bat.

Search is hard, but this is one area that can be done so much better than it currently is.

Answer 3

Point 2 is particularly bad in some areas. For example, the [objective-c] and [iphone] tags are basically filled with questions which are all specific cases of the general question "how do I do reference counting?". It's much easier to just answer the new questions with boilerplate answers than to dig through finding appropriate duplicates, promoting one question to be the general exemplar, and creating a general answer. Perhaps there needs to be incentive for identifying duplicates, or a way for an answer to accrue additional rep if it's a good answer to more than one question.

Answer 4

I was thinking that there are 2 different intentions at work on Stack Exchange:

• to create 'canonical' or 'generalised' best answers similar to Wikipedia,
• to provide quick answers to specific questions of users.

My thought was that there should be the option (with enough rep) to create a canonical question (and answer), which all the specific 'duplicates' can be linked to. This canonical question / answer could then be edited by the community in a similar manner to a wikipedia article. Future 'specific' questions can still be asked and answered but also linked back to the canonical question and any extra insight from the specific question can be refactored into the canonical version.

For example I recently answered a question about Unix time and how to convert it to DataTime in Ruby. I could have created a 'canonical' question about what Unix time is, what the Epoch is, how to generally use it etc. but specific questions such as how to use it in language x, with constraint y, or in situation z, could be asked, answered and linked to the general question.

Answer 5

In what I felt to be the spirit of this question, I opened What is a NullReferenceException in .NET? on Stack Overflow. It was suggested I raise this issue on meta.

Answer 6

At the moment duplicate closes are implemented by redirecting people to dead questions. This feels rude, both as a questioner and as a close-voter.

Here's one possible fix:

• Change a close vote into a "duplicate suggestion" which is listed among the answers
• Duplicate suggestions can be voted on and accepted
• If the poster accepts the "duplicate suggestion", the question is closed as a duplicate, and the poster gets reputation for accepting an answer. Comments on the question are still allowed.
• If the poster does not accept the "duplicate suggestion", nothing happens

Answer 7

It's basically the tag that needs to be standardized, so we can find duplicates easier. We have already some reference questions which solve the problem. Mind you, not enough to make people search for duplicates there first.

I can't put my finger on it, but I don't like [faq] as tag. Looks too generic, while [c++-faq] seems too mixed up.

OT: Some of the reference questions in [php] for example are tagged with [reference], which is too ambigious because there also is the language feature called variable references. But there is also [reference-question] which is easy to bookmark in connection with a language tag. Yet, there's only one till now.

In regards to #2 The existing system encourages answering duplicates - I do not agree this is necessarily a bad thing. Newcomers need to aquire reputation somehow too. But if you really wanted to fight the repuation gain, it would be as simple as turning all answers into community wiki when a question is closed as duplicate. Okay, if you really wanted to penalize answering duplicates, a moderator would have to delete the posts. - So the question becomes, do we really want to flag and delete to discourage answering duplicates? Seems too harsh since most answerers just have a form of helper syndrome.