43

This is specifically about the apparent frequency of the appearance of high-vote answers in "related links" that don't seem to be related. The examples below will use SO but this applies anywhere.

For the most part, the related links are useful. However, there are certain questions that I constantly see in the related links, but that aren't really related. In particular, the ever-popular Why is it faster to process a sorted array than an unsorted array? is almost always in that list, e.g. it's in the related list for Why am I able to have a public member in a non-public class?:

enter image description here

Of course, the sorted vs. unsorted array question is not related to every single question about Java, ever. However, StackOverflow seems to think it is, and so it's essentially just reducing the number of slots for truly related questions.

Another similar question is The Definitive C++ Book Guide and List, which pops up on the majority of C++ questions regardless of their content.

Point: I think that something may be off with the way that the vote count affects the "related-ness" of a question. Is there something that can be done to clean this up a little? Perhaps a hard upper limit on the amount that vote count contributes to the relatedness (high enough that it does not interfere with the current otherwise-good calculation, but just low enough to keep monster questions from "tainting" the lists)?

This is not intended as a comment on the general quality of "related" questions; I think it's actually quite good (and there are other discussions about that). I'm specifically referring to certain questions with high vote counts that constantly appear. It's not a major issue, it's also easy enough to ignore; but it does seem like a flaw of some sort (perhaps this post should be tagged bug instead of discussion?)


Other examples of good related lists "contaminated" with an unrelated high scoring question:

The "sorted vs. unsorted array" question, of course, is the biggest offender, along with the "c++ book list". In general, if you pick 10 random questions (especially from or ) you can probably find one of these topics in at least 7 or 8 of them.


As time goes on, the issue grows:

It's starting to get a bit out of hand (just over one year after I originally posted this). For example, on SO:

enter image description here

60% of the "Related" list is unrelated high-voted content. That is not the intent of this list, at least as long as it's titled "Related" instead of "Popular". As popular content becomes disproportionately more popular over time, the issue will only continue to worsen. It is beginning to affect the quality of the list.

Now, I do notice that the two main examples above (unsorted arrays, and C++ book list) in particular don't seem to appear in many related lists any more, so clearly something was changed -- but it seems like the change was just to explicitly exclude those specific questions from related lists (?) because the other examples besides those remain unchanged, and the behavior still exists in general.

7
  • 1
    Given that that's the highest voted question on SO, I'm guessing the algorithm sees the high score and gives it slightly more "relevance weight". Mar 3, 2014 at 3:02
  • 3
    @michaelb958 Yes, that's what I assume as well. I'd like to see the amount of contribution from score be capped. As it stands now, it's almost the same as that "sorted array" question being hard-coded to be in the list. There's only a small handful of questions like that, but they constantly dominate the related list for a huge fraction of the other questions on the site, so the effect is very visible.
    – Jason C
    Mar 3, 2014 at 3:12
  • 4
    (Also, incidentally, that's probably one of the reasons it's the highest question on SO; I know I personally upvoted it -- it's a great question with great answers -- but I only discovered it because it happened to be in the related list of an unrelated question. It's a bit of a cycle.)
    – Jason C
    Mar 3, 2014 at 3:13
  • To be fair - I click them from time to time. Most high voted questions are entertaining - while the "being related value" is an important factor - reducing this will also potentially cause random first time visitors to stick around less. If anything, I suggest showing different related links to logged in users - preferably based on what you like/dislike - but that might be a lot of work. Mar 3, 2014 at 23:09
  • 1
    @BenjaminGruenbaum I like them sometimes too, but that'd be more appropriate for a separate "popular" or "interesting" questions area. I'd rather have "related" be exclusively limited to questions that can be used to find more information about the current topic. In addition, certain questions come up very often, the novelty has worn off and so they just take up space. I see the "c++ book" and "sorted vs unsorted array" thread a dozen+ times per day. Besides, I like to think that SO is attractive to users for more reasons than an occasional interesting random link being displayed. :)
    – Jason C
    Mar 3, 2014 at 23:14
  • Fair enough, I'm convinced +1. I'd still like to see how these links convert new users though :) Mar 3, 2014 at 23:58
  • This seems to have been at least partially taken care of now, the related question list is a lot saner; is this true?
    – Jason C
    Sep 21, 2016 at 19:06

1 Answer 1

2

The key thing that the relevance algorithm works off if is the tags. When there is only one tag, well, thats all it has to work off of. In this case, it goes and picks highly voted questions with that set of tags where the set is {'java'}.

The algorithm also does not select questions that are locked. The C++ book list question has been locked (and it was done shortly after this question was posted). Thus, it no longer shows up in the related lists.

The key thing to do to fix crappy related lists on the right hand side is to use a set of more than one good tags. This is a mistake that new users to stack exchange make - they will often just put one tag in there. This will then narrow down the candidate questions to those that contain the set and you won't see the absurdly popular results that the single tag brings, unless you are still using a subset of the tags that the super popular question has (in which case, it may very well be related).

1
  • 1
    It's based on question content (note that the lower vote count questions in the new example are related, at least as much as they can be). I surmise that certain tags are only used to constrain the results, or at least factored in to the similarity search in addition to content. Evidence strongly suggests that votes are weighed in as well, perhaps in the same similarity calculation, or perhaps for sorting afterwards; and that the algorithm does not handle ultra high-voted questions (approaching 1000) well, giving disproportionate boosts to questions that are otherwise dissimilar.
    – Jason C
    Apr 26, 2015 at 13:53

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .