27

It's been fairly well established at this point that the Hot Network Questions mechanism can sometimes cause a question to blow up out of control (basically because they cause an influx of visitors who have low reputation, and these users can perform actions that increase hotness, such as upvoting and answering, but not actions that decrease hotness, such as downvoting and closevoting). If the question is good and the answers are good, this isn't a problem. If, however, the question is dubious (ontopic but not really the sort of question you'd prefer), and the answers are trivial, it can cause serious problems.

Most approaches I've seen suggested to the issue are based on disallowing drive-by votes. However, an alternative approach would be to automatically detect questions which are unsuitable for the list, and leave them off. There's probably no way to automatically find all unsuitable questions, but there's a relatively simple test that can be used to find a subset of them.

Here's a good example of the sort of question which will a) inevitably hit Hot Network Questions, despite everyone's efforts, and b) can easily be recognised as being inappropriate by a computer.

From the human point of view, the most obvious property of the question is that it's trivial – really trivial; if you know how to program, you can most likely write a decent, competitive answer to the question in ten seconds (and lots of people have). (There have been at least two attempts to close the question based on dislike of it, but none of the close reasons fit, so it's been allowed to stay open.) Of course, this triviality means that it's inevitably going to attract a lot of answers, and shoot up the Hot Network Questions list as a result, where it can attract even more.

From the computer point of view, we merely have to look at some of the most basic information about the question: it has (as of the time of writing) 33 upvotes, 32 downvotes, and 57 answers. That's a pretty bizarre range of statistics; and because the question was on Hot Network Questions for a quite a while, it indicates that the site's community had a very negative view of the question (because downvotes must have come from people with 125 rep, but anyone with an association bonus can cast an upvote, and we can assume a number of upvotes came in "from outside"). It's unsurprising, though, that really trivial questions will tend to get more answers than upvotes. Looking at things in reverse, if a question has more answers than upvotes, it means that some people put in the effort to answer the question, without wishing to put in the much smaller effort to upvote the question, and thus there's hard evidence that some number of people decided that it wasn't a very good question.

The question I linked might be an extreme example, but lesser examples play out on PPCG every few days, and I understand that similar issues happen on other sites too. As such, my proposal is to exclude a question from Hot Network Questions if there are more answers to the question, than net votes on the question itself. (The answer scores are irrelevant here.) Using upvotes on the question, rather than net votes, would also work and be even more conservative against false positives; however, net votes are probably more appropriate (a downvote also sends a strong signal that a question is unwelcome!), and I also understand that they're much more efficient to calculate (an upvote/downvote count takes noticeable time, but "number of answers" and "net votes" are shown pretty much everywhere in the site's interface). This check is unlikely to have false positives, and would catch a reasonable proportion (based on my own subjective judgements about which questions are reasonable for the sidebar, about half) of the PPCG questions which end up in the sidebar but really don't belong there.

This proposal seems likely to help bad questions running away with one-sided moderation, and keeping trivial questions out of the sidebar is also likely to help with Hot Network Questions' goals (as directing visitors to trivial questions is unlikely to give the visitors a good impression of the site, nor the site a good impression of the visitors).

  • 1
    How many questions would this actually affect? – Sklivvz Feb 19 '17 at 10:45
  • 9
    @Sklivvz: On codegolf.stackexchange.com/?tab=hot at the moment, 13 of the 48 entries. So quite a reasonable proportion. Those 13 are all pretty easy to answer, and the majority of answers to those questions aren't particularly interesting. Three of them are closed (but on the hot list anyway), and another currently has two close votes. – ais523 Feb 19 '17 at 11:03
  • Why would you not simply protect the question once it hit the HNQ to prevent the onslaught of answers? A close-reason of "too-broad" may also have fit as there are "too many possible answers". – Skooba Feb 25 '17 at 14:56
  • @Skooba: The answers aren't actually wrong or bad answers, just trivial. You can't protect a question until it has a certain number of deleted answers, and nothing about the answers merits deletion. – ais523 Feb 25 '17 at 23:51
  • Anyone with 15k+ rep can protect any question regardless of how many deleted answers it has. – Skooba Feb 26 '17 at 1:04
  • I just looked up the rules; it needs answers from new users to be proteted, but they don't need to be deleted. That said, it seems unfair that regulars are allowed to post trivial answers (as is anyone who got in early and got one upvote), but new users from outside aren't. – ais523 Feb 26 '17 at 1:18
  • I answer far more questions than I upvote. – Sparr Dec 8 '17 at 21:33
3

I wonder if that 'hotness' mechanism is that good after all. At the moment of writing I am presented this question as hot question. In my opinion it's definitely not hot. It has 1 upvote and 1 answer that is also upvoted. I suspect but can't prove it that asker and answerer upvoted each other. Furthermore the question is trivial and solllicits for information that can be found all over the internet. I believe there's another factor that inflates the importance of questions: reciprocal voting also known as 'the old boys network'. It's particularly strong on certain SE network sites. On such sites mediocre questions and evenmore mediocre answers on said questions receive roughly the same (inflated) number of upvotes.

Screenshot

  • 3
    I don't see that question on the HNQ list myself, and my script says its hotness score is only 8%, basically the same as for every question that's less than 6 hours old with a total answer score of 1. That shouldn't usually be enough to get on the HNQ list, but there's a site-specific scaling factor that's AFAIK not really publicly documented anywhere, so I suppose it's possible that the algorithm just decided that it really wanted to feature a hot question from RPi.SE, and that was the best choice available. – Ilmari Karonen Feb 20 '17 at 2:06
  • 1
    I've added a screenshot showing the question mentioned as 'hot'. It's taken from the Android app. – wie5Ooma Feb 20 '17 at 3:21
  • @IlmariKaronen as of now second page of HNQ list shows 7 questions with score under 8 points (screen shot) meaning that 8 could sometimes be enough for a question to get into the list – gnat Feb 20 '17 at 7:06
  • 1
    @gnat: That's the adjusted hotness score, with "per-site traffic adjustment" and the cumulative penalty for every hotter question from the same site factored in. (And that comma is actually a thousands separator, not a decimal point.) The 8% I was referring to is the "raw" hotness score, as calculated by my user script using the formula in the answer I just linked to. For the math.SE question you mentioned, that raw hotness score is currently 43%. – Ilmari Karonen Feb 20 '17 at 9:30
  • 1
    ... Anyway, I see that wie5Ooma is using the Android app, which AFAIK uses its own, completely different system to decide which questions to highlight as "hot". I'm not sure if that system is documented anywhere. (It might be using something like the same formula as the within-site "hot" tab, but that's just a guess.) – Ilmari Karonen Feb 20 '17 at 9:33
  • 1
    Thanks everyone for your feedback so far. Interesting to read all this. I thought the SE network was as open as possible but all this almost sounds like there's some kind of bubble filtering going on. – wie5Ooma Feb 20 '17 at 22:22
  • 2
    @wie5Ooma you need to understand that the very goal of hot questions is somewhat slippery: to entertain lemmings. And this often harms sites that host these questions, see eg Recent Trouble With Popularity, this may be the reason why Stack Exchange overlords prefer to keep things around these somewhat opaque – gnat Feb 23 '17 at 21:46

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .