46

One of the contentious issues on today's Stack Exchange is the Hot Network Questions sidebar, which can drive a lot of traffic to junk-food questions that can be very poor fits to the sites that host them (and, because of that, it can land a ton of rep on askers and answerers that by normal site standards should not get any at all).

Unfortunately, it is very hard to audit what does and does not get on this list, which makes it very hard to gauge the impact it has on a given site. A previous question asks how to get information on this via the data explorer, and it seems this is not possible.

So, I have a feature request: please make the Hot Network Questions list auditable from within the Stack Exchange Data Explorer.

What I have in mind is a new table which contains:

  • the question id
  • the date and time the question got on the list
  • the date and time the question got off the list
  • if feasible, the number of views driven to the question via the sidebar
  • if feasible, the per-site breakdown of those referrals
    • if that's not feasible, maybe a breakdown per category as separated on the main SE list, or just
    • a per-site breakdown only for sites within the same category as the hosting site

A table like this would go a long way to help understand the effects of the sidebar. For example, it would enable queries to dig out

  • HNQs that were closed after their time on the spotlight, or
  • that were heavily downvoted, possibly
  • preferentially by users with 200+ rep on the host site.

These give a better idea of what sorts of questions cause tensions within the site, which can help suggest ways to minimize those tensions, as well as provide a more accurate picture of the extent to which it is a problem.

A table like this would also provide of information on how the sidebar can help a site, including

  • users whose first answer is on a HNQ that go on to become established users, and the questions that capture them
  • good answers (as e.g. with a high score accumulated after the spotlight goes away) posted during the spotlight period, and the questions that sparked them.

A lot of the good that the HNQ does to a site is much harder to measure (as it is mostly in the form of traffic, what it does after landing, and whether those users return later or not) but a simple table like this provides a fair amount of insight.

I am aware that any addition to the SEDE needs to be thought about carefully, and there is always a development cost, but this one looks rather simple to me. It rides on logging that is (hopefully) already done, and which otherwise should pose a very small performance hit, and the rest of the processing can be done offline. The table itself should be pretty light (particularly if it only contains on and off dates), and it represents a small subset of questions to begin with. If nothing else, can the dev team comment on how feasible this is?

  • 14
    +1 for objective data to shed light on presently-ill-informed controversy. – Nathan Tuggy Nov 15 '16 at 23:22
  • Seems to be a hot topic on getting those featured questions changed. I do agree that some form of user preference should intervene. – user354226 Apr 14 '17 at 9:38
  • I just had a look at my RSS reader database and I have every single item that was in the HNQ RSS feed since January 23, 2016. It only has the <published> and <updated> fields though, and it probably has gaps once in a while. Would that help? – isanae Apr 14 '17 at 11:45
  • I would think "Number of views driven to the question via the sidebar" would have to be coupled with "Number of times it was served in the sidebar" in order to be meaningful. – jscs Apr 14 '17 at 12:19
6

Bullets 1 & 2 (the question ID and the moment of entering the list) will be available via the existing PostHistory table, for hot questions on or after February 28th, 2019. The PostHistoryTypeId for these events is 52. Next Sunday, we'll get to execute queries like this which will list the latest 100 hot network questions from a site:

select top 100 postid as [Post Link], creationdate as [Entering the list on]
  from posthistory
  where posthistorytypeid = 52
  order by creationdate desc
10

First off, view-tracking and referrer-tracking are not feasible; they'd be nice (for all sorts of things, starting with plain old question-views) but that's an insane amount of data. In theory, one could extract it from the raw request logs but in practice that's a massive headache for more than a few days at a time.

And we're already talking about quite a bit of data. Figure that the hot list is refreshed every 3 minutes, and contains 100 entries (neither of those are probably quite right, but close enough for a napkin estimate). That's 48 thousand rows every day, pretty close to the number of PostHistory entries created per day on Stack Overflow.

Of course, that's across all sites. It wouldn't really make much sense to put these in an individual site's database, since you'd want the full list to interpret them. So we'd have to put it somewhere else. Internally, there's a Sites database that holds on to stuff that isn't attached to a specific site, but exposing that in SEDE would be a bit odd. In any case, you'd have to be doing cross-database joins to get any of the information you're after, unless we denormalized it (as we do internally for generating the hot list)... But of course, that would increase the size of the data being stored as well.

Finally, some of what you're after cannot be exposed in public data ever: revealing the reputation of voters.

In any case, this is a lot less trivial than it sounds. And that's not even getting into the secondary issues of which "hot" posts are actually displayed - the sidebar lists pick a random sample, and those are cached too. And of course, this does nothing for questions that get heavily-shared on social media or end up in the newsletter or whatever.

Not gonna say this can never happen. I'd enjoy having it, if for no other reason than it'd stop folks from interrupting me with conspiracy theories posted to ancient answers. But it's a lot more work than it appears at first glance.

  • Thanks for this perspective. (and you're right, I hadn't thought through the voter-rep thing.) Mostly, I would like to see more SE acknowledgement that HNQ does have problems. We know the fixes are not trivial, but that doesn't mean we need to stop looking. What I would really see implemented is meta.stackexchange.com/questions/286020/…, but that's a separate story. – E.P. Dec 7 '16 at 0:28
  • 2
    wrt 48 thousand rows every day I wonder if you are familiar with the concept of sampling. A while ago it allowed Sony engineers fit Beethoven 5th into the limited volume of compact disc. (I for one much prefer the sound of vinyl but majority seems to think CDDA format is good enough already and vinyl supremacy is just a conspiracy theory) – gnat Dec 7 '16 at 6:54
  • So what's the Nyquist frequency for hot questions, @gnat? – Shog9 Dec 7 '16 at 15:25
  • 2
    you got to learn how to handle stuff like that from Sony folks. Don't ask "how do I keep high fidelity", ask "what frequency is technically feasible". One sample per hour cuts 48 thousands rows by 20x, is that good enough already or we need even lower frequency – gnat Dec 7 '16 at 15:38
  • So... @gnat... It would appear that you're all about that bass? – Shog9 Dec 7 '16 at 15:40
  • 1
    not me, engineers who made a solution that seems to be popular for almost 40 years (I personally prefer vinyl but who cares) – gnat Dec 7 '16 at 15:44
  • Well, the problem here is that it is quite likely we'd ignore a great deal of high-frequency activity in the list, which would tend to skew the interpretations of results toward questions that are more "sticky". A great many questions are "hot" for a few minutes after they're posted, but few of these stay in the list very long. – Shog9 Dec 7 '16 at 15:48
  • 1
    yeah that's where you have to make a choice, either you have no data at all or you have some data with known limitations. Guess back then folks at Sony had to make similar choice too – gnat Dec 7 '16 at 15:56
  • 3
    ...frankly who cares about data of question that was occasionally bumped into HNQ for a few minutes and collected handful extra eyeballs because of that. Whatever difference this may make is statistically indistinguishable from regular fluctuations of regular questions. It's questions that have been there for noticeable time and gained substantial sidebar attention that are interesting – gnat Dec 7 '16 at 18:25
  • 1
    @Shog9 Would it be feasible to just add a PostHistory entry to a question the first time it made it into the HNQ list? Then it could also be visible in the revision history, which might be useful. And it seems reasonably small compared to the amount of other normal PostHistory entries. And, presumably, it doesn't seem like it'd be too hard to implement. And it reveals no sensitive information. – Jason C Apr 14 '17 at 11:51
  • 1
    Still not trivial since the HNQ list generation isn't directly connected to any site, @Jason. Could be done, but gets into all the caveats that I list above. IMHO, the most accurate way to do this is to have a script just poll the list every n minutes and record the results for some limited period of time... Which also happens to be the easiest solution to implement... But I haven't had the time to do it, and as far as I can tell no one expressing concern about this is interested in accurate sampling. If you're interested, I recommend taking 1-minute samples for 30 to 90 days. – Shog9 Apr 14 '17 at 19:49
6

Stack Exchange team doesn't log hot questions at all and last time I checked they didn't intend to do this in a foreseeable future, not even internally:

It would amount to little more than a tool to feed speculation and paranoia...

Given above you could consider using history of HNQ list provided by Wayback Machine.

Their log of top half of the list looks pretty solid: "1,754 times between October 17, 2012 and February 10, 2018".

Second half of the list looks less representative but still: "Saved 670 times between May 5, 2012 and February 10, 2018".

https://i.stack.imgur.com/sTt4W.png

Based on sampling like above it looks possible to do reasonable studies and estimates of hot questions and answers in these.


It is worth noting that one can manually trigger saving a snapshot of hot questions page as shown at screen shot below:

https://i.stack.imgur.com/KuxBb.png

If there are enough people interested in such manual saving this may eventually lead to establishing a really solid historical archive / log of hot questions data.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .