-217

Some things happened yesterday that caused a need for us to (quickly) remove a site's eligibility to contribute to the list of hot network questions. For those of you not familiar with what we've come to call the 'HNQ' list, please visit the link; the list of questions shown as 'hot' on stackexchange.com are displayed in the sidebar of most Stack Exchange sites. Here's an idea of what it looks like:

Screen shot of hot network questions in the sidebar

It's a list of titles, with their origins vaguely identifiable by the presence of the 'favicon' element that identifies each site. We're extremely proud of the quality of content that can be found on every single site in our network. The titles, however, when shown out of context, can be problematic. Anything out of context can be problematic.

This isn't a new problem. We've dealt with trigger words in titles that occur naturally through good-faith questions on sites like Parenting, Gaming, Christianity, Islam and more. While we probably deserve some egg on our face for relying on regular expressions as a go-to quick fix, they've been sufficient for ensuring trigger words and overt clickbait was kept out of the list for the most part.

Then we launched a site that tended to have 3-4 hot questions asked each day (by any way one could conceivably measure velocity and good reception). The titles of these questions, out of context, are just too likely to be inappropriate on our professional and academic sites, so we took the site out of the contributing list. Still with me? Thank you, because there's no way to open this discussion without background.

This is one of many times we've talked about problems with hot network questions. This isn't a new problem. What escalated the problem is that we had to take an entire site out of the mix, which isn't ideal, and I'll explain why.

What goals do we have for the hot question list?

  1. They help with discovery. User interviews continue to reveal that folks simply don't often discover just how big of a resource we've become collectively. The list boosts engagement in quite a few ways (we'll admit some of them aren't optimal)

  2. They often remind communities to have discussions about scope. If folks see a clear demand for certain kinds of questions, it's worth revisiting conversations about how valuable finding ways to meet those needs might be.

  3. Hot questions are extremely great fodder for evangelists, the kinds of people that reach thousands of folks that are likely to identify with the way we do Q&A.

  4. Let's face it, they're a jackpot and a curse. A little patch of chaos that grows pumpkins and poison ivy - just usually more pumpkins.

What's broken?

The list of questions is generated through a very expensive query, cached, and refreshed on a set schedule. Giving moderators or even community managers the ability to 'evict' a problematic title while keeping the system in check with what we consider performant and sane are .. mutually-exclusive goals.

Now that we have gotten to the point where we've gone from saying "Let's ignore titles with these words.." to "Let's ignore sites with these titles..", we're seeing a pattern of scale that starts to suck if you think about it protractedly.

What do we need from you?

We want some ideas on ways we can continue to meet our goals of getting the right sites in front of the right people who are likely to appreciate them and participate.

Automatic selection simply will not scale if we continue to push the limits of subjective topic spaces, and we need to continue to do that for the network to grow and remain relevant to the issues folks want to use our software creatively to solve.

We don't expect to solve this problem in one post right away.

Let's say we agree to take out the hot network questions entirely. But, we want something else in its place that continues to pull in some of the benefits, without being such a mindless pain in the neck.

In order to find out what that could look like, we need to consolidate what we want out of it. And that could be "Just turn the darn thing off and be done with it", or "I found my favorite site through that list, you gotta preserve that somehow!" -- all of this input at this stage will be equally good.

Suffice to say, the HNQ (as we know it) is now looking at numbered days, but that number is probably kind of high because it's going to take some work to figure out what voids would be worth filling if we yanked it as we know it now.

So, let's talk about it. And, if this seems like it's a continuation of a conversation, it is, from the moderator chat room. Even though the scope of the discussion is still rough, I wanted to get it out 'in sunlight' early, this isn't something to be settled in an all-mod room.

Thanks for your time, we look forward to finally (at least committing) to settle on something we can all like a bit more than what we've currently got.

And in the spirit of causing tons of people to flood into a single discussion, I'm going to feature this; I just can't help it.

  • 142
    You are aware that by doing this you killed off some 75% of the site's traffic, yes? – Magisch Oct 17 '18 at 17:52
  • 89
    Can you please respond to the relevant meta question on the site in question, with what exactly those "things" were? – Sonic the Inclusive Hedgehog Oct 17 '18 at 17:52
  • 72
    Unrelated, but perhaps we should allow Meta Stack Exchange to be placed in the HNQ list. This can draw attention to important discussions here. – Sonic the Inclusive Hedgehog Oct 17 '18 at 18:18
  • 211
    @NicolBolas I had no idea what site it was, and so it made for a very annoying post. – Azor Ahai Oct 17 '18 at 23:44
  • 148
    I fail to see what's so problematic about the two posts in question. Sure, the topic does not seem to be particularly professional, but it's not vulgar, obscene, offensive, or anything like that. Is the problem that two posts in a row are related to sexuality? Are we really so puritan as to get offended by that? – forest Oct 18 '18 at 5:55
  • 216
    To clarify, I absolutely support a profile option to disable HNQ, or even disable questions that are not related to programming (I might not want to get sucked into politics while I am working, period), but to declare an emergency and disable it for everybody because someone on twitter who doesn't even use the site is a puritan who tries to see hidden subtext in everything they read? That's just silly. – forest Oct 18 '18 at 6:20
  • 71
    @E.P. That's not true in my opinion. Just because something is a trope doesn't mean it's sexist. Context matters everywhere, and the title on itself was nothing of the sort. "How do I tell someone that they might be autistic" is not an ableist trope, and suggesting it is is in and of itself quite ableist. "How do I stop people from flirting with me at a school I volunteer at" doesn't even mention either person's gender, and inferring any would be a biased inference and also sexist. I don't understand your line of argument here. – Magisch Oct 18 '18 at 12:55
  • 150
    @TheLethalCoder It is very very sad to see that People on Twitter who do not bother to read the posts they are complaining about are more important for SE now than Mods who care about their communities. That person didn't even mention DMs and yet there's SE employees scrambling to rescue her from the "trolls", who actually did nothing more than replying to her tweet to explain the post respectfully. If SE keeps doing that to become a brand, they will lose the users who made them the valuable product they're. All they will be left with will be the people who like to create fake controversies – NSNoob Oct 18 '18 at 14:25
  • 70
    @TheLethalCoder until we all collectively quit, SE employees will continue to act the same way towards us (as users) as they have in the past. That is to say, condescending and disrespectful. The bright side of seeing SE folks on Twitter is... well it's unfiltered. But the attitudes there reflect how I've felt that SE feels about me as a user for years now. – enderland Oct 18 '18 at 15:45
  • 227
    This is hilarious. During years of asking for changes to and/or mod tools for managing HNQs (e.g. during "Second Iteration of the Stack Exchange Quality Project" which ... just died), nothing happened. For Computer Science, the feature is, plainly spoken, crap, and we don't get any good traffic from it (afaict). Removal of the site from HNQs would have been an attractive option, even. But some Twitter outcry gets things moving in under an hour? Come on. – Raphael Oct 18 '18 at 16:49
  • 224
    I've downvoted this because users have been asking for years to make changes to HNQ and you've done nothing, while a single tweet is enough for you to take action. Way to spit in the face of the SE communities. – Stijn Oct 18 '18 at 19:21
  • 187
    @TimPost I mentioned this in TL (♦ only) and should also say here: One of the breakdowns here is that community members have been asking for change to this feature, and pointing out problems with disturbing titles, for years, but have been consistently met with silence and inaction. But it took one person tweeting and now there's suddenly lots of action. Why did no internal community feedback ever receive this response? That's worth introspecting on as staff, because there's an untenable problem to be found there. – doppelgreener Oct 19 '18 at 12:19
  • 123
    I read the related Twitter thread yesterday. Seeing how unreasonable and toxic the poster was being towards other people trying to calmly explain the situation at hand, I just muted that profile and hoped no one else would fall victim of a disrespectful exchange from there. Seeing that the company chose to listen and drive this entire Meta discussion out of that only makes me sad. – E_net4 Oct 19 '18 at 13:16
  • 137
    I'm also downvoting this post despite agreeing with the need to work on HNQ because A) I really don't like how this just kicked a whole site without any real notice B) Ignores SE users for Tweets (or at least ignores channels to get feedback within SE) C) because you have again taken action then asked for ideas - ideas don't help when SE's minds are already set – JGreenwell Oct 20 '18 at 20:53
  • 82
    Why in the hell are you people listening to Twitter? Congratulations; you've just given the entire world an enormous lever. All someone has to do is put a post on twitter implying that your website is not inclusive or offends someone's delicate sensibilities, and they can get anything they want here. – Robert Harvey Oct 23 '18 at 16:12

67 Answers 67

457
+750

The PR and optics of this discussion could not be worse and less fortunately timed if someone tried to orchestrate it that way.

Some things happened yesterday that caused a need for us to (quickly) remove a site's eligibility to contribute to the list of hot network questions.

What happened was that someone called SE out on Twitter for something you could conceivably see as problematic (two questions with out of context bad titles showing next to each other in that list). After that, not only was that change done within 40 minutes of it being pointed out, this happened after MONTHS of engaged users of that site asking for the HNQ to be adressed.

Yet, this happens only after Twitter outrage from non-users of the site. Why is that? Even if you have the very best of intentions and had this cooking internally for a long time (which I'm going to just assume for the purposes of this argument - good faith and all), this couldn't possibly have had less fortunate timing.

I'm not trying to rag on Stack Exchange for doing this, but why was such a massive change made without consulting, collecting feedback from or even notifying the site's active user base? Why does an engaged user of IPS have to visit twitter of all places to find out SE has cut out more than half of their site's traffic overnight?

Why wasn't the community consulted on this? We had discussions on it before, a lot of people came down in favor of restricting IPS from showing up on the sidebar in some fashion or another, and now we get this. No feedback, no discussion. Someone that apparently SE wants to placate made a stink on Twitter, and somehow that's more effective than months of constructive reasoning in driving change. What reason, if at all, does an engaged user of the site have to trust the community governance model with this?

If it sounds like I'm really annoyed by this its because I am, yes I was in favor of removing IPS from HNQ before, but the circumstances under which it happened is making me lose all hope I have for SE's leadership's ability to formulate concrete plans to make changes constructively.

  • 30
    It's possible that they've seen those posts, and that tweet was just the "last straw". But SE needs to at least comment on those requests indicating that they're being considered. Not doing so makes them seem like they were ignored altogether, and makes the outward situation you pointed out happen. – Sonic the Inclusive Hedgehog Oct 17 '18 at 18:16
  • 51
    @SonictheInclusiveWerehog Exactly. Even if that's the case, they should have learned at least the tiniest lesson from the very recent and very messy Twitter debacle that went exactly the same (where it was allegedly also a problem they've been working on for much longer than the Twitter drama suggested). – Christian Rau Oct 17 '18 at 18:22
  • 64
    FWIW, your hope in SE's leadership seems to be holding out longer than many other peoples'. – BJ Myers Oct 17 '18 at 21:11
  • 26
    I feel bad for the veteran users. – user202729 Oct 18 '18 at 6:13
  • 163
    Twitter-driven development, "Sites are excluded only after a twitter demand, from someone with at least 1k followers. That's how it works. Feature requests here regarding those things are 100% ignored." – gnat Oct 18 '18 at 7:39
  • 18
    @SonictheInclusiveWerehog If that were the case, why wasn't IPS alerted earlier that the straws were almost gone, so to speak? Why stay silent? – JAD Oct 18 '18 at 11:27
  • 177
    Bah. These weren't even "out of context bad titles" at all; they were perfectly reasonable interpersonal problems to ask about. You accepting the idea that there's something inherently offensive about men asking how to avoid or deal with unwanted sexual attention is ceding far more ground to the feminists than we ought to be; the company blocking an entire site from network exposure for permitting such material is pandering to bigotry. There's nothing wrong with somebody of either gender discussing such things. – Mark Amery Oct 18 '18 at 13:26
  • 23
    We've been asking for something to be done with the HNQ for the better part of a decade over on Workplace, so... while months is fun, we're sitting back here amused you're still in the first year of lamenting the HNQ... – enderland Oct 18 '18 at 14:53
  • 108
    But seriously though, this is a giant slap in the face of the actual contributing users to Stack Exchange. – enderland Oct 18 '18 at 15:24
  • 48
    @NicolBolas I don't see it. Neither question title that caused the offence here was "how do I convince my coworker to leave her beta boyfriend for me?"; one was a man asking how to avoid uncomfortable sexual attention, and the other was a man seeking help to deal honourably with a third party's infidelity. (Going by the titles alone. I haven't read the posts themselves, but their content is irrelevant to this discussion anyway.) While I'm not going to pull up /r/redpill right now in the middle of my office to check, I'm pretty sure that if I did I wouldn't see questions anything like either. – Mark Amery Oct 18 '18 at 16:07
  • 74
    Ah yes, "We've made these changes in a way that seems erratic, arbitrary and motivated by some non-users being meanies on twitter. But I can assure you this has been long coming and we've been discussing this internally for a very long time". Where have I heard this before? Oh yes, here on MSE. And this is the good-faith scenario. – Andras Deak Oct 18 '18 at 21:18
  • 31
    @user202729 That's not the correct way! The correct way, if you want change, is to complain on Twitter! Meta discussions do nothing. – forest Oct 19 '18 at 3:40
  • 23
    @Narusan questions tagged with discussion get exactly that: a discussion. Magisch is discussing how the company has been consistently taking a dump on multiple communities this year. Would you prefer a strict answer to the question, one that can be ignored the exact same way (and has probably been posted on many of the relevant per-site metas)? The communities are getting really fed up, hence the votes here. And the question is at +100/-68 right now. I believe this answer is perfectly adequate and constructive. – Andras Deak Oct 20 '18 at 10:30
  • 29
    "Yet, this happens only after Twitter outrage from non-users of the site. Why is that?" Because SE no longer cares about how they treat their use base. The welcoming push and "New Contributor" flag have made that clear. – jpmc26 Oct 20 '18 at 19:40
  • 17
    @WebHead Whether an external person has valid concerns is unrelated to whether SO is treating their most engaged veterans with respect or dignity. They are two completely separate issues, especially when the community's similar concerns fall on deaf ears. Besides that, having "welcomingness" as a core value is fundamentally stupid. SO cannot be welcoming to people who post low quality content and simultaneously be a site filled with high quality content; they are mutually exclusive. SE is not perfect, but how the community treats you is more often than not based on the quality of your content. – jpmc26 Oct 21 '18 at 19:46
344
+50

First and foremost, I don't understand Tim Post's reticence to say which SE post or posts were "responsible" for the Twitter spat. Which, if I understand correctly, lasted a relatively short time.

The tweet that started the controversy was this

stack exchange: the #1 site for your questions about dataframes and female treachery

enter image description here

The second tweet, which I believe is the one that caused the SE team to internally groan was a retweet by a female developer.

When people seem confused about why Stack Overflow might not be the most welcoming/comfortable place for people to find answers to programming questions, show them this

enter image description here

She then tweeted this

Cool ableism on the front page of a website for dev questions
“How do you tell a Facebook friend that they might be on the autism spectrum?”

The accusation that SO/SE is not inclusive was retweeted... once. Yes, it spread like extra mature cheddar.

The SE developer who removed the site from the HNQ list (Since when has the term "flirting" been unethical?), replied [emphasis mine]

Woof, yeah. I agree that's entirely inappropriate for SO. I just pushed a change to remove questions from that site from the Hot Network sidebar. I'll ping folks internally to review the site's content more closely as well.

Now, here's my main point. If I had been that developer, I would have DEFENDED that question and EXPLAINED that the question is not taken from Stack Overflow, but from a site called Interpersonal Skills. I would have also explained the site's scope and informed the twitter user their sanctimonious reflection and supercilious attitude was based on false presumption. Well, maybe I would have refrained from using the terms sanctimonious and supercilious to soften the tone.

The same twitter user (Twitterer?) point blank refused to engage in any civil discussion when they realised that their high horse didn't have a leg to stand on.

In other words, the management's reaction and the action taken was excessive and unnecessary.

Bring back IPS to the HNQ. They did nothing wrong.

Disclaimer: I am no longer a member of IPS, I left that community several months ago. But I bear no grudges to its users, many of whom I secretly admire.

  • 172
    @ZachLipton people on Twitter were polite, they explained the person's error and hasty judgement, but she rudely dismissed them. It was rude. She didn't engage at all. If I want to express an opinion on the Internet, that's my choice but I have to accept that I am open to criticism and/or correction(s) from outsiders. Just as I am now replying to your comment, acknowledging your dissent and trying to explain myself. – Mari-Lou A Oct 18 '18 at 9:16
  • 10
    Strangers on the internet do not have an obligation to engage with all comers or anyone at all in fact. She dismissed people who, instead of participating in the discussion on Meta, went to go bother her on Twitter because they were upset at about SE community management decision. Blame SE for its actions all you want, but don't take this out on a Twitter user who you insult and demand must have a good attitude after she's had 10 different people show up in her mentions to argue about it (in tones ranging from civil to downright rude) when there's a perfectly good discussion here on the site. – Zach Lipton Oct 18 '18 at 9:26
  • 12
    It looks like the Tweets are in the wrong order: The retweet of 'normal website' (that got a reply) happened before they added their own tweet about the autism question... Also, very much appreciate the sentiment of your post. As the person who wrote the Tweets has been very clear in not wanting to engage further, I'd like to conclude by asking a small favour from everyone here and respect that wish by focusing their energy on constructively cleaning up the mess and moving on, here and on IPS meta :) – Tinkeringbell Oct 18 '18 at 9:40
  • 34
    @ZachLipton where am I discussing the issue, here or on twitter? Which I wouldn't even be able to do b/c I don't have a twitter account. One social media account is more than sufficient, but I see many people who have several accounts, etc. and some are more eager to express their opinionated viewpoints than others. You are criticising my position just as I am criticizing her reaction, but my main beef is with the mod who removed the IPS post from HNQ because of one tweet. A post that was not in the slightest offensive or disrespectful. A post that SE should not be ashamed of. – Mari-Lou A Oct 18 '18 at 9:51
  • 120
    The problem is that a single moaning tweet from someone who doesn't even use the site got something to happen almost instantly, whereas this discussion has been going on for years on this own site and nothing has happened. That's what people are upset about. – Pyritie Oct 18 '18 at 11:28
  • 65
    It's twitter. It's designed to filter nuance from any discussion topic. The more troubling thing here is that SE devs apparently care about the opinions of non-members on twitter more then about meta consensus. – Magisch Oct 18 '18 at 11:30
  • 116
    About that "ableism" tweet. That tweet is actually doing exactly what it accuses the question of. How about some self reflection on that user's part. I'm autistic and I find the tweet significantly more offensive then the question could have ever been. Do I get to remove that user from twitter now? I don't think so. – Magisch Oct 18 '18 at 11:32
  • 52
    @Pyritie it's worse than that, an entire site has been made the scapegoat for SO's poor reputation online. It's an absurd and ridiculous situation, the tweet should never have been taken seriously. As for fixing HNQ, if you get rid of it, sites with a dwindling user core base will inevitably suffer. – Mari-Lou A Oct 18 '18 at 11:32
  • 13
    This information (what happened) should be in the question rather than an answer. I tried to edit it in, but the edit was rejected. Maybe i should use my twitter account to suggest the edit instead.. – De Novo Oct 18 '18 at 18:48
  • 13
    @E.P.that's an exaggeration, saying I am repeatedly asking IPS to be reinstated. As for the meta post you provided a link, a cursory glance tells me it was posted over a year ago and the community did not seem to reach a unanimous decision. As for moving things forward, the decision taken by the SE employee was bad, made in good faith no doubt, but bad all the same. – Mari-Lou A Oct 18 '18 at 21:40
  • 31
    @NicolBolas that's not entirely true. In Adam Lear's tweet: I'll ping folks internally to review the site's content more closely as well. This heavily implies that IPS should have done something different in this case. – JAD Oct 19 '18 at 17:05
  • 21
    @NicolBolas so why were the question titles described as being "entirely inappropriate for SO"? Why has IPS been excluded from HNQ? Why was there no pre-warning, why did no one have a word with the IPS mod team, and ask them to monitor dubious question titles? Why was action taken so swiftly without consulting the community? Obviously, they felt the earnest need to repair the damage, but there was no damage in the first place. – Mari-Lou A Oct 19 '18 at 17:33
  • 27
    @TimPost You're comparing a few question titles on IPS to videos that glorified sexual violence. That's a bit extreme. If the problem had long existed ,SE's responses didn't seem to acknowledge it. Or maybe they did? Can you please show me responses from SE management that answered or responded or explained or defended the several questions and issues raised about HNQ over the years by users. I have tried looking, but I probably have missed them. – Mari-Lou A Oct 22 '18 at 12:20
  • 21
    @TylerH How is using the pronoun "she" and describing her as a "female developer" slandering her? I think her sex is pertinent, she claims that SO is hostile to women computer programmers. That was why, she claims, she posted the IPS titles in the first place. Quote: …annoying mansplaining stack exchange mods in my mentions while I was offline. Helpfully telling me why I'm so wrong about their precious forum site being unwelcoming to women – Mari-Lou A Oct 22 '18 at 15:44
  • 14
    @TylerH ok, lets back track a bit. Where did Mari-Lou start labeling people. Please, cite examples. On this answer I see only descriptions of what happened and later how it made feel the author. In no way, I see metaphors that implies generalization. I see that Mari-Lou properly abstracted itself from the events it was describing for the audience, until it decided what was its issue with the handling by the staff. – Braiam Oct 23 '18 at 0:59
296

The HNQ implementation has at least three problems:

  • It optimizes for controversy, not quality. Sensationalist questions draw lots of rapid response, which feeds the HNQ algorithm, and then once it's on HNQ it gets even more rapid response, which keeps it there (and also distorts Q&A on that site).

  • It's completely automatic; sites have no way to intervene and remove a question short of closing it. But closure is frequently the wrong answer; the question is perfectly valid, but controversial.

  • Everybody sees it, all the time. If that list on the side of the page has even one or two "exciting" questions, we can pretty much guarantee that some readers will be upset. (There is a regex filter on titles, but it comes with all of the usual regex challenges.)

If we want to have something like HNQ -- and I've heard enough people on smaller sites saying they value the promotion to go with that for now -- then we need to find a way to let communities curate the content. We should also consider user controls for those who need them. (I get it; I don't want my coworkers to see that question about Klingon mating rituals either.) We should also adjust the algorithm so it doesn't so strongly prefer controversy.

Here are some ideas along those lines:

  • To aid curation: provide a way, from the question page, to see if this question is currently hot or likely to become so. If automatically displaying that would be an expensive operation, let's add a way to check. Perhaps checking the hotness score and comparing to the minimum score from the HNQ is good enough; we don't need to actually do the lookup.

  • Automatically exclude questions that the community has signalled have issues: protected questions and questions with two or three close votes. (Probably we should make that number adjustable.) More thoughts on this here.

  • Give moderators and high-rep users a menu item for "exclude from HNQ" so they can be proactive.

  • Suggested in comments: allow communities to review hot questions (also suggested on other answers here)

  • Factor downvotes into the hotness formula.

  • I don't have concrete suggestions here, but let's review the velocity part of the formula. Lots of answers isn't necessarily a good measure. Lots of quality answers might be, but that's hard to measure. But let's factor answer score and length in more and number less.

  • Allow users to (persistently) collapse the HNQ, and start in the collapsed state.

As noted in comments, we should also review the effects of visitors bearing association bonuses on hot questions (should they be able to vote? comment?). I've been focusing here on how questions get (or don't get) to HNQ visibility, and with all the voting that's already happened I don't want to expand the scope of this answer now. Let's use other answers for that part of the problem.

  • 19
    I've seen at least one case where a site closed a good, on-topic question to remove it from HNQ, then reopened it once it became 30 days old (and thus ineligible to appear there). – Sonic the Inclusive Hedgehog Oct 17 '18 at 17:59
  • 6
    I particularly like the recommendation allowing moderators to exclude certain questions from the HNQ. This seems like it would be a useful stop-gap until a revision of the formula could be worked out. – James Oct 17 '18 at 18:37
  • 147
    +1 The HNQ absolutely optimises for controversy. On RPG.SE the effect of a HNQ is often also pretty bad: if we've got a controversial, difficult topic it needs careful community cultivation, expert answers, and highly knowledgeable voters more than ever... Then it hits HNQ and we see an influx of dozens of votes upvoting poor advice in exponential proportion to whatever the scores already were, amplifying the fastest gun and drowning out up-and-coming better advice completely. That effect is a disservice to everyone and degrades the quality of help RPG.SE provides. – doppelgreener Oct 17 '18 at 18:50
  • 63
    @doppelgreener: Philosophy has this problem as well. Controversial questions that have "God" in their title are regularly on the HNQ, even if they are ill-stated, unclear, etc. and attract a high number of opinionated comments and answers of which two-thirds have to be deleted. The only other things popping up there are similarly bad questions on evolution and quantum mechanics/free will. I have yet to see a well-stated question on philosophy in this list. Yes, it helps with traffic for smaller sites. But I doubt that it is traffic we want to have on these sites. – Philip Klöcking Oct 17 '18 at 21:46
  • 74
    Would +1 again for 'factor downvotes into the hotness formula' - indeed, factor them heavily. And the same with close votes. A question on the list gets three close-votes? Take it out of the sidebar before it blows up, and give it time for the site community to put it into shape, and once it gets fixed put it back on the bar if you want. – E.P. Oct 17 '18 at 22:49
  • 9
    I like this except for excluding protected questions - being on the HNQ list is often when you most need protection. – curiousdannii Oct 18 '18 at 0:36
  • 15
    @MonicaCellio Right, and I think that would be a mistake. Even good questions that don't overly attract controversy on the HNQ just attract crappy answers. I'd even say it would be good to preemptively protect all HNQs, at least temporarily while they're on the list. – curiousdannii Oct 18 '18 at 0:40
  • 4
    @curiousdannii one advantage of my approach is that if a hot question gets those bad answers and the community deletes them, the question will be auto-protected and thus kicked off the HNQ list automatically. On the other hand, "well-behaved" hot questions (I've seen some) can remain without pre-emptive protection. (SE has in the past pushed back against pre-emptive protection; that's why community members can't do it until there's actually a qualifying deleted answer.) – Monica Cellio Oct 18 '18 at 0:44
  • 38
    +1. I'd just change one thing: not only mods but also other high-rep users should be able to (vote to) exclude a question from the HNQ list. There are only so many mods to go around. I'd see anyone with at least 10,000 rep as understanding enough about a site and having seen their share of HNQ blowups to meaningfully vote on this. (Giving this right only to mods might be easier to implement.) – Stephan Kolassa Oct 18 '18 at 7:27
  • 7
    One way to handle the discoverability of a collapsed-by-default HNQ list would be to add something like the placeholder Watched Tags box with text along the lines of "See great questions from some of our other nice sites!", with the added advantage that you can then link directly to a customizing checkbox list that puts the which-sites-do-you-want choice right in front of new users. – E.P. Oct 18 '18 at 15:36
  • 7
    @E.P., most of the sites with an HNQ problem don't have any non-mod 30k users. IPS has one. Politics has one. Christianity has three. Philosophy has two. Skeptics has six. The only sites likely to benefit from making this a 30k privilege are Roleplaying Games (28 users), Arquade (18), The Workplace (23), and Worldbuilding (22). – Mark Oct 18 '18 at 20:55
  • 11
    I would also like to see a method for mods and high rep users to promote a question to HNQ. Some good questions just don't quite get the views/answers they need to make the list. If X votes from the upper rep users of a site can push a question to HNQ, and if that can trump hotness score, then that could help promote the content the site wants to see out there. – Kendra Oct 18 '18 at 21:09
  • 85
    These are great suggestions. Someone should post them on Twitter. – Hong Ooi Oct 19 '18 at 5:47
  • 34
    This is definitely 100% on point, optimizing for controversy / clickbait is hugely dangerous on the SE network. I wrote about this in 2012, same thing applies stackoverflow.blog/2012/01/31/the-trouble-with-popularity – Jeff Atwood Oct 19 '18 at 21:46
  • 8
    @Lilienthal timing of the question, answers, and votes all factor into the formula, as I understand it -- a new question that gets five upvoted answers in the first couple hours is almost certain to hit HNQ, for example. We should stop counting that kind of speed as a positive; on some sites, if a question gets that many answers that quickly it's because the question is either too broad/opinion-based (which can be put on hold) or controversial and many of those people are soapboxing. A delay before a question could even enter the HNQ would help a lot; it lets the community respond first. – Monica Cellio Oct 21 '18 at 0:36
264

As a user

I love the HNQ list because I really enjoy learning about all manner of diverse topics. A jaunt through the HNQ can see me hopping from Christianity to Physics to Parenting to Aviation to Skeptics to Worldbuilding and so on and so forth. Without the HNQ I wouldn't put in the effort to check for interesting questions on two dozen sites.

For a site

For some beta sites in particular, the HNQ very often brings in a great deal of that site's traffic. Removing an entire site from the HNQ necessarily means taking a road to that site and turning it into a trail. I don't think isolating this or that site for any reason is in keeping with the spirit of the Stack Exchange network.


All that to say: please don't remove the HNQ list from the sidebar. There are certainly ways to improve it and I'm all for that; I'll let others cover that.

  • 92
    This, big time. The HNQ is a source of absurdly great reading on any given day. Please keep it if you can. – vektor Oct 18 '18 at 7:01
  • 68
    As a possible addition to this post: one of the sites that I most enjoy reading questions about on the HNQ list is the one that has just been taken off it. – John Gowers Oct 18 '18 at 12:07
  • 1
    As a user, at work I often get lost in following those links from a SO question where I looked up something, so there I would like to hide it. (I used to have a stylish script for this, but stylish was then banned, and I didn't set up an alternatives.) At home I often enjoy just clicking from hot question to hot question. – Paŭlo Ebermann Oct 18 '18 at 21:12
  • @PaŭloEbermann Yeah, I've seen that sentiment (HNQ being distracting on SO is undesirable) in a couple other answers and comments, which suggests that perhaps it should be hidden on SO specifically. But on the other hand, SO is likely the biggest source of traffic via HNQ (it would be nice to see actual numbers on this!), so there's a bit of a Catch-22 there. – El'endia Starman Oct 18 '18 at 21:53
  • 11
    Just allow a user to fold it away, and remember this in a cookie (not a user account), that should be enough. – Paŭlo Ebermann Oct 18 '18 at 22:15
  • 21
    +many: HNQ-discovery is how I landed on the site I now moderate. I hope the idea of "a selection of interesting stuff from around the network that you might not otherwise see" never goes away. – nitsua60 Oct 19 '18 at 1:01
  • 1
    I was just about to add an answer to say exactly this - it would be a great shame if this feature were to go away, and it is a shame that IPS is now excluded from it. – Mark Hughes Oct 19 '18 at 11:16
  • 7
    Totally agree with this sentiment. Why not let users pick which sites' "hot" questions show up? Then you know what you're signing up for. I loved IPS, just as I love Workplace, just as I enjoy seeing interesting Math and DBA content. It would be similar to ignoring tags on SO (except they wouldn't be highlighted, they just shouldn't be in the list) – OverMind Oct 19 '18 at 15:52
  • 3
    Thanks to HNQ, I have joined sites I otherwise wouldn't have known about and ended up contributing to either the hot question itself or other questions on the site because of this. So I agree 100% with this answer: HNQ should stay. – Robotnik Oct 23 '18 at 6:14
159

Let's stop over-promoting some sites at the cost of other sites rarely making the list at all.

The network has more than 170 sites. Some of them are small and rarely make an appearance on HNQ at all. Others are large and active and can have five or more questions on the list (of 100) at once. That's working against the goal of broadening people's horizons and, on those small sites, can feel kind of crappy (what do we have to do to get some attention already???). Let's fix that.

There are 100 questions at a time on the HNQ, with a random subset shown on each page load. 100 < 170+. Therefore a site's "fair share" of the HNQ real-estate is <1 -- round up to 1.

No site should have more than one active HNQ at a time. Yes, that means that a question might need 50 hotness points on SO and 5 on Astronomy; that's ok. Show us the best hot Astronomy question anyway. And cycle them more quickly, so it's not the same Astronomy question that was there yesterday.

I know that on some sites (most notably SO) the HNQ gate rises with each question already on the list (it's harder to get HNQ #3 than it was to get HNQ #2). I'm proposing a simplification of that: each site gets one, period, and it shouldn't stay there for a week like some HNQs do now.

  • 9
    I think that's a good idea overall. I have nothing against limiting how often TWP, PPCG, and IPS get on HNQ, since they are most definitely overrepresented. But Only one no matter what forever? I think that's too harsh. Maybe limit it to 2-3? Or perhaps even increase the total number of questions that can be "on the HNQ" – DJMcMayhem Oct 17 '18 at 20:29
  • 78
    +1 for "cycle them more quickly"! If a questions does not interest me today, it won't interest my tomorrow or in a week – aloneprism Oct 17 '18 at 20:33
  • 4
    @DJMcSpookem I won't object if they set the limit at two rather than one. I proposed one to give smaller sites a chance to show up there at all. – Monica Cellio Oct 17 '18 at 20:34
  • 3
    The problem I see is that some sites get less than one question per day, so I'm not sure if they're getting enough questions to have something new and interesting for the list at the frequency that you're suggesting. – Laurel Oct 17 '18 at 20:41
  • 3
    @Laurel those sites would then be among the 70+ that aren't currently in the HNQ list, then. When a small site does get a good new question, though, it can actually hit HNQ, which rarely happens today. I don't think we have dozens of sites in the category of "less than one question per day". – Monica Cellio Oct 17 '18 at 20:45
  • It's actually less than 100... if I remember correctly, only the top 50 of the full 100 list ends up in the sidebar, which is where I'm guessing most of the traffic comes from. – Catija Oct 17 '18 at 21:06
  • 4
    Monica, you can find some more details on that issue here, there is a little analysis based on snippet of stats shared by Shog a while ago, "top 5 sites (Workplace, SO, SFF, Worldbuilding, Code Golf) get 50x to 200x(!) more clicks than bottom 10 (Hinduism, Android Enthusiasts, Latin, Chess, French, Law, Board Games, Russian, Mi Yodeya, Christianity, Motor Vehicles). It gets even worse if you take into account that mentioned snippet lists only 70 sites of total 130+, meaning that about 60 sites get even less views..." – gnat Oct 17 '18 at 22:58
  • 2
    Thanks @gnat. Clicks are affected by both the presence of the HNQ entries in the first place (you can't click on what you don't see) and interest. I'm not surprised that more people are interested in some questions from SO or Workplace than they are in ones from French or Latin. Do you have any data on distribution of HNQ questions, as opposed to clicks, so we can isolate these variables? – Monica Cellio Oct 18 '18 at 0:00
  • 2
    @Catija In my experience that's not the case. I've had several borderline HNQs that got (relative) spikes in traffic while never getting into the top 50, and I'm pretty sure I saw them on the list. – Nathaniel Oct 18 '18 at 1:19
  • one of MSE regulars collected fairly representative sample of distribution data and reported their findings here and here. Back then I briefly skimmed it and there were some interesting differences supporting your idea that clicks and distribution are not the same. It felt a bit troublesome that clicks seemed to depend too much on distribution. Kind of self fulfilling prophecy when clicks are mainly caused by questions presence in the sidebar and not by them being genuinely interesting – gnat Oct 18 '18 at 9:01
  • To make a parallel, this answer is going for the US Senate approach of "X representation per community, no matter the population(/activity)." Is this really a good approach? Small communities may have very few questions, and of them quite possibly no great candidates, while a large site might have many to choose from. Should our metaphoric California get the same number of questions survived as North Dakota? – Nex Terren Oct 18 '18 at 17:42
  • 2
    @NexTerren fortunately, the HNQ changes its representatives more frequently than the US Senate does. Also, the SE equivalents of DC and Puerto Rico get representation. And there are only 100 slots, so if a site doesn't have any good questions at the time, somebody else gets that slot. We can still set minimum thresholds (if we can decide what they should be). One of the points of HNQ is to get exposure for some of those small obscure sites. Because they're small, they don't already have tons of votes -- but that doesn't mean the questions aren't good. – Monica Cellio Oct 18 '18 at 17:49
  • 10
    I love this idea. To take it a step further: if the list doesn't need to be an arbitrary 100 questions long, it could be changed to be 3N questions long, where N is the number of sites in the network. Then HNQ could always be the "hottest" (by whatever measure) 3 questions from each site. The sidebar could then show a selection chosen uniformly randomly from that list. – trichoplax Oct 18 '18 at 22:19
114

Just to start, I'm going to run down a few of the problems I have with the HNQ, as it currently stands:

  • It does a terrible job of promoting good questions from the really small sites that could use the exposure the most. I can count on one hand the number of HNQ questions Astronomy has had within the last three months, I think.
  • It's optimized for hot questions, not good questions. There's a difference, and arguably, question quality and activity can be orthogonal in some cases. I'll claim that the very best questions don't get 10 answers within an hour. The problem is, what makes a question good can be hard to objectively determine.
  • It doesn't look at the actual content of the questions. At all. It's blind to what topics are represented, and that, too, can be a problem, skewing the HNQ questions of a given site to certain subsets of its scope.
  • We, the users, have essentially zero control over whether a questions makes it to the HNQ or not. We don't have control over how a prime tool for evangelism and outreach is used.

So, here are some point-by-point solutions:

  • Guarantee every site, no matter what the size, one HNQ question per period, by essentially adjusting the requirements on each site for a question to make the HNQ. And I mean making big adjustments to thresholds, whatever they may be.
  • Maybe let the community choose certain questions to get more attention - i.e. let each site create a "Best of" list of questions from the past X days they might want promoted (maybe with X=14?). I'm thinking of some sort of voting process on meta. An issue with this, of course, is that it probably requires some manual work, but - and correct me if I'm wrong, since I've got no technical expertise here - we could use a system similar to the one that determines whether community ads are displayed.
  • Urg, I can't really think of ways to get around some topics being overrepresented among a site's HNQ questions. Maybe that's not an issue?
  • See the second bullet point, here. I'd love a "Best of" list, with a couple questions to be shown on the sidebar every week.

Basically, I care about the HNQ bringing attention to sites that need it, and doing a good job of representing site scope. By ensuring representation among all the sites - or as many as possible - we can cultivate growth. Heck, the HNQ is how I found Worldbuilding four years ago, a site I now moderate. I also want more control over what questions we feature, which is why letting sites collectively pick the best questions from the past 1-2 weeks would be nice.

  • But if they do all that will the questions be hot? Would anybody want to click the links? – user000001 Oct 17 '18 at 20:55
  • 1
    @user000001 I'm not quite sure what you mean; can you please elaborate a bit? – HDE 226868 Oct 17 '18 at 21:15
  • I was talking about the proposals for having users select the hnq questions, and about the equal representation if all sites. I'm afraid that the selected questions wouldn't be as interesting as they currently are. But now I noticed that they're going to remove it completely, so anything is better than that :( – user000001 Oct 17 '18 at 21:20
  • 8
    I think this is a good solution. If the goal is to get eyes on other network sites, I'd much rather have good questions than questions that may be a flash in the pan. – James Skemp Oct 17 '18 at 21:57
  • I think regular voting should be good enough to select good questions. Show questions that exceed a site’s median vote count by a certain proportion. I like guaranteeing that every site gets at least one question on the list every so often. – Cris Luengo Oct 18 '18 at 1:08
  • 6
    @user000001 Content should be "good", not "optimized for clicks". I firmly believe such a focus is better for everyone over the long term, including raw business/traffic for SE. – Wowfunhappy Oct 18 '18 at 1:09
  • 9
    Exactly. Optimizing for clickbait is one root cause of the problem here. If a community legitimately believes that a question is one of their best questions that highlights what they're all about, that's something that should be showcased more than a question that happens to generate clicks because it's salacious or eye-catching. – Zach Lipton Oct 18 '18 at 5:43
  • 2
    Guaranteed slots will cause problems with slow sites such as Community Building or Windows Phone: the single-digit-per-month question rate means that every upvoted question on those sites will show up on HNQ, and will spend considerable amounts of time there. – Mark Oct 18 '18 at 22:18
  • 1
    @Mark That could be a problem; there definitely needs to be a workaround in cases where the site get's ~1 question per week. Some people have suggested promoting old (or unanswered) questions, which could be a nice substitute in those instances. – HDE 226868 Oct 18 '18 at 22:46
  • 2
    @Mark could the time-frame from which HNQ candidates are drawn be scaled to question-rate? I.e. on Mathematics (huge site) HNQ draws might come from only the last 24 hours; on sports (1.3 Q/day) the draw comes from the last month? 0.1 Q/day... I dunno. Not exactly sure how that site exists, to be honest. – nitsua60 Oct 19 '18 at 1:11
  • Perhaps squareroot(Q/day), or erf(Q/day) Something monotonic, smooth, but still less-than-linear. – uhoh Oct 20 '18 at 11:23
108

Please, please get rid of the clickbait

When you say this

they've been sufficient for ensuring [...] overt clickbait was kept out of the list for the most part

y'all are just utterly fooling yourselves and/or burying your heads in the sand. HNQ absolutely selects for clickbait. It's what makes it tick — it wouldn't be what it is right now without it. And it's what fills it with terrible posts instead of picking stuff from the vast barrels of useful content on the network.

And, frankly, it's an absolute shame.

  • Stack Exchange shines because of its quality content.
  • Clickbait just isn't quality content. (And even if it was quality content originally, if you just slap a clickbait title on it and show it to enough people, that deluge of traffic that's uninterested in the real qualities of the content is just going to skew it so much that it ceases to be.)

So what on Earth is clickbait doing on Stack Exchange? We've been calling the sidebar the SE-Wide Advertisement for General Enlightenment for some time now.

(Which isn't to say that it is always dreadful, and indeed I'm not above hijacking some of the HNQ mechanisms if it helps produce an effective piece of science communication. But it's a mistake to put the cart before the horse, i.e., to emphasize the make-HNQ-do-good-things aspect above the don't-let-HNQ-do-awful-stuff part.)

The key problem with HNQ that causes it to select so strongly on clickbait is its intrinsic dependence on a positive-feedback loop:

  • If a question is on the list, it gets a bunch of traffic which brings with it a bunch of upvotes for the questions and the answers.
  • The 'hotness' score that determines whether a question stays on the list is a direct measure of how many upvotes it gets.

The clickbaitier the title, the more traffic, the more upvotes, the higher the question and answer scores, the higher the 'hotness' scores, the longer the question stays around, the more the traffic, and round and round it goes until the votes saturate. You're basically trying to fight a positive exponential with an algebraic time decay, and if you were paying attention in math class, you know who wins that fight. And, moreover, the exponent on that exponential is basically just how clickbaity the title is. Way to go.

This isn't a new observation, either — by sheer coincidence, Breaking the HNQ feedback loop on bad questions is on the brink of turning two years old, with zero attention or engagement from the SE community or dev teams. If you guys had turned up and said "but perfooooormaaaaance" at the time, then we'd have had the opportunity to convince you that you're wrong about that.

Anyways, I stand by my proposed solution to the positive-feedback problems from two years ago:

Let association-bonus users vote on HNQs, but don't count those votes and views into whatever algorithm does the decision that a question is Hot.

Cut the feedback loop, and see the exponential die out.

And of course, since you've spent a great deal of time detailing how performance constraints are a real solution-buster in these parts, let me spend some time detailing how those constraints don't apply to this solution. The modified-hotness score is indeed a much more expensive construct, absolutely, but here's the thing:

You only need to calculate the modified-hotness score for a few hundred questions at a time.

Internally within each site, use the regular hotness score. (Or tweak it, if required.) You only need to start discarding association-bonus votes from the hotness scores after it gets on the list, and those are a manageable set.

This comes down to a point made by gnat earlier: the dev team's attitude over the past few years has consistently had notes of looking at the fanciest way to implement the requested feature, saying "that's not doable", and then shutting down their involvement with the discussion. Instead of, as gnat suggests, looking at the resources you do have and then looking at what can be achieved with those.

  • …fills it with terrible posts… a bit hyperbolic, there have been some very interesting questions that have piqued my curiosity, questions on SE Workplace and Accademia to mention but two. If questions are always so bad, people would stop visiting HNQ – Mari-Lou A Oct 18 '18 at 9:22
  • 6
    Of course it's not "full" of terrible posts in the sense that 100% of what's on the list is terrible. But would certainly put down money on that 90% of the time, 5% of the list is stuff that really shouldn't be there, and it's those posts that persist the longest. – E.P. Oct 18 '18 at 9:34
  • 3
    And as for that last sentence - that's not how clickbait works. The internet is full of sites that use misleading titles to funnel audiences to content that the audience itself knows it probably doesn't want to see. It's a pervasive and well-known problem with the internet. SE generally strives to be better than the average but in this aspect it's dropped the ball significantly. – E.P. Oct 18 '18 at 9:36
  • What would you say if the title was clickbaity, but the content was more than valid? Good questions sometimes need a hook in the title to catch people's attention. SE developers do it all the time! See the cheese question that is still attracting new answers as we speak, have you seen the number of views on that Q? – Mari-Lou A Oct 18 '18 at 9:41
  • @Mari-LouA The cheese question is Featured and not HNQ - it is a straw man. I'm happy to discuss but I won't discuss straw men. – E.P. Oct 18 '18 at 9:45
  • There are plenty of meta posts that use clickbaity titles but whose content is valid and deserve our attention. I'm not going to condemn someone on a smaller site who wants to attract visitors to their question b/c they have created a catchy title. It's the content that matters in the end. Great title but sh**ty content gets closed pretty quickly in my experience. – Mari-Lou A Oct 18 '18 at 9:57
  • 2
    @Mari-LouA It sounds like you either don't understand what clickbait is, or you do understand it and you don't care about its presence, in which case we'll have to agree to disagree. I do condemn someone who is happy to write a mis-representative title in order to draw traffic, and I think that should not have a place on SE. There's a difference between "catchy" titles that are honest about the question's contents even when taken completely out of context (it is possible to write them) and clickbait. The latter should have no place on SE. – E.P. Oct 18 '18 at 10:05
  • OK, maybe my understanding is flawed. Can you please give a few examples of clickbait titles that hit the HNQ and enjoyed unmerited upvotes so I may understand better. – Mari-Lou A Oct 18 '18 at 10:10
  • 12
    @Mari-LouA Try "By 18 years of age, I want a brand new car that's $43,668", or "I slept with my advisor's daughter and she is blackmailing me now. What can I do?" or half of the posts here. But providing a good listing is hampered by the fact that there is no reliable HNQ log (which we've complained about for years). – E.P. Oct 18 '18 at 10:21
  • if I understood correctly, and I won't bother you any further after this, the meta post is implying that one user is making up scenarios in order to win rep. So, it's not just the titles which are suspect but entire posts, and maybe the user themself is "fake". I agree, knowing how many Qs actually entered HNQ would be useful to know, as it is only the number of views accumulated gives some kind of indication. – Mari-Lou A Oct 18 '18 at 10:30
  • If a question is on the list, it gets a bunch of traffic which brings with it a bunch of upvotes for the questions and the answers. and The 'hotness' score that determines whether a question stays on the list is a direct measure of how many upvotes it gets: That can be circumvented: once in the HNQ, no longer consider this question as a future candidate. Together with limiting the maximum period (as Monica C suggested), it will drop out earlier. This may be what you mean with "cut the feedback loop". – Jan Doggen Oct 18 '18 at 12:35
  • 2
    @trlkly Out of curiosity, would you consider a title like "Ten terrible SE question titles that nevertheless made the HNQ list" (where the question contained exactly that) to be clickbait? (Or, to make this explicit: your definition of clickbait differs significantly from any widely-accepted ones.) – E.P. Oct 18 '18 at 14:00
  • 1
    @EP No. I would consider it a listicle, and would consider those also undesirable. That said, I've also never seen those in the HNQ list. – trlkly Oct 18 '18 at 14:02
  • 7
    @GalacticCowboy I don't understand your comment. I am not proposing any changes to who can vote on what. I am proposing that the hotness score disregard any upvotes that come from users who, absent their association bonus, would not be able to upvote, i.e. the same sort of deal as with protected questions - you have to earn 15 rep in the site itself for your upvotes to be counted towards the hotness score. – E.P. Oct 18 '18 at 21:07
  • 2
    @trlkly well, sometimes factual title can also still be clickbait without knowing context... like, How can I kill adorable animals?, which I believe it's also upvoted because of the title (interesting), not because the voters play the game (usefulness). – and myself Oct 19 '18 at 3:22
97

I wasn't going to go out on a limb but since a statement has been made...

The list of questions is generated through a very expensive query, cached, and refreshed on a set schedule. Giving moderators or even community managers the ability to 'evict' a problematic title while keeping the system in check with what we consider performant and sane are .. mutually-exclusive goals.

...I would like to address that.

This is technically inaccurate up to the point of being totally misleading, as was explained in prior discussions, eg here, here, here and here.

Whatever feedback from community / moderators we would want to take into account, reasonably close "approximate" solution can be achieved with negligible (constant time) performance impact.

Specifically, system can first pick larger than 100 (that is 150, 200, 300 etc) amount of "candidate" questions for the hot list using the very same query as it uses now - and this step will be as performant as it is now.

And at the second step, system could take into account whatever feedback / adjustments we would want to make on these candidate questions (fixed amount of them, meaning it's a constant time cost) and filter out 100 final questions to pass to the hot list.


While we're at it, there is another myth that hasn't been mentioned yet but is quite likely to pop up in further discussion, namely that it is prohibitively difficult to track and share meaningful information about questions that get to hot list - such as changes in views, votes, "hotness score" etc.

This is based on a statement that hot questions change way too frequently (every few minutes) so that retrieving, storing, and using much details about 100 questions at such a high rate is not feasible. While compelling on a surface, this reasoning is not really bulletproof because if you think of it, it is not absolutely necessary to catch up at such a high rate to get usable tracking data.

It is entirely possible to get reasonably solid "approximation" by sampling at much lower rate - we could retrieve desired details once in 1, 2, 5, 10 etc hours - and that would give us totally manageable amount of data that would be close enough to use in whatever way we would want.

For example, this "downsampled" tracking data could be used to inform site moderators of questions that likely entered hot list in a few recent hours. This data can be also stored and later used for historical analysis, and since sampling rate is low, this won't be too hard resource wise. Etc etc etc.


Speaking of goals for having HNQ, as of now I don't have strong preferences about this. Main point of this answer is to explain that whatever goal we set here, there will likely be technically feasible way to implement and track things "approximating" it with reasonable accuracy.

You know, it sometimes looks like SE team invents artificially complicated problems as if to justify doing nothing when it comes to hot questions. The starting post of this very discussion seems to be yet another case of that kind. Here I am talking about obsession with problematic titles and how it is difficult to deal with these.

But really, this seem to be missing one simple question, what makes clickbait titles stick so much in the hot list? For those who know how system works the answer is obvious: these titles by their very nature tend to attract multiple answers and the system is currently configured to make it count most.

Titles are totally secondary here and those who pay attention to stuff that gets to hot list can remember examples proving that - specifically "big list" questions (at sites allowing these) easily get and stick into the list despite having fairly bland titles - purely because of many answers.

Two IPS questions that started recent turmoil make good example: one of these was fairly old but system still kept it in the hot list because it had ten answers. This is because question with that many answers can easily have "hotness score" many times higher than one with, say, 3-5 answers: "The benefit of many answers is capped at 10".

  • Visitors of hot questions and maybe especially regulars of the sites that contribute most to the hot list may wonder why system is configured that way, why it keeps accounting for answers up to 10 and not to some other number. To understand that you need to realise that this was set many years ago when hot list was totally different, it was almost exclusively dominated by questions from about handful technically oriented sites. Back then, this parameter didn't really matter much so I guess it was picked without much thinking - and worked fairly well until SE network has grown and much expanded with new topics.

You see, above suggests that thing worth trying is to change that old parameter in hotness score formula from 10 to something like 3 or 4 or 5. This has a good chance to tame artificial domination of clickbait titles. Especially if you take into account that per se, these titles are more likely at disadvantage because many active regulars at hosting sites consider such titles poor and vote down for this reason. It is only that outdated setting in the system that sticks them in hot list, totally inorganically.

Though given how two issues mentioned before were handled I would expect this natural approach to be ignored. Instead we will probably see team wasting efforts on something like inventing regexes to detect possible sexual interpretation of the title or trying to involve target site regulars in cumbersome race to catch and edit this stuff manually before it starts bothering our Twitter overlords. Maybe they will even block other sites besides IPS, "because titles".

  • 14
    ...whatever goal we set here, there will be technically feasible way to implement and track things "approximating" it with reasonable accuracy - so that the only reason that it won't be achieved will be lack of desire of Stack Exchange team to do anything in that regard :) – gnat Oct 17 '18 at 22:38
  • 28
    +10 for your middle section. We've been heading towards HNQcalypse for years, and it's been obvious for a long time that the more we waited, the bigger the trouble would be. Many of us have been yelling as hard as we could for a way to get data on HNQs, to absolutely no avail despite the obvious solutions presented and linked here. And now that the trouble's erupted, we have no reliable past data on what's actually been on the list. The whole thing just blows my mind, frankly. – E.P. Oct 17 '18 at 22:40
  • 10
    Well said. There are certainly technical constraints on any solution, but this needs to be addressed from first principles to come up with what we think is the right approach, and then worry about whether that can be handled within the bounds of reasonable performance. As SE staff has straight-up said, the HNQ system was cobbled together based on what's easy to query rather than an actual design for what's right for the site. Maybe it's worth investing in dedicated infrastructure, if necessary, for HNQ given its apparent importance to the site. – Zach Lipton Oct 18 '18 at 5:56
  • 3
    "many active regulars at hosting sites consider such titles poor and vote down for this reason" - except those downvotes don't have anywhere near the weight that they should have for this. If the formula were changed from Score to something like, say, UpvoteCount - 5 * DownvoteCount then the opinions of those community members could count all that much more. (That multiplier is a bit arbitrary, but remember how an upvote gives you five times more rep than a downvote removes?) – E.P. Oct 22 '18 at 13:22
  • 1
    I'd upvote again for the titles thing if I could, though. – E.P. Oct 22 '18 at 13:22
  • Over sampling and then filtering on a per site basis (no questions from IPS on SO for instance) would seem to make this technically a minor issue. – jmoreno Nov 25 '18 at 15:18
91
+100

I submit that HNQ is fundamentally broken. Why? Because it is about Hot Network Questions. That is, questions which are getting lots of attention.

Just look at your #1 goal: increase site discovery. Questions which get lots of attention rarely represent the best of a site. Controversial questions are "hot", and therefore will attract people attracted to that controversy. But are such people going to become regular users of the site?

No. Or worse... yes. I say that them staying is worse because people who are attracted to controversy are people who will inevitably create controversy (or at least fuel it). And that's not what we're supposed to be doing here.

Worst-case, controversial questions can give people the wrong idea of the culture or scope of a site. This is also why your #2 goal is problematic. Reassessing the scope of a site should not be done due to a mob of people who randomly showed up thanks to an algorithm. It should be done carefully and deliberately.

And all of this comes down to "Hot" questions. If we want to attract the right sort of people to a site, then that can only be done by promoting questions that demonstrate the best face of the site, not merely the one that has attracted lots of attention.

  • I say that them staying is worse. People can come to different insights, maybe especially so if they see what the quality of the site is. – Jan Doggen Oct 18 '18 at 12:49
  • 7
    interesting analysis of implications if we choose discovery as a goal of HNQ, this looks worth a bounty – gnat Oct 19 '18 at 22:48
  • 28
    The fact that "Hot Network Questions" does not correlate with "Good Questions on Other Sites" or even "Typical Questions on Other Sites" does call into question whether users are getting the right impression when they discover a site. – Thunderforge Oct 20 '18 at 2:07
  • 2
    Some sites could end up blocking all external traffic by trying to 'demonstrate the best face of the site'. On Physics (and most science/math/programming sites), the community inside loves the deeper posts which deal with more technically intricate concepts, but the HNQs are usually popular-science which is really basic (basic enough for outsiders to understand; surely you'll empathize) and took authors (including me) very little effort and knowledge to write. To show the best face of the site, I'd try to stop most of that from going HNQ. – Chair Oct 20 '18 at 3:19
  • 6
    @Thunderforge that's a good point, I've seen bunch of comments under other answers where users complained about how hot questions misrepresent their sites. This strongly indicates that current way is poor for discovery. Stack Overflow folks also appear to unhappy about these matters, see eg Why are so many useless questions ranked highly, and vice versa? and a bunch of other meta discussions linked to it. Though it is probably less of an issue over there because system was hacked to push SO questions off the hot list after 7-8 hours – gnat Oct 20 '18 at 9:50
  • 6
    @Chair: "Some sites could end up blocking all external traffic by trying to 'demonstrate the best face of the site'." I trust the users of a site to be able to determine what represents their best face better than an algorithm. And if that best face isn't the popular face... so be it. That allows the community to define itself, not be defined by one random question that attracted a bunch of people one time. – Nicol Bolas Oct 21 '18 at 22:25
  • @NicolBolas Exactly, the community's defining itself and everyone's happy with what's displayed and voted on, but is that what SE wants? That doesn't align with the people saying they want to find new sites where they like the content. – Chair Oct 22 '18 at 12:50
  • @Chair: "the community's defining itself and everyone's happy with what's displayed and voted on, but is that what SE wants?" Um, what? The current system does not allow community's to define themselves; they're defined by an algorithm that, by design, tends to select for controversy, not quality. – Nicol Bolas Oct 22 '18 at 13:23
  • 3
    @NicolBolas Oh, no, I was ambiguous. That stuff about the communities defining themselves is a description of what would happen if the system where communities make a conscious decision to send some questions for HNQs is implemented. – Chair Oct 22 '18 at 13:32
80

I hate sounding pessimistic since it's really not my style, nor do I want to tread in already covered ground that Monica and Magisch have already broached. But I will say this much: I'm not sure that we're really having a "discussion" about this. This is more of a venting period, and then we all move on from it.

As evidenced by...

Suffice to say, the HNQ (as we know it) is now looking at numbered days, but that number is probably kind of high because it's going to take some work to figure out what voids would be worth filling if we yanked it as we know it now.

We're going to talk about this feature before we remove this feature.

Okay.

To be less sarcastic about this, I'll give you an outsider's perspective. I was contemplating joining the IPS community because of the seemingly interesting topics that were being discussed. I've even perused a handful of them and many of the answers provided seemed insightful and useful. Some of those titles wouldn't be ones I would want to click at work, but even then, I'm an adult and I know that the Internet is full of content which may be inflammatory or spark a heated discussion, so I've worked around this by...simply not paying it much attention while at work, and leaving an open tab to come back to it when I have a quiet moment on break.

Kinda like browsing Facebook or Reddit.

And no soul here can say they've never done either of those activities while at work.

But it seems to be fairly standard practice nowadays with Stack Exchange that someone from outside of the network decides to point out a flaw or two, and changes happen overnight whereas well-thought discussions or communities willing to talk about these issues are simply...well, abandoned.

The Stack Exchange network's greatest strength is its community. Brushing that aside is proving to be the greatest single mistake that's been made.

I get it; we're not the most pleasant bunch of people to solicit feedback from. We're passionate, obstinate at times and think we know better. But shutting out an entire community from the discussion around this whole thing was a backhanded move, even if the whole HNQ flow is broken and even if it was going to be the right thing to do.


And I know I said I didn't want to sound pessimistic, but that's kind of how I feel about this whole thing now. You're going to do what you see fit with this feature. I could come up with some new ideas as to what the actual point of this whole feature is, and prove where it's strongest and weakest, as well as allowing people to see this content if they opt in to it.

But I'm not convinced you'll listen to me or anyone else in this discussion.

Prove me wrong. Please.

  • 13
    +1 for "But I'm not convinced you'll listen to me or anyone else in this discussion. Prove me wrong. Please." – X-27 Oct 22 '18 at 17:46
69

Please don't listen to Twitter

It seems that SE management is giving tweets a higher priority than community feedback. Just as an example: the Code of Conduct has been triggered by tweets, not by community discussion or by management decision. It seems, that if we want change on SE to happen, we have to tweet our wishes, and SE management will listen. One tweet has obviously more weight than 1000 meta posts.

How should we deal with complaints from Twitter?

If you think a complaint from Twitter is right, just post about it on Meta and resolve it from there. A tweet must not hold more weight than a comment or question or answer on a meta site.

I think it is a big shame that tweets have so much power upon Stack Exchange Management.

It seems like the priorities of SE management are:

  • make feminists on Twitter happy
  • make new users feel welcome
  • listen to the community

in this order. SE says that they value their long term community members. But actions speak louder than words. I ask for a change. I'm angry and annoyed. Would you listen to my anger if I posted it on Twitter? My trust in SE management has just gone away.

  • 14
    Yes. Even though a lot of the detailed criticism which provoked the CoC was on Medium, it's the tweets which directed all the attention in that direction. Tweets should only be treated as evidence for the existence of certain opinions outside. It fits with what IPS members wanted: they wished that the decision had been made on meta, even if it's an SE person saying, "hey guys, this is what people are being made to think. Perhaps we need the change after all.' – Chair Oct 20 '18 at 12:00
  • 13
    Downvoted for two reasons. 1) Being attractive to new users is important to SE, especially for beta sites. It's important for SE's management to pay attention to how the sites are perceived outside -- potential users and new users give us valuable insight that long-term users can't have because they've become too close to the site. 2) Your comment about 'feminists on Twitter' is hostile to those of us who use SE and Twitter and are feminists. Feminists have every right to participate on SE as much as you, and I don't appreciate you implying that you are normal and feminists aren't. – Jan Murphy Oct 20 '18 at 20:55
  • 16
    @JanMurphy While feminists have rights to participate they have to play by the rules. If they start twitter wars instead of going thru defined process they do not deserve to be heard here (unless they complain about unfair bans and thing like that). – talex Oct 22 '18 at 7:18
  • 8
    If Stack Overflow was dripping fabric softener all over people, and a feminist said STOP DRIPPING FABRIC SOFTENER ALL OVER ME, listening to them isn't catering to feminists. Nor is listening to someone reporting content on the site that we must assume is inappropriate for work or a shared family computer until the user expressly says it's no big deal (by visiting those sites). Our reaction would have been the same if you had reported it on Twitter and we saw it. The fact that Kelly's voice is amplified had a lot to do with us seeing it, but not the decision. – Tim Post Oct 22 '18 at 12:33
  • 39
    @TimPost The backlash isn't so much from the fact that it seems to be listening to someone with an agenda outside of SE, but the fact that these problems have been brought up for years here, and they've just been ignored entirely. Are we happy that it took someone that seems that hostile to SE to actually get the changes made? Nope. But we're less trusting than ever, because it seems like there's no way for us to actually effect positive change, using the very platform you've given for that purpose. Communication and trust are very much on the wane. – fbueckert Oct 22 '18 at 14:04
  • 5
    @talex Feminists are not some separate/special group of beings; it is just a label applied to humans who think men and women should have equal rights and treatment. Also, one person does not make the rule, rather they are an exception. – TylerH Oct 22 '18 at 14:35
  • 11
    @TimPost So are you saying Adam Lear is the employee who has to see a problem report before the company can act on it? And are you saying he doesn't read MSE/MSO ever? If the decision to remove IPS from the HNQ list is based on you guys seeing a problem report about it, then I expect it would have been removed a long time ago. – TylerH Oct 22 '18 at 14:37
  • 9
    @TylerH: Do you have any idea how rude you actually are? Anyone who doesn't claim to be a feminist is misogynist and sexist in your eyes. – Michael Oct 22 '18 at 15:14
  • 8
    You said, "It seems like the priorities of SE management are: Make feminists on Twitter happy. Make new users feel welcome. Listen to the community." Yes yes yes! That is EXACTLY the feeling most of us have. As an added bonus, you have the "new contributor" label right now so your voice counts for more than mine. Doubt they'll listen either way though :/ – X-27 Oct 22 '18 at 17:49
  • 9
    @TylerH there are lot of different groups that call them self a feminists. Those groups have completely different views. You represent one of those group. Why do you think only your group have copyright to feminism? But that's off-topic here. – talex Oct 23 '18 at 1:40
  • 9
    @TylerH: it is also about priorities. You seem to think it is more important to keep the name of feminism holy than it is to condemn the behaviour of these women. This just makes me assume that you think that what they did is actually not that bad. Or that you think that my misuse of the word feminism is worse than actually harassing a man over a sexual issue. Maybe I'm wrong though, but then why not just saying "I am a feminst and I think what these women did is totally unacceptable" - such a statement would have more weight for me than any wikipedia articles. – Michael Oct 23 '18 at 14:17
  • 7
    @TylerH if you didn't notice there is more thing associated with feminism then just "just a label applied to humans who think men and women should have equal rights and treatment". You can try to ignore that, but it will only lead to miscommunication. For example: you may not like it but a lot of people associate "everything is sexist" with feminism. You can say that (at least I hope you do) that people who say "everything is sexist" is not real feminist, but I suggest you to think about why that happens first. And you are sure that you don't misuse that term after all? – talex Oct 23 '18 at 14:21
  • 6
    @TylerH You would struggle to find a single living person raised in western society who wouldn't characterise themselves as supporting gender equality. By your definition, then, we all here are feminists, and the word is essentially useless. Yet we all know that in reality somebody described or self-identifying as a "feminist" predicts many more political positions than merely approving of gender equality in the abstract. An extensional definition of "anti-feminist", consisting of "opposition to the positions that mainstream society typically labels as feminist", is therefore more useful. – Mark Amery Oct 24 '18 at 9:53
  • 8
    @TimPost Your fabric softener example is predicated on the problematic behaviour being one that ordinary people would find objectionable for non-ideological reasons. But these Twitter feminists' objection to the posts in question was that it was "sexist" for IPS to allow the two questions they screenshotted to be posted - as I understand it, because they portrayed female sexual behaviour negatively - and therefore alienating to women to allow that sexism to be visible on the main site. That's a uniquely ideological objection, and - given that none of the parties involved object to [1/2] – Mark Amery Oct 24 '18 at 10:53
  • 9
    [2/2] discussion of problematic male sexual behaviour - a pretty objectionably sexist one, as well. There's a significant difference between "we acted on a complaint that just happened to come from a member of ideology X" and "we acted on a complaint that was inherently rooted in (a fairly extreme and openly discriminatory interpretation of) ideology X, while openly supporting the motivation behind it". It seems to me that Michael would have no beef with the former, but that's not what happened, and I think your dismissal fails to really engage with his complaint. – Mark Amery Oct 24 '18 at 10:57
67

Put the decision to participate in the hands of the community

Some sites have asked to be excluded from the HNQ. Giving sites the choice to put themselves onto the existing blacklist might be useful, and would involve the community that has to shoulder most of the negative side effects into the decision.

Give the community a way to experiment with the exact formula

This might not be possible with a reasonable amount of effort, but it would be very interesting to have the ability to simulate alternative HNQ formulas. It would probably have to be based on a snapshot of data because it's expensive, but even that could help to find better ways to select HNQ.

Allow users to collapse the HNQ sidebar

Currently users don't have a choice whether they see the HNQ or not. Give them a choice, and there are fewer reasons to complain about inappropriate titles. If you don't want to see the HNQ, simply switch them off.

Limit the self-reinforcing aspect of the formula

A question with a lot of activity has a better chance to get into the HNQ. A question in the HNQ list often gets drastically more activity and votes than without it. Maybe only counting votes from users that have contributions on the site could help.

Normalize between the sites better

Some sites are clearly overrepresented in the list. Normalizing the formula for differences in voting behaviour and other factors might help to avoid focusing on the problematic sites so much, and bring some smaller sites into the spotlight.

  • 1
    For your first point, do you mean "Let sites choose whether their questions appear in the HNQ" or "Let sites choose whether the HNQ sidebar appears on their site by default"? Both seem potentially reasonable to me. Most people on SO are there to solve a programming problem and probably don't want to see questions about flirting in the sidebar. People on worldbuilding.se are probably more open to seeing an interesting title and going "ooh that looks fun let's read it." – A_S00 Oct 18 '18 at 17:47
  • 1
    @A_S00 let the site choose whether their own questions appear in the HNQ. – Mad Scientist Oct 18 '18 at 18:42
  • 6
    +1 for "Maybe only counting votes from users that have contributions on the site could help." That's a great idea. – Wildcard Oct 18 '18 at 20:17
  • Please let me turn it off. its a major distraction. I'm easily distracted. When I go to a programming site. I don't want to see questions about aliens, dating, physics, especially when I'm interested in the click bait title. The network is now far to diverse for my intrests to align with many others. The goals of HQQ is in direct conflict with the goals of getting information about my chosen reading field. – Wes Oct 23 '18 at 16:12
  • @Wildcard At first glance, yes, but that would make the query even more expensive with the temporal aspect dependent on other factors. In relational terms, a left outer join with a null check in addition to date filtering on an entity level. Worth it? Maybe. – Physics-Compute Nov 7 '18 at 17:00
  • 1
    @Physics-Compute designing according to what’s easiest to implement instead of what produces the most desirable result is the basis for all of HNQ’s problems of today. – Wildcard Nov 7 '18 at 17:15
  • @Wildcard Agreed. Correctness first. I'll retract the "maybe" part. I definitely like the idea. Will just add that it may be a lot more expensive. – Physics-Compute Nov 7 '18 at 17:44
67

The fact that an entire site could be blacklisted from the Hot Network Questions list by a single SE employee because they saw someone complain on Twitter about the word "flirting" and the phrase "sleep with me" appearing on the HNQ list is highly disturbing. Using this new standard, we need to blacklist most of the other popular sites as well. In no particular order:

Workplace

"How can I stop a college from flirting with me?" - would be on topic as long as it related to workplace interactions. But let's not consider hypotheticals. There are some great examples of actual questions!

"How should I deal with an employee who has slept with my wife?" is the second highest voted question on the site.

There are also questions about flirting.

And if we're strictly talking about question titles looking bad out of context, there is "How can I prepare for getting hit by a bus?".

Sure, these could be overlooked as not too serious. So let’s look at some other examples!

Academia

Academia has some great examples as well:

"Had sex with a student of mine, how to proceed now?"

"I slept with my advisor's daughter and she is blackmailing me now. What can I do?"

This second one was actually on the HNQ list for a few days, before being closed - long after the question had gotten its attention.

Travel

This site has a few titles that would look great out of context:

"Is it permitted to ask questions of female sex workers in Thailand to be familiar with their real life as a tourist?"

"Are laws about sex outside marriage less rigorously enforced in Dubai for people on layovers?"

Role-Playing Games

Not as bad, but still an example:

"As a DM, is it improper of me to flirt with one of my players 'off the clock'?"

Movies

"Are Terminator robots able to have sex?"

"Mimi knows she has HIV but still wants to have sex with Roger?"

World Building

"How can a demon who feeds on sex survive in the modern world without exposing itself?"

"How can a parasite that feeds on sex take over a human biochemically?"

"how can you place limits on sex-fueled magic?"

Science Fiction & Fantasy

"Artificial organ on humans for sex?"

"How does human-robot sex work?"

"Can Bruce Banner Have Sex Without Becoming the Hulk?"

Parenting

This answer is to make the point that IPS was not a problem, it was merely an indicator of the flawed HNQ system. I hesitate to bring up these next examples because with the various changes SE has been making recently, they might actually blacklist Parenting.

"My son is 5 years old and sexually active"

"How do parents have sex when there's a toddler in the house?"

"Sex play in young children"

"My daughter is having sex with her cousin"

"Is it possible for an 8 year girl to be sexually abused and enjoy it?"

"My son's girlfriend has been cheating with my best friend's son. Should I get involved?"

"How do I change my son's views, after he confessed to me that he rather be with prostitutes over having an actual girlfriend?"

Law

This is another site that I hesitate to bring up in light of the recent decisions made by SE, but it has great examples.

"Did my girlfriend rape me? Sexual assault? Nothing?"

"If an adult is raped by a minor, is it statutory rape?"

"Is it rape to have sex with a man without disclosing that one is a male-to-female transgender?"

"If two individuals who are too drunk to give consent have sex are both guilty of rape?"

"Posting Sex Offender Information Online"

"The legality of fictional stories involving sex with minors"


The statement "Let's ignore sites with these titles" isn't being applied fairly. It seems that all we need to do now is start posting on Twitter about how offended we are. After all - if the above sites contain content like that and can still appear on the HNQ list, someone might post a similar question, it might get on the HNQ list, and someone who doesn't want to take the effort to look at the context of the question could be offended!

Is this a side effect of trying to be "welcoming" to everyone?

  • 17
    Don't even get me started on Arqade; we positively delight in misleading titles. – fbueckert Oct 22 '18 at 18:09
  • 10
    How can you not mention Mathematica? That site was literally classified as porn at one point. :P – Laurel Oct 22 '18 at 22:14
  • @Laurel I wasn't aware of that happening to Mathematica - was it because of a bunch of question titles that would fit here? XD – X-27 Oct 22 '18 at 22:36
  • 2
    I like your point but, to be honest, there is definitively title here that I wouldn't want to see on my HNQ list (the ones about sexual assault) and I absolutely hope that they are already banned from the HNQ list. – Noon Oct 23 '18 at 4:56
  • 10
    @Noon I fully agree that questions like that shouldn't be on the HNQ list, since they absolutely do look bad out of context. But my main point was that if SE seriously wants to use the logic that they stated in the question, "Now that we have gotten to the point where we've gone from saying 'Let's ignore titles with these words..' to 'Let's ignore sites with these titles..'" They'd have to ignore most of the well known sites on the network, because basically any site could have titles that look bad out of context. – X-27 Oct 23 '18 at 5:28
  • 3
    My word! That's some question list you've compiled there! I really hope no-one tweets it out! – RyanfaeScotland Oct 25 '18 at 14:17
  • @X-27: I don't understand the purpose of this answer. The question is not stating that banning IPS or a few keywords is expected to fix everything. The question is asking about ways to move forward in a way that things like this don't happen. That is, how to fix the HNQ so that it has the effects they want but without the downsides like what happened with IPS. They're going to overhaul it; the question is merely how to do that. – Nicol Bolas Oct 26 '18 at 16:25
  • I was expecting controversial question titles from mathoverflow as well, bummer. That would be hilarious xD – Firebug Oct 27 '18 at 18:46
  • a recent example from Writing.SE: What is a subtle way of mentioning one got an erection? (title was edited after it has been sitting erected for about a day at the top of the hot questions list) – gnat Nov 29 '18 at 9:29
53

Allow us to hide/show the HNQ sidebar and control which sites are allowed in it!

Like some others are saying, start with the HNQ list hidden, something like:

Hot Network Questions (show) ⚙️

On clicking "(show)," the usual list will drop down, showing only "non-controversial" sites (excluded sites like those you list in the main question). On clicking the settings gear, the user is taken to their "Site Preferences" page and scrolled down to the "HNQ Sidebar" section.

Here they can set whether the list is by default shown/hidden when the site is visited. They may also see which sites they allow to be shown and which they don't in their HNQ sidebar. Perhaps those sites we deem "controversial" will be marked so in the list (maybe with a link to see what some of those controversial questions may look like?).

This solution will do a couple things: it'll let those who like what we have now to keep what we have now, while letting those who want less HNQ in their lives to have less HNQ in their lives. This could also allow you to see more questions you like. Maybe you have the same length HNQ list regardless of how many sites you're watching, so now questions with fewer hotness points that would normally not make the full HNQ list will make your custom list!

Note: I think the contents of the HNQ list itself definitely need to be revisited as has been detailed in other answers, but this answer is focused on how we'll view that list.

  • 40
    It goes without saying that this has been highly requested for many years; SE point-blank ignored the request, and a whole garden of client-side Ignore HNQ solutions sprung up. Imagine if it had been implemented back in 2014 - all those oodles of useful UX data about who likes the list and when that would be there when things came to a head. – E.P. Oct 17 '18 at 23:17
  • 2
    +1. I like this approach, giving user the control to show and hide sites selectively. They may even go ahead and provide much finer control, such as giving us two lists with one curated for professional/academic environment and the other for non-professional environments. A drop down menu could help with selecting any list on the fly. However, this still doesn't address the clickbait questions in HNQ which gives a wrong impression of the quality content a site hosts. – Firelord Oct 18 '18 at 8:19
  • @Firelord that to me seems like a Very Hard Problem. But I think regardless of how we solve that, I'd still want these additions. I'll edit my answer to make it clear I'm not addressing what should be considered a "HNQ" – scohe001 Oct 18 '18 at 12:59
  • 1
    One important thing to remember about a setting is it relies on people to know it exists and care enough to use it. While our power users might benefit from this, the majority of people who visit won't. They'll see uncomfortable or nonsense looking question titles in the sidebar and just be upset, because well, they already saw them and that leaves an impression regardless of what tools are available to hide them after the fact. Development time is usually better spent on prevention rather than curing. Would this feature be necessary or even useful if it didn't happen in the first place? – animuson Oct 18 '18 at 16:53
  • 7
    @animuson note that I’m advocating for the sidebar to default to both hidden and to not include “controversial sites.” Essentially if they want to prevent the average user from seeing these kinds of questions, then make it reversible for those users who want it or are curious enough to find it and give us some customization perks while you’re at it. – scohe001 Oct 18 '18 at 17:09
  • 5
    @animuson Then make it discoverable. You guys are great at it, and you've already got plenty of tools that can be adapted to make this work well. – E.P. Oct 18 '18 at 21:53
  • 1
    This. I use an adblock rule to hide the HNQ, to avoid endless browsing and heavy loss of productivity. Unfortunately, I only set it up on SO. As soon as I'm looking for an answer in another site, I slip and fall for a long time. – Pac0 Oct 20 '18 at 9:44
  • 4
    I currently use uBlock Origin to block HNQ. Firstly, I use SO extensively at work, and a good proportion of HNQ has no business showing up on my work PC. Secondly, HNQ is actively damaging to my ability to concentrate on the task I have at hand, at work or at home. – Clement Cherlin Oct 22 '18 at 13:04
  • 2
    Joined just to agree with @ClementCherlin's point. People unfamiliar with SE sites looking at your browser just see "Playing with the musical turtle", "When/how did Harry Truman give the order to bomb Hiroshima?", "How to increase a monster's CR without adjusting its flavour too much?" (picking the least work appropriate from my current HNQ list - it looks like you're browsing something inappropriate, or just skiving. – tomRedox Oct 23 '18 at 14:36
  • 1
    I use this Tampermonkey script to block the top 20 or 30 most annoying sites for HNQs, FWIW. (This should totally be an actual site feature.) – David Moles Oct 23 '18 at 19:03
41

I'll add my 0.02€.

As prompted by the question, I'll start saying that I came to Aviation, the site I am now a mod of, through the HNQ.

I think that the list is a good tool to discover new sites, but given the following conditions:

  • the user is in a setting that allows them to look for new sites, i.e.: not in an office

  • the user is open to experience new sites

  • the user is familiar with the network and knows/can guess before clicking where the question comes from

I'll expand on the last one, since I think it was behind the message that started this.

If a user only ever visits the network to go on SO, and has no idea that there are other sites beyond that, or is not even aware of the concept of "network sites", those questions tell them "this are good questions for a professional website like SO".
This is fine if they come from SU, or any other IT-related site, as the userbase will greatly overlap with SO, but is already less fine with sites like mine or Politics or Skeptics (examples of sites I frequent) because these can be seen as odd, given the environment.
"I am here to work, what is a question about the elections of a country I don't care about doing on the side of the page?" "I don't care about aircraft, why is that question there?".
At the end of the spectrum you have the reactions that started this: "oh, great, the site I come to for technical advice is filled with male tropes, how nice"

Possible solution:

  1. hide the sidebar for new users by default
  2. have a small panel saying "SO is part of a network of several sites, would you like to know more?"
  3. if they want to know more, bring the user to a page where you clearly explain that in the network there are technical sites, as well as less technical ones
  4. explain what the HNQ is and what they will see if they opt to see it
  5. let a user opt-in the HNQ based on the macro categories you have here:
    • All
    • Technology
    • Culture / Recreation
    • Life / Arts
    • Science
    • Professional
    • Business
  • There would be another discussion to have then, and I am aware of it. e.g.: what's Academia doing in "Life/Arts"? – Federico Oct 18 '18 at 7:15
  • As to 5, frankly those macro categories can be quite inaccurate and also seem lacklusterly maintained, primarily based on the problem of many sites actually belonging to multiple categories equally well. I do professional technology business and watch cultural arts for recreation. – Christian Rau Oct 18 '18 at 9:30
  • 2
    @ChristianRau sure, the categories are far from perfect and I'm the first to acknowledge that some sites could be miscategorised (see my comment above yours), but they are at least something. – Federico Oct 18 '18 at 10:11
  • 3
    I was about to write an answer essentially saying what you've got as point 2 - in this particular instance, there are later tweets (I think by a different tweeter) saying things like "website for dev questions" and "corner of SO" in the context of the HNQ, so part of the issue does seem to be that people aren't aware that SE is a network which isn't just for devs and isn't just SO – Mithrandir24601 Oct 18 '18 at 23:40
  • 1
    I agree with all of this except the categories. Those are busted. Maybe instead when they first opt in we show them sites with good affinity from the current one (SE has that data), and then they can take a second action to say "open the fire hose". Or maybe we just jump to the fire hose. But not the categories; there's more harm than good there. – Monica Cellio Oct 19 '18 at 16:01
  • 1
    @MonicaCellio I'm fine with those categories not being the ideal solution. This post wants to be a suggestion, if then we have better options, I'm ready to support those. – Federico Oct 22 '18 at 8:25
41
+200

HNQ is intended to show users other sites. There's no assumption that they'll be experts on the topic; the assumption is that they are simply interested. With that end goal in mind, some suggestions:

Showcase older questions rather than currently active ones

Why does good content have to be current content? More often than not, current questions are undergoing some form of discussion to figure out what the questioner wants, and the traffic from HNQ just muddies that discussion. A two-day old, well-received question (upvoted question, upvoted answers) is probably a better way to showcase the site's best content.

Choose just a few sites, show upvoted front page content there

Rather than choosing "hot network questions", only showcase a few sites (3-5) each pageload. On the next pageload you could showcase a different few sites. Over time each visitor will see a wide variety of content from a wide variety of sites.

Choose from either well-received current questions or—as per previous recommendation—well-received older questions. This rotation will still benefit sites, and over time visitors will see that there's more to this place than just <site they happen to be on>.

  • 7
    I like the first suggestion, I'm not sure on the second suggestion. Do you mean show rotate through all the sites but only show 3-5 at a time or only show 3-5 sites? The former isn't a bad idea though I'm not in favour of it, I don't like the latter at all. – TheLethalCoder Oct 18 '18 at 14:33
  • 5
    This is unusual yet convincing. It would be very interesting to see what the rationale for choosing only new questions (less than 30 days old, if I recall) as HNQs. One of the arguments against HNqS is that wrong FastestGunInTheWest answers rise to the top, but if the question has been sitting there for a week (or so), it's likely that experienced/knowledgeable members have seen the content and reviewed this, which can, to a certain extent, prevent wrong content from rising unchecked if the question goes HNQ after that. – Chair Oct 18 '18 at 15:28
  • @TheLethalCoder - Thanks for comment, sorry the second one is confusing. Edited to clarify. – eykanal Oct 18 '18 at 17:08
  • 6
    To clarify, I'm voting up specifically for the first suggestion. I'm indifferent to the second. – trichoplax Oct 18 '18 at 23:27
  • first part looks like an original and promising approach, worth a bounty. "A two-day old, well-received question (upvoted question, upvoted answers) is probably a better way to showcase the site's best content" -- I am not sure about concrete value of proposed timeout but it certainly would be interesting to try delaying exposure of questions for few hours, to give hosting site community chance for initial evaluation and polishing before advertising the question to inexperienced network-wide audience – gnat Oct 26 '18 at 22:21
  • A two-day old, well-received question Who gets to select the two-day-old question? The system? Users with 10K or the mods? There's no guarantee that a highly upvoted question that survives longer than 48 hours should be showcased. I know plenty of posts that attract 10 or more upvotes because they are fun to answer but show absolutely no effort on behalf of the OP and no research. Qs can be hugely popluar if they are spurred by current events (i.e. controversial)... – Mari-Lou A Oct 28 '18 at 7:52
  • How often would this rota occur. Once a week? Every three days? What would be the criteria? If the mods choose, will they select the Qs according to personal preferences? (Watch the number of users complaining on meta about biased and prejudiced mods rise). Number of upvotes? Number of visitors? Number of answers? The quality of a single answer? Aren't we going back to the HNQ query again? – Mari-Lou A Oct 28 '18 at 7:53
  • 2
    Two days cooling period is a great idea for absolutely most of the questions on the HNQ list. There are a few sites that discuss current events, for example Politics may talk about current news, Information Security about a recent vulnerability, or Movies & TV about a new film or episode. These aren't common and don't represent even the majority of questions on these sites, but it'd be a shame losing these questions, and it will be weird displaying them two days "late" - it might make the HNQ list stale. – Kobi Oct 29 '18 at 11:47
39

There are plenty of answers here that explain problems with the almost NSFW nature of some of the questions, the promotion of click-bait titles and the positive feedback cycle introduced by the HNQ.

They are all about how the quality of one site is hurt by having inappropriate links to another site.

However, I want to emphasize at one particular aspect that bothers me more: the cost to the target site.

While the HNQ might help a site by attracting new users to it, it is often actively harmful to the particular question that it links to.

Even if the question is a genuinely good question that isn't click-bait, it can be hurt by making an appearance on the HNQ.

The HNQ attracts a lot of users who aren't from the target community. Their votes are wildly distorting and their answers and comments don't follow the community standards.

The site I frequent most, Skeptics.SE, gets hoards of users upvoting unreferenced, theoretical answers - especially if they are politically partisan - which is an anathema to the entire reason for the site.

But we are not alone. I see comments above such as these that I want to highlight:

+1 The HNQ absolutely optimises for controversy. On RPG.SE the effect of a HNQ is often also pretty bad: if we've got a controversial, difficult topic it needs careful community cultivation, expert answers, and highly knowledgeable voters more than ever... Then it hits HNQ and we see an influx of dozens of votes upvoting poor advice in exponential proportion to whatever the scores already were, amplifying the fastest gun and drowning out up-and-coming better advice completely. That effect is a disservice to everyone and degrades the quality of help RPG.SE provides. – @doppelgreener Link

and

@doppelgreener: Philosophy has this problem as well. Controversial questions that have "God" in their title are regularly on the HNQ, even if they are ill-stated, unclear, etc. and attract a high number of opinionated comments and answers of which two-thirds have to be deleted. The only other things popping up there are similarly bad questions on evolution and quantum mechanics/free will. I have yet to see a well-stated question on philosophy in this list. Yes, it helps with traffic for smaller sites. But I doubt that it is traffic we want to have on these sites. – @PhilipKlöcking Link

Many of the programming-related sites have the benefit that upvoting tends to be clear cut. Questions are only of interest to relevant experts, answers tend to be clear cut (they objectively work or they don't), and non-experts have no desire to jump in and express opinions or vote. They may welcome questions appearing on the HNQ, knowing that they will only attract other experts. Some of the other sites don't have that luxury.


What's the solution?

  • 4
    some evidence of how HNQ hurt the target site can also be found in SE.SE meta discussion: Recent Trouble With Popularity. "While these questions were open, they were widely advertised on the Hot Network List and gained relatively high scores despite piling on of close votes. All of them are now closed, but due to their high scores they present appealing examples for newer site visitors to try their luck asking similar questions. High scores send a fairly strong signal that those sorts of questions are welcome here..." – gnat Oct 18 '18 at 13:20
  • 4
    See also my answer to Let's Plan the Second Iteration of the Stack Exchange Quality Project! and the links therefrom. – Peter Taylor Oct 18 '18 at 13:48
  • 2
    @PeterTaylor: Yes, thanks, and that brought me to Sklivvz's suggestion which I looked for but couldn't find earlier. – Oddthinking Oct 18 '18 at 16:47
  • speaking of troublesome answers, a while ago I studied these matters at SE.SE and reported my findings in much details at meta: Answers quality in hot questions. "Summing up, I would downvote about 101-118 of 218 answers I reviewed per above research..." – gnat Oct 20 '18 at 10:23
  • 2
    Another possibility for you to consider (same thing different flavour): meta.stackexchange.com/questions/238420/… – DVK Oct 22 '18 at 17:08
  • 1
    @DVK: Excellent. I am going to edit it into the answer. – Oddthinking Oct 23 '18 at 5:51
  • +1 the vast number of answers contributed by people drawn to RPG.SE from a HNQ are bad answers from people who don't know an RPG from a DMZ, requiring downvotes and sometimes deletion and moderation, causing consternation and hurt feelings and - more importantly - crappy answers for the querent. Which is supposed to be why we're here. – mxyzplk Oct 28 '18 at 15:41
33

I do think keeping it is a good thing. I know for myself, it's introduced me to sites and topics I'd not have expected to see on SE, and I know for younger sites, it can be yet another way to get a critical mass of people in the days when you really need them because no one knows about you yet. Sure, not all of those people stick around, and sure, not all of those people are there for more than like...rubbernecking at what feels like an impending drama storm, but there is good in it.

I know what happened to cause this, but I feel like the reaction was kinda throwing the baby out with the bathwater. A lot of sites have problems sometimes with "offputting" sorts of titles (I moderate Arqade, so I have some awareness of how an honest title can sound highly weird to people who don't know what's going on, and on the flip side, I have experience with having to bop things over the head when people (myself included!) sometimes have a little too much fun with titles), but I think there are ways to potentially deal with that beyond just going "this sucks lets toss it out".

I think there are a few problems, though - number one is that anything controversial is going to pop up there. Which causes more eyes, which causes more...everything, which causes more "hotness" and thus it lurks in the list when it might not be the best and brightest that site has to offer.

In a way, maybe we should be looking less at "hotness" and more like, perhaps, some sort of curated list of "the best and brightest"? I am not sure how much filtering can be done, but if there was a way to be like "these are actual good examples of current questions that are a good example of the awesome we are bringing to the SE table", that would go a long way.

Also, I think there needs to be a way for individual questions to be pulled based on the community/mods/whoever going "okay nope, this is not what we want to be known for". I think something like that might have helped this last issue. And if we can get some sort of notification that something has hit HNQ, that would also be good - then no one is surprised at the sudden attention, we know what potholes we might fall into if something that needs some title nudging/editing gets there so we can work with it, etc. Right now, there's not much we can do short of closing a question, and that's not always the best fit - title alone might not make it a bad/unworthy site question, it just might not work as something shoved to people who didn't ask to be here in the first place.

Also, I have seen a number of proposals that maybe HNQ should be opt-in/opt-out, or at the very least, collapsible or something - so I can interact with it if I want, but if I am only here for my cooking questions/programming questions/dog care questions, I don't have to worry about seeing something from Parenting/Workplace/IPS/Arqade etc that might not be my jam so much.

  • 4
    Re "the best and the brightest": I suggested something much the same (sorry for not seeing yours first!), and it makes a whole lot of sense. Hot != best. – HDE 226868 Oct 17 '18 at 18:13
33

This has already been brought up tangentially in some answers, but:

Do HNQs really need to be that fresh?

The vast majority of the problems arise from HNQs being selected by an algorithm only. The only reason I see for having it done this way is that you want eligible questions on the list as soon as possible. While I can understand that advertising yesteryear’s questions there is against the point, I don’t see why we cannot wait a while to let humans decide.

As a brief sketch, I would suggest the following:

  • Once a question has acquired enough hotness, place it in the moderator queue of that site, where moderators can forward it to the actual HNQ list or reject it.
  • Provide moderators with some guidelines regarding titles, content, and so on.
  • Of course, moderators can also use this opportunity to polish the question (in particular the title), perform some comment clean-up, or protect it pre-emptively.

On most big sites, this should impose a delay of a few hours to half a day. On slower sites, it may take a while longer, but then those sites are slower anyway, so the delay doesn’t matter that much to them.

  • @Mari-LouA: Would it be fair to leave all the decisions to a small team? – Well, the alternative are regular reviewers and given robo-reviewing and other bad decisions made by them, I really wouldn’t trust them with this. Mods can at least be held accountable to some extent. Also, keep in mind that the main point of this is to filter out the really bad stuff. That being said, some bias is unavoidable, but that also applies to all methods of choosing HNQs. – Wrzlprmft Oct 18 '18 at 9:34
  • 11
    Instead of mods; maybe xxxx+ rep users. Finally something serious to do as a 10k user ;-) – Jan Doggen Oct 18 '18 at 12:51
  • 2
    There's a feature request for your first bullet point at Add a Hot Network Question review queue for cleaning up questions that are about to become hot. – Thunderforge Oct 19 '18 at 21:20
  • eykanal answer also touched this sensible point. Isn't one of the goals of the network to build an encyclopedia of knowledge? It doesn't really matter if something was posted last year or yesterday, does it? – brasofilo Oct 20 '18 at 0:10
  • Thanks @Wrzlprmft you said this better than I would have. Since moderators for a site are most likely to have the best idea of what questions would promote their site (rather than put people off) I think they are in the best position to decide what questions should be eligible for the HNQ. I agree that 10k users might also be worth adding into the mix though too. – Mark Booth Oct 24 '18 at 9:19
  • From another angle, I've long thought smaller sites are most harmed by the freshness factor - even moreso sites where good answers often take a lot of work to compile (science/math/certain programming/etc). I'll often bookmark a Q and come back to it months later to put together a quality answer. But I know no matter how good the answer is, it'll never gather much interest. A fair % of users in the community will see it back active and assume it's just the same old question. But no matter how many users at the site upvote, it still fails the 1 month rule for HNQ. All certainly discouraging – JeopardyTempest Oct 26 '18 at 21:17
31

The titles of these questions, out of context, are just too likely to be inappropriate on our professional and academic sites, so we took the site out of the contributing list.

To be honest, when speaking just for myself (and probably a part of the IPS community too)... I don't mind being 'the one site that's out of the mix' for a while. It offers some interesting opportunities for IPS, such as some breathing room to actually moderate our questions and answers, and see just how much voting coming in from HNQ skews the view on what a quality answer on IPS is.

That said... This would've gone much, much better if there had been some warning, some discussion, on IPS first. Even if that means a programmer from SO coming into our meta, saying 'HEY! your question titles could be improved upon a lot, that would really make my day'. I guess we wouldn't be having the entire discussion right now then.

But now that we're having it, you have a set of goals... let's see if there's other ways to meet them.

They help with discovery. User interviews continue to reveal that folks simply don't often discover just how big of a resource we've become collectively. The list boosts engagement in quite a few ways (we'll admit some of them aren't optimal)

Perhaps a thing to do this would be to feature some sites, some topics, instead of questions? Have some kind of sidebar 'advertisement', that invites people over to discover the site, instead of reading it's currently most contentious question? Have a list of 'featured' sites? Of 'Hot Network Sites'?

A personalized list, like Glorfindel suggests, sounds great but it constructs some kind of 'social media bubble': it doesn't get you out of your comfort zone the way HNQ currently does. It's okay, but then you'd probably have to give up on the goal of reaching new people with new stuff. I'm personally quite fond of that functionality. It's made me discover more of the SE network than just StackOverflow, and I wouldn't be a moderator on IPS now without HNQ breaking into my bubble and introducing me to a part of the network called 'Interpersonal Skills'.

Linking just the site instead of a post would mean that there's no huge reputation gains for people asking a question or writing an answer anymore, instead, hopefully, traffic to a site is distributed more evenly across posts, instead of just among the three posts in HNQ. This could be a blessing and a curse, I don't have any data to back up this claim, just thoughts.

They often remind communities to have discussions about scope. If folks see a clear demand for certain kinds of questions, it's worth revisiting conversations about how valuable finding ways to meet those needs might be.

This one is hard. Perhaps the people that do land on a site due to an advertisement will start asking their own questions, will write answers to the questions of their interest, will feel free to ask 'Hey, your site was advertised to me, can I ask about X here?'

To be honest, I again don't see much difference here between either advertising the entire site vs. just a question. Even if people see just a question, it's going to give them pretty much the same impression as a good advertisement for your site will do.

Hot questions are extremely great fodder for evangelists, the kinds of people that reach thousands of folks that are likely to identify with the way we do Q&A.

Those people likely need a way to keep track of what's hot across the network. Perhaps this doesn't need to be featured in a side-bar though. You could feature the 50 hottest sites right now, and have more sites (and thus questions) in your Hot Network Sites than you get posts in your Hot Network Questions. You can have a list of hot questions per site? Let's face it, there's more sites than just IPS that generally have more than 1 question in HNQ. The only downside to this would be that whatever happens, really really low traffic sites will still end up on the short side of the bargain and not gain much exposure.

All in all, I'm sure there's other ways to reach out to your evangelists. Try and find out how much work those evangelists are willing to do to get their material, what would they like to see? If they know where to find HNQ, perhaps a single link (instead of a list of question links) to the general HNQ site may work fine. Or make that list collapsible.

Automatic selection simply will not scale if we continue to push the limits of subjective topic spaces, and we need to continue to do that for the network to grow and remain relevant to the issues folks want to use our software creatively to solve.

I would warn against making too much of the selection manual though. Currently, the algorithm relies heavily on actions taking by the community (voting on question/answers and answering the question) to get a question into HNQ in the first place. Taking that away from the community and putting it in moderators hands may be less than ideal. But I also don't know if you can expect the community to manually keep coming up with good posts to put in HNQ, the 'best answer of the past three months' contest on IPS died out due to lack of interest.

Suggestions like 'allow moderators to pick what reaches HNQ' carry the big downside that if moderators do get that power, we basically can decide who wins the imaginary internet points lottery and who has to scrounge through dustbins for next points. That's not really fair to users, and it's not really fair to moderators either. I'm expecting a lot of complaints if people are given too much power to select what can reach something like HNQ.

I prefer being able to blame something automatic when someone complains about reputation from HNQ not being fair ;)

The same goes for some of the points Monica mentions, like giving moderators the power to exclude questions from HNQ. Maybe the question asker really, really wants their question out of HNQ. But is it really fair for us to take down a question, just because a single person wants to? We'd be robbing honest answer writers of their exposure too.

Excluding protected questions carries a risk too. People already find it 'elitist' sometimes that they can't answer a question due to it being protected. How would people like it if three nasty people with malicious intent answered a question, just to get their answers downvoted/deleted, and the community user to protect a question, which will lead to it's protection and it being taken out of HNQ? As long as the metrics for protection are visible, and every community member with a certain amount of reputation can protect/unprotect stuff... I'm expecting a lot of trouble by implementing such a system.

  • 2
    I wouldn't see deHNQification (patent pending!) as a power to be used willy-nilly at the request of a user, even the asker. I'd treat it more on the scale of closing/deleting, which are actions with important repercussions, but which we do lay into the moderators' hands and trust them to use them only when necessary. If someone complains about it, you better had a good reason to do it. – Christian Rau Oct 17 '18 at 18:34
  • 1
    @ChristianRau perhaps. But then, if that really, really good reason is already present, we can even just close the question for a while (and that's something that a community can do, even). Making it easier to get questions in or out of the current HNQ carries a lot of problems, especially if that power is just in a mods hands. Having a question in or out of HNQ carries a lot more consequences than having it open/closed/deleted. Even those are preferably still always done by the community. Having mod power to do that would still seem like a lot of willy-nilly to me – Tinkeringbell Oct 17 '18 at 18:42
  • 24
    Closing an on-topic question that somebody wants to get answers to just because some network-wide algorithm is currently doing bad things with its advertising seems really unfair to site users. I don't think mods (or anybody else) should close a question that shouldn't be closed just to dodge HNQ. – Monica Cellio Oct 17 '18 at 19:12
  • @MonicaCellio That might be true... it was more to illustrate that there is a way to take stuff out of HNQ (and that that already is unfair). I don't see how giving moderators a way to just jank questions out of there that we don't like improves the situation. You're still silencing people, still censoring perfectly fine on-topic questions. (only now you have the tool to do it) – Tinkeringbell Oct 17 '18 at 19:14
  • 25
    No, yanking a question from HNQ while leaving it open and answerable isn't silencing anybody. It's putting that question in the same situation as all the other questions on the site that weren't pushed into HNQ. Think of HNQ as advertising, not entitlement. – Monica Cellio Oct 17 '18 at 19:16
  • 3
    @MonicaCellio I'm more than willing to think of it as advertising... I still don't feel comfortable with the idea of being able/having the power to decide 'you go, you don't.'. Especially when taking questions that are already in HNQ out of it again. But also in the proactive stage. You're essentially saying 'you're not good enough for HNQ' to that particular question, which makes it different from all those other questions that didn't reach HNQ (because hey, there were already 5 IPS posts in there at the time they were asked, and nr 6 didn't reach the heavier threshold). – Tinkeringbell Oct 17 '18 at 19:21
  • 3
    By being able to say 'you're okay for HNQ, you're not'... you're not putting questions in the same situation. – Tinkeringbell Oct 17 '18 at 19:23
  • 1
    There's also the perspective of users who wrote answers for the HNQ: they may be hoping for the rep and they've technically put in the work for it. It's sticky to have other people decide that the content isn't appropriate to be exposed to potential voters; this could be used competitively. – Chair Oct 18 '18 at 13:31
  • I am glad to hear you say that Tink... I agree HNQ is not necessarily IPS's friend, to understate it to the extreme. HNQ turns IPS into Springer. (a tawdry daytime TV talk show host who showcases dysfunctional relationships, often with spectacular reveals of paternity tests). I think without HNQ's "help" IPS has a better chance of discovering its own natural identity and role. – Harper Oct 18 '18 at 18:56
  • 2
    Simply linking a site instead of questions makes it far less useful. I don't know how many people click on those community ads, but I never have. OTOH, almost every non-SO community I'm part of I found via the HNQ list, and I often like to read topics on sites I'm not a part of. I essentially use it as an "interesting question list" aggregated across a dozen or more sites. Simply linking sites would lose that benefit because not every site is discussing something interesting all the time. – Troyen Oct 19 '18 at 11:17
  • 3
    For example, while I generally find the Worldbuilding HNQs to be interesting, the WB homepage is not. Most of those homepage questions have lingered around for weeks, months, or years (so I've already read them), and a large portion of the new ones are downvoted or on their way to closure. Seeing a question on the HNQ list is a sign that "hey, finally some new stuff to check out!" (Well, ideally, though in practice questions can linger in HNQ for days.) I don't think linking to the site would serve as that kind of signal. – Troyen Oct 19 '18 at 11:21
  • Good answer, upvoted, but disagree that a valid function of HNQ (or any site feature really) should be "taking people outside of their comfort zone". I think I see what you mean by this, but honestly my comfort zone is something I need respected, especially on a site I'm using at work. – jkf Oct 25 '18 at 21:01
31

I have added some more analysis on IPS meta using (give or take) two weeks of data. This can be found here.


To give an idea what the presence/absence of HNQ does to a site, these are the Google analytics data for pageviews for IPS the last week:

date        page views  visits  new visits
2018-10-10  24496       9315    2718
2018-10-11  31098       11335   2748
2018-10-12  33044       12596   2781
2018-10-13  11883       6289    2271
2018-10-14  12796       7059    2518
2018-10-15  40497       16328   3491
2018-10-16  29594       12672   3350
2018-10-17  13698       5801    2608

Halfway through Tuesday the 16th the removal happened, making Wednesday the 17th the first full day of data we have. Comparing that to the Monday before, the traffic is ~33% of what it used to be, comparing to the same day a week before, it is 54% of what it was.

That is however including the traffic generated by this incident, so the final numbers might be even lower.


Let's compare this Wednesday to Wednesdays in the past:

    Week `page views`
   <dbl>        <int>
 1    25        31861
 2    26        29288
 3    27        26134
 4    28        50706
 5    29        49773
 6    30        32966
 7    31        41723
 8    32        30800
 9    33        19670
10    34        35570
11    35        20137
12    36        28289
13    37        27842
14    38        29110
15    39        40004
16    40        20902
17    41        24496
18    42        13698

enter image description here

Lower, but those values are kinda all over the place, so let's compare it with what we might expect based on how this week started.


I am comparing Monday to Wednesday, because Monday is the last complete day of this week with HNQ, and Wednesday is the first complete day afterwards.

    Week Monday Wednesday RatioWedToMon
   <dbl>  <int>     <int>         <dbl>
 1    26  22842     29288         1.28 
 2    27  28242     26134         0.925
 3    28  39476     50706         1.28 
 4    29  43726     49773         1.14 
 5    30  44029     32966         0.749
 6    31  42273     41723         0.987
 7    32  34528     30800         0.892
 8    33  31826     19670         0.618
 9    34  26157     35570         1.36 
10    35  30651     20137         0.657
11    36  23212     28289         1.22 
12    37  24193     27842         1.15 
13    38  25534     29110         1.14 
14    39  44303     40004         0.903
15    40  26644     20902         0.784
16    41  24323     24496         1.01 
17    42  40497     13698         0.338

enter image description here

That's a stark difference. Looking at the summary statistics of the ratio before this week, we can see that week 42 is a definite outlier.

     Ratio       
 Min.   :0.6180  
 1st Qu.:0.8651  
 Median :0.9971  
 Mean   :1.0061  
 3rd Qu.:1.1678  
 Max.   :1.3599

On average, the traffic is equal between Monday and Wednesday; this week that was a third.


Fitting a simple linear model on the previous weeks: lm(data = train, formula = Wednesday ~ Monday)

Coefficients:
             Estimate Std. Error t value Pr(>|t|)   
(Intercept) 6185.6933  7634.4636   0.810  0.43136   
Monday         0.7978     0.2316   3.445  0.00395 **

It's not a perfect model, but I don't have an awful lot of data to work with. Based on this, we would predict a traffic of ~38500. In reality, this was 13698, or 65% lower than expected.

Gist with my code. I don't think I'm allowed to share the Google Analytics data sadly.

  • 11
    @NicolBolas: The 13th and 14th are weekends, and it's not an unresaonable pattern to note that traffic across the network sags on the weekends. – Makoto Oct 19 '18 at 17:49
  • 2
    It will be interesting to see if the quota of questions asked per day, currently at 4.2, will shrink in the coming week/s. – Mari-Lou A Oct 20 '18 at 11:07
  • @mariloua I'd guess the effect on number of questions is relatively lower, but we will definitely be keeping an eye on the site stats. – JAD Oct 20 '18 at 11:22
  • Related to this, here is a query to check how many questions per Monday we usually get (this Monday isn't in it yet but, for now, 0 question have been asked). – Noon Oct 22 '18 at 13:58
  • (1) The effect on IPS is likely atypical - for a small site it had a lot of HNQs. (2) If the drop-off in views is due mainly to the loss of people looking at just the HNQs before leaving again, it's perhaps not to be much lamented. Counting questions asked, as @Mari-LouA suggests, or the number of new users signing up might be more telling. – Scortchi Oct 24 '18 at 12:02
  • 3
    @Scortchi it's definitely worth taking a second look at the wider sitestats. The number of questions per day vary a lot more though, so it's better to wait a little while to gather more data. – JAD Oct 24 '18 at 12:19
  • Is it too soon to ask for an update? – Mari-Lou A Nov 1 '18 at 8:06
  • @Mari-LouA not at all. I'll see if I can cook something up when I get home. – JAD Nov 1 '18 at 8:56
  • @Mari-LouA posted it here. – JAD Nov 1 '18 at 21:45
  • 1
    If I've understood correctly, 17 42 40497 13698 0.338 refers to 17 October. Your stats only refer to the previous weeks (am I right?) What has happened since then, in the two weeks after IPS was "banned" from HNQ? My impression, from just looking at the number of questions posted on IPS, is that things have actually improved. The number of questions posted is higher but I have no idea about the number of views. To recap, are the stats in your answer updated, i.e 1 November, or not? – Mari-Lou A Nov 2 '18 at 7:35
  • area51.stackexchange.com/proposals/92736/interpersonal-skills Now if only there was a way to get the stats from two weeks ago. Would there be any difference? – Mari-Lou A Nov 2 '18 at 7:38
  • 1
    @Mari-LouA the stats in the answer have not been updated. I did make a more in-depth post based on two weeks of data on IPS meta, which I have linked to in the top of this answer. As for the A51 stats, from the top of my head, the views/day have dropped, the answers/question have dropped, but the other stats appear to have remained stable. – JAD Nov 2 '18 at 7:50
  • But if you managed to produce (interesting) stats for weeks 26 to 42, what happened to week 43 and 44? Surely that is the most relevant data, we would see a trend of some sort, wouldn't we? – Mari-Lou A Nov 2 '18 at 7:59
28

I really like the HNQ, and have often learned about interesting topics because of it. I would hate to see it go, and I don't think it actually has serious problems. The fact that SO felt the need to remove some items from it after a random person on Twitter complained about it seems like an overreaction to me.

(Those IPS questions looked genuine, asked in good faith, on-topic for the site and correctly titled. I hope you're not going to censor contentious subjects; one of the most interesting questions I saw in the HNQ recently was titled "My 3 year old thinks she's white; should I correct her?")

Having said that, there is one thing about the HNQ that annoys me immensely: the fact that I can't choose to hide certain sites. I have no interest whatsoever in Gandalf, Dumbledore or R2D2, or what type of stone victims of the Petrification spell turn into, or whether it is a sin to ride a unicycle on Tuesdays in a certain religion.

The fact that a lot of the tiny site icons are similar and hard to recognize means I'm often halfway through reading a title in the HNQ when I realize it's from one of the nonsensical sites.

So please, let me click somewhere so I never have to see Role-Playing Games, Science Fiction & Fantasy, Christianity, Buddhism, ... ever again. (But don't group them into categories, because I quite enjoy World Building, e.g.)

(Of course you don't have to agree with my selection of uninteresting sites, because that is part of the point I'm making: everyone will have a different selection.)

25

And in the spirit of causing tons of people to flood into a single discussion, I'm going to feature this; I just can't help it.

There you go, you already have the solution. Give trusted users the ability to tag questions as featured. Then apply the arbitrary hotness points calculator to the featured questions to determine which ones to show in the HNQ list.

Alternatively, create a featured review queue where questions that have enough arbitrary hotness points get placed. If enough trusted users vote to make the question featured, it automatically gets the tag and shows up in the HNQ list.

If a site wants a question off the HNQ list, all they need to do is remove the special tag.

  • 5
    Not the biggest fan of a tag-based solution (for technical reasons), but the idea of a review queue is neat. However, opt-in on per-question scale might require a little too much manual effort, especially on smaller sites. But for notification purposes (and opt-out), it would be quite useful. – Christian Rau Oct 17 '18 at 18:38
  • 2
    How do you decide who is trusted and how would we keep this from being abused by trusted users? A question that hits the HNQ can easily get 10x as many upvotes as one that doesn't so the motive for abuse is certainly there – Charlie Brumbaugh Oct 17 '18 at 18:44
  • 2
    @StrongBad sounds like an idea, but given the title issue that got IPS out of HNQ, would you solve this by reviewing? The titles are, in a sense, representative of their posts, but out of context they look off. Those questions certainly weren't HNQ unworthy (to phrase it one way), it's the titles that caused the problem. They may end up being voted into your list this way? – Tinkeringbell Oct 17 '18 at 18:44
  • 3
    @CharlieBrumbaugh a trusted user is a term for a 20k+ user of a full site and a 4k+ user on a beta: meta.stackexchange.com/help/privileges – StrongBad Oct 17 '18 at 18:48
  • 8
    @Tinkeringbell sure so the review queue has two questions: Does the title contain anything that when taken out of context could be considered offensive and Is this question representative of your site. – StrongBad Oct 17 '18 at 18:50
  • 1
    This is possible at the moment in a way, you just need 5 users to close. However the apparent amount of effort people seem to put in before VTC doesn't give me a whole lot of confidence that this would work even with the higher reputation threshhold – Charlie Brumbaugh Oct 17 '18 at 19:03
  • There is already a Featured list, and users already can earn the privilege to put questions on it (by placing bounties). And there is already a list of questions with high hotness points that might get selected by HNQ - the Hot tab on the front page. They're not quite functional equivalents to your proposals (and they might occupy the semantic space you'd want for a different feature), and you probably want to build something different (more review-queue-y, say), but lots of what you're discussing here is already out there in some form or another. – E.P. Oct 17 '18 at 23:08
22

I would start by leveling the playing field between the narrowly focused and the overly broad questions by reducing the number of answers that go into the hotness equation.

Right now of these two types of questions,

  1. Narrowly focused with one right well researched answer.

  2. Broad question that has many possible answers and everyone gets to weigh in with their opinion but not broad enough to be closed.

Arguably type 1 questions are better but right now the broader questions have a better chance of getting onto the list and staying on it for longer since it has more answers to upvote and therefore gain more hotness points.

  • how do you propose this playing field is leveled? – Will Barnwell Oct 18 '18 at 0:38
  • 5
    @WillBarnwell Right now the score of the question plus up to 10 answers go into the hotness score. If it was reduced to say 5 or maybe even 1 answer, then the advantage of broad questions would be much less – Charlie Brumbaugh Oct 18 '18 at 0:56
  • 4
    or you could normalize the answer score (i.e. sum the answers, but divide by the number of answers) – Federico Oct 18 '18 at 6:42
  • 2
    @Federico That would incentivize protecting hot questions right away because otherwise, one good question with one good answer would have it's hotness points cut in half by the addition of one more answer. – Charlie Brumbaugh Oct 19 '18 at 3:00
  • Perhaps include a view count or at least age factor in the normalization. – JeopardyTempest Oct 26 '18 at 21:24
22

Here is one concrete proposal on how to give sites a chance to control the quality of HNQs.

Whenever your automatism detects an HNQ candidate, put it into a dedicated review queue.

  • Explain to the reviewers what the purpose of the queue is: polish the candidate and make sure it's representative of the best the site has to offer.
  • High-rep users
    • can edit titles, tags, and posts to shine, and then
    • accept or reject the candidate for publication via HNQ.

The votes needed for/against need to be calibrated carefully, of course. Never should a question go "hot" without the approval of the community.

Ideally, all mods and accepting reviewers should get a notification once the question goes hot so they can closely monitor and, if need be, moderate the incoming traffic.

  • 2
    This is a popular suggestion: it's very similar to the exclude-from-HNQ idea suggested in Monica Cellio's answer, and I'm quite sure it's been suggested even more directly somewhere else. – Chair Oct 18 '18 at 17:01
  • 7
    I like the idea of sending it to a review queue. This lets the community review and improve likely-to-be-popular posts regardless of whether they actually hit HNQ in the end. – Monica Cellio Oct 18 '18 at 17:54
  • 2
    This is a brilliant and extraordinarily simple suggestion, and very nicely fits in with the ethos of the Stack Exchange network—community-driven, automation-assisted collaboration in building a knowledge base. – Wildcard Oct 18 '18 at 20:38
  • 2
  • 1
    +1, while I have posted my own answer suggesting human curation, the specific method you propose also has the advantage of making use of the existing mechanism of the review queue. I would still argue that the preselective algorithm should be tweaked to be less indicative of "what catches people's attention", but overall a great idea. – rumtscho Oct 25 '18 at 20:04
  • 1
    +1 This is what I was going to suggest as well, though you've fleshed the idea out a bit further than I had in my own mind. Glad I scrolled down enough to find it! – Dan Henderson Nov 29 '18 at 19:51
21

What's in a name?

The Hot network questions cause a problem right from the first word. Why do we care if a question is "hot"? Don't we want "good" or "informative" or "searchable" or anything other than hot questions? A dumpster fire is hot, that doesn't mean that it will draw me over to look at it.

Show Users Good Network Questions (GNQs)

Wouldn't it be better to bring new users to a site via [good], [great], exceptional, or outstanding questions? Or to use more adjectives, maybe [popular], [notable], [famous]? Some of those already have badges that are associated with them. This seems like a very easy way to find the best questions for a site and show them to the outside world. These questions are likely on-topic and show what a site has to offer.

Make a List, Rotate Through It

Use the badges noted above to create a list of possible questions for each site, and then rotate through them. This keeps the same questions from popping up all the time and should be easy from a performance standpoint.

Give each site an equal chance to have a question on the list or base it off of some other metric so that the advertising potential of the GNQs list is spread out.

This gives the sites a chance to make sure that questions are friendly through the natural progression of a site instead of having to jump on the new questions.

  • Questions with these badges (1) are typically already exhaustively answered, so there's little chance of having something to contribute yourself (which sometimes happens with HNQ), (2) may not be that relevant any more, (3) are much less likely to appear on smaller sites (where we may miss even really popular questions) and (4) I'm, without much data, inclined to say are the less interesting questions (and yes, interesting is good - something that every person and their dog finds via Google is probably very useful, but (a) I'd find it if I cared and (b) I probably don't even use those tools) – Dukeling Oct 17 '18 at 22:22
  • 13
    Everyone has different reasons for visiting sites, but I most of the time I click on HNQs because they look interesting. I usually visit sites to learn, not to answer. – hazzey Oct 18 '18 at 1:21
  • How can "goodness" be measured? ... – user202729 Oct 18 '18 at 9:09
  • 1
    SE already captures some good measures of question quality, e.g. views and votes. Those stats are used at the basis of the question and answer badges already, so SE thinks that those are acceptable values to base rewards upon. This proposal extends what is already done to also aid in populating a list of good questions. – hazzey Oct 18 '18 at 12:49
  • @user202729, I think Monica Cellio and gnat's answers give very good ideas on how to measure it – brasofilo Oct 20 '18 at 0:22
20

While I'm working at my job (which accounts for probably 90%+ of my use of Stack Exchange sites), after probably the 20th+ time that I was distracted from a work task by an interesting link in the Hot Network Questions sidebar, I decided to hide that section of the sidebar, via a change to my client browser.

(Various methods for doing that can be found in answers to this question: How to avoid "Hot Network Questions" on the sidebar)

So, while I'm at work, I personally wouldn't mind at all if the Hot Network Questions just went away -- since that's the situation I've already set up for myself anyway.

When I'm browsing Stack Exchange sites for fun in my free time, I do find the Hot Network questions very interesting, in a good way; however, that use case is of secondary importance for me personally.

  • 14
    Exactly. This feature is counter to the ethic of the network. – user Oct 17 '18 at 23:20
20

Break the feedback loop by ignoring all interactions from the HNQ

If we ignore all interactions coming from people who visited the Q&A from the HNQ, there is no longer any "positive" feedback and people from those communities can more easily regulate if the site is on the HNQ list by not voting at all.

This makes the system still automatic, but prevents a ruh of new people who visit a site in the HNQ making the question more likely to stay inside the list

  • 1
    Do you mean like a hell ban, or visibly say "Thanks for riding in on the party bus; unfortunately we can't let you get off at this stop because we know you're gonna litter" sort of thing? – Tim Post Oct 22 '18 at 12:27
  • 6
    @TimPost More like that the system also keeps track of a number that increases for the number of HNQ visits and a number that keeps track of the HNQ upvotes, and then subtracts that number from the real upvotes and visits before calculating the HNQ score. Doing it this way also prevents most abuse by serial voters – Ferrybig Oct 22 '18 at 13:35
20

Well this has whole debacle been saddening and enlightening.

As a silly example: deleted meta SO question

If that or this was a twitter rant instead of a meta post we never would have had another hat that depicts meat. No "Taco Tuesday Any Day," no "Extra Toppings" etc. Contrast that with what actually happened there.

Can we just take a step back here and think about what happened to spark the HNQ change.
A non SE user complains on twitter and nearly instantly the "offending" site is blacklisted.
Just read that last sentence there again. Persons, issues, money and politics aside that is crazy.


I am very disappointed in how SE handled this situation. Putting aside the end result, (the end does not justify the means) what SE did was shameful. In about 40 minutes SE had decided to unilaterally dis an entire site in the network, to placate an offended person on twitter. SE did not stand up for itself, for its (perfectly acceptable and on topic) content, for its mods, or its users. We got nothing but a knee jerk reaction along the lines of "Eek a semi famous self proclaimed feminist activist said something bad about SE. We don't want to look bad. Quick throw IPS under the bus, and pull everything." To quote a line from a post Monica Cellio wrote about this:

I feel like placating one person on Twitter is more important to you [SE] than doing right by your moderators and communities.

I would have much more faith in SE, and feel better about this if you (SE) had either responded from the company, and had a meta and or blog post for what you are going to do about the "HNQ issue". Or skipped responding on twitter (remember phases like "adding fuel to the fire" and "don't feed the trolls") and just went straight to a community post. (This question would of been a much better way of starting.)

No instead we get a really bad taste in our mouth, and a feeling that SE cares more about its inclusive and diverse image then about its users and content.


Now Tim I know you will read this, I'm not upset at you, in fact I'm glad that there is a messenger between the inner workings of SE and us. I know you had to handle a pretty ugly situation. I trust that everybody responding had the best of intentions. However actions speak louder then any words. By SE's swift and decisive response, unfortunately SE's priorities have become clearer.


As an aside I waited over a week, and have read through everything that transpired so I could detail my thoughts with a clear and cool head. There were some tweets, that were very upsetting.

How do we move forward as a community? I do not know. The damage has been done. Regaining trust is a tricky process.

  • Don't agree. The outsider perspective is worth its weight in gold. It is exactly how 99% of humans see you, and you only get it once in a lifetime. How quickly we forget. – Harper Jan 15 at 0:33
19

The general problem

  • HNQ needs improving. Let's continue discussing that long term.

The specific problem

  • Someone made a public complaint out of context. It is claimed that person does not have a Stack Exchange account.

This makes me ask, why are we showing HNQ in the sidebar to users who are not signed in? Is a significant amount of the benefit HNQ brings coming from such users?

How about only showing HNQ in the sidebar when signed in? Users who are not signed in already see more advertising (which spares signed in users from it). Why not show reasonably non-distracting adverts in the sidebar, and replace them with HNQ when signed in?

This is not a fix for the many other problems that need discussion. It's just one simple suggestion for one simple problem.

  • 1
    From the list of goals in the OP it seems that exposing questions from sites that the user is unfamiliar with is central to the purpose of HNQ. Excluding non-signed-in users may shrink the problem but would also shrink HNQ's usefulness at all in pretty much direct proportion. – Will Oct 25 '18 at 8:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .