Revisiting the "Hot Network Questions" feature, what are our shared goals for having it?

Question

Thanks for your input here and elsewhere on Meta.

Please see the results of this discussion on the update post:
Updating the Hot Network Questions List - now with a bit more network and a little less "hotness"!

---

Some things happened yesterday that caused a need for us to (quickly) remove a site's eligibility to contribute to the list of hot network questions. For those of you not familiar with what we've come to call the 'HNQ' list, please visit the link; the list of questions shown as 'hot' on stackexchange.com are displayed in the sidebar of most Stack Exchange sites. Here's an idea of what it looks like:

It's a list of titles, with their origins vaguely identifiable by the presence of the 'favicon' element that identifies each site. We're extremely proud of the quality of content that can be found on every single site in our network. The titles, however, when shown out of context, can be problematic. Anything out of context can be problematic.

This isn't a new problem. We've dealt with trigger words in titles that occur naturally through good-faith questions on sites like Parenting, Gaming, Christianity, Islam and more. While we probably deserve some egg on our face for relying on regular expressions as a go-to quick fix, they've been sufficient for ensuring trigger words and overt clickbait was kept out of the list for the most part.

Then we launched a site that tended to have 3-4 hot questions asked each day (by any way one could conceivably measure velocity and good reception). The titles of these questions, out of context, are just too likely to be inappropriate on our professional and academic sites, so we took the site out of the contributing list. Still with me? Thank you, because there's no way to open this discussion without background.

This is one of many times we've talked about problems with hot network questions. This isn't a new problem. What escalated the problem is that we had to take an entire site out of the mix, which isn't ideal, and I'll explain why.

What goals do we have for the hot question list?

1. They help with discovery. User interviews continue to reveal that folks simply don't often discover just how big of a resource we've become collectively. The list boosts engagement in quite a few ways (we'll admit some of them aren't optimal)

2. They often remind communities to have discussions about scope. If folks see a clear demand for certain kinds of questions, it's worth revisiting conversations about how valuable finding ways to meet those needs might be.

3. Hot questions are extremely great fodder for evangelists, the kinds of people that reach thousands of folks that are likely to identify with the way we do Q&A.

4. Let's face it, they're a jackpot and a curse. A little patch of chaos that grows pumpkins and poison ivy - just usually more pumpkins.

What's broken?

The list of questions is generated through a very expensive query, cached, and refreshed on a set schedule. Giving moderators or even community managers the ability to 'evict' a problematic title while keeping the system in check with what we consider performant and sane are .. mutually-exclusive goals.

Now that we have gotten to the point where we've gone from saying "Let's ignore titles with these words.." to "Let's ignore sites with these titles..", we're seeing a pattern of scale that starts to suck if you think about it protractedly.

What do we need from you?

We want some ideas on ways we can continue to meet our goals of getting the right sites in front of the right people who are likely to appreciate them and participate.

Automatic selection simply will not scale if we continue to push the limits of subjective topic spaces, and we need to continue to do that for the network to grow and remain relevant to the issues folks want to use our software creatively to solve.

We don't expect to solve this problem in one post right away.

Let's say we agree to take out the hot network questions entirely. But, we want something else in its place that continues to pull in some of the benefits, without being such a mindless pain in the neck.

In order to find out what that could look like, we need to consolidate what we want out of it. And that could be "Just turn the darn thing off and be done with it", or "I found my favorite site through that list, you gotta preserve that somehow!" -- all of this input at this stage will be equally good.

Suffice to say, the HNQ (as we know it) is now looking at numbered days, but that number is probably kind of high because it's going to take some work to figure out what voids would be worth filling if we yanked it as we know it now.

So, let's talk about it. And, if this seems like it's a continuation of a conversation, it is, from the moderator chat room. Even though the scope of the discussion is still rough, I wanted to get it out 'in sunlight' early, this isn't something to be settled in an all-mod room.

Thanks for your time, we look forward to finally (at least committing) to settle on something we can all like a bit more than what we've currently got.

And in the spirit of causing tons of people to flood into a single discussion, I'm going to feature this; I just can't help it.

Answer 1

All debate of who did what and why aside, you asked for

We want some ideas on ways we can continue to meet our goals of getting the right sites in front of the right people who are likely to appreciate them and participate.

Determining what is the best content is not a new problem. If all our algorithms fail, we could fall on good old human curation. Which is really the gold standard in the "choose relevant information" field.

I don't think that an automated solution based on behavior will ever be very good, simply because what you want to showcase is not behavior, but the self-image of a community, and these are two different things. For a comparison, look at how people decorate their living rooms. They put in objects which reflect who they are and what they value. This might include things like a happy picture of the extended family, or a piano. In everyday life, family reunions may happen at Christmas only, and the person might spend more time vacuuming than playing the piano. But still, they want the piano to be placed prominently and the vacuum cleaner to stay in a closet. Similarly, when a community shows itself to new users with self-advertising purpose, it will want to choose its nicest questions and not the ones which happen to elicit most activity.

1. Questions would better reflect what the community considers "the best example of what we are about".
2. We have enough users interested in coming here and spending their time on reading questions, answering them and voting on them, so there is a good chance they would also be willing to vote for the "best representative question of this week". We have an example of this functioning with Photography's picture of the week.
3. It can even increase engagement, if people who don't visit a site else start coming back for the weekly vote.
4. We don't actually have a need for automated mining of preference data. The advantages I can think of (e.g. that it eliminates sampling bias) are not really relevant for this specific use of the data.
5. Sites will have equal representation, eliminating the bias towards sites where everybody has an opinion on every question (which coincidentally punishes sites where expert answers predominate).
6. The questions will be only shown after they have spent almost a week on the parent site ** . This means that, when the influx from outside comes, the community will already have cast its votes on the existing answers, downvoted nonsense into oblivion, made its comments and requests for clarification, etc. Once the nonexperts come in, they will at least see some guideposts, instead of drowning out the community's actions.
7. The community will be able to prepare for the question becoming HNQ, e.g. by making the title understandable outside of the site's context.

SE will obviously have to spend resources to get the system in place (and this doesn't have to involve writing their own poll functionality, we can live with setting up something parallel for the voting process), but once it is in place, the actual effort of keeping it going can be shouldered by the community.

Of course, the idea is not without drawbacks. My largest concern is: will the results be any different from just picking the highest upvoted question of the past week? My hunch is that there will be a difference, since question upvoting is not necessarily done with "I want the users of other site to know me by this content" in mind. Still, a low-resource pilot test will be needed to see if this actually the case.

---

* the time period can of course be chosen differently

** We could for example implement a lag time such that questions posted very shortly before voting are only eligible for next week's voting

Answer 2

Instead of "Hot Network Questions", here's what I'd actually like to see:

Highly up-voted, unanswered questions - these are interesting to a lot of people, but an answer isn't inherently clear

Highly up-voted answers - answers that a lot of people appreciate

Bounties - people are willing to spend rep to get an answer, I think we could be better at getting eyes on these

I also personally think that there should be an age requirement for questions to appear on these lists. I don't know what the correct answer is, but a week would probably prevent most forms of abuse happening as at that point it should have been caught on each community.

Answer 3

We already know that Stack Overflow is different. Let's act on that.

Stack Overflow is the flagship site, the one that gets the vast majority of the Google hits and activity. It's designed for that. Its stated purpose is to be a place for expert programmers to get in, get answers to their questions, and get on with programming.

Stack Exchange is the rest of the network, a mix of high-end professional, amateur-tolerant professional, hobby, entertainment, and very miscellaneous sites, about 170 of them.

We already treat Stack Overflow differently. It has features no other site has, and it has needs that no other site has. Why should we treat it as if its HNQ needs are the same as those of other sites? They're not.

In deciding what to do with HNQ, we should make a distinction between SO and SE. The behaviors might need to be different.

Here are some possible applications of that principle:

• On SO, don't show HNQ to people who aren't logged in. Those swarms of Googlers will never see HNQ and thus won't be upset by what they find therein. Do those swarms of Googlers even click through on HNQ enough to be relevant? I don't have the data, but SE does.

• On SE, include all sites by default like we do now. On SO, exclude sites by default; SO should use a curated list of technical sites.

• On SO, apply more stringent gatekeeping -- more excluded words in titles, more hotness needed, a requirement on answer scores, or other factors I haven't thought of yet. Basically, let SO tune the definition of "hot" to SO's needs.

I mean these as independent suggestions, though they could be combined.

SO already has a different top bar, a different left nav, triage, careers, different ad handling, and probably other things. Clearly it's ok to be different, so let's add HNQ treatment to that list.

Meanwhile, Stack Exchange, the rest of the sites, can continue to benefit from the cross-pollination and advertising that (a hopefully reformed) HNQ can bring.

Answer 4

Whatever you do, please make it simpler and more transparent than what we have now.

There is an unbelievable amount of conflicting information about the HNQ criteria throughout the network but particularly on MSE. Most people – even experienced people – just don't know how it works.

• Is there a difference between HNQ and the hot questions tab?
• Do questions 51–100 ever appear in the sidebar?
• Can a question that is several years old appear in the HNQ list?
• Do self-answers count toward the calculation?
• Are all sites (other than SO) weighted equally?
• Why doesn't my question appear? (a.k.a., What aspects of the calculation of hotness points are cached?)

Given that HNQ is something that serves as a major source of traffic and new users for many (smaller) sites, it's worth coming up with something simpler and more transparent, whether that is a simplified formula or a manual process. When it's in place, put a help page for it on every site that describes the goals and functionality.

We need to take the mystery out of HNQ.

Answer 5

HNQ is there to entertain, to educate and to promote

I have the feeling that HNQ in its current form mainly focusses on the entertainment. The algorithm favours questions with many different answers, which either come from sites like IPS or Puzzling/Code Golf, where everyone has a different opinion / solution, or from highly controversial posts. A single, good answer is often sufficient and there's no need for 10 more, but this doesn't show up in the algorithm.

Promotion of small sites is virtually impossible, because on Beta sites, it is difficult to get a lot of views and votes in.

The Solution

Why not

• guarantee each site on SE one, and only one spot in the HNQ (Yes, that would lead to 174 HNQ posts, but currently we already are at 100, so I don't think the list would explode)
• pick a rather new post from each site, and only factor in the score of the question and the score of the answer. Much discussion isn't a good indicator of hotness, it's an indicator of errors in answers that needs to be fixed, or a great controversy. Some option for a formula would be (QuestionScore + Top-AnswerScore) / Time[since max-vote answer posted] and exclude questions with the top answer older than 36 hours or so
• Sort the HNQ list by relative viewcount. As an example, use Qviews / AverageSiteViews, this way you get a relative hotness, and smaller sites get a bit more visibility.

This way, small sites are promoted, and good questions from larger sites are promoted. With only one HNQ-Question per site, it is far more easier to review them to ensure the title is not out-of-context-clickbaity.

Essentially, the Hot Network Questions List should become a Recent Great Network Questions List

Answer 6

Allow the community to control what represents them in HNQ.

I see Monica has already mentioned this, but I want to focus on it as one possible solution.

On IPS at least, and I suspect on most SE sites, a core part of the community and moderators hang out in a chat room and leverage some clever bots to make moderation a breeze. Either through a chat bot system or through a review queue, allow the active and curating members of the community decide when a question will show up on HNQ. This would follow a flow somewhat like this:

1. Question meets criteria to make HNQ (what these criteria are is a can of worms I will not be addressing)
2. Question enters review queue (bot alerts community chat)
3. Some set of voting rules (3 promote votes from normal users with 1 or less dissenting block vote, diamond votes are instant promotion/block)
4. Questions representative of the community but not as 'controversial' show up in HNQ and attract users interested in these 'good' questions

Potential drawback is that this allows active participants in this process to suppress the "scope re-evaluation" factor from HNQ.

This system need not be in place for all sites, I don't think hot questions coming from Physics are going to cause a fuss.

Answer 7

My feelings are mixed. I'd prefer not to lose the feature, but it's not good in its current incarnation.

On one hand, it has made me aware of (a very very few) sites I might not have found otherwise that are relevant to my interests. (However, most sites in which I participate, I found via good quality answers searchable on Google and targeted at a specific current need.)

On the other hand, it doesn't seem to go a good job of publicising good questions and/or small sites that would benefit from the traffic. It's dominated by sites that already have high traffic (I guess that's the nature of 'hot') and prioritises (as others have commented) 'controversy rather than quality,' as well as anodyne questions on sites that attract a high volume of answers, which may not be the most attractive questions for much of our wider population of users (although they're clearly valuable to the person asking).

If the sites eligible to participate in HNQ is increasingly restricted, we'll end up with a worse mixture of pap and click-bait. One of the small sites I participate in prides itself on curating high quality questions (yes, we get rubbish but it's usually down-voted/closed/improved pretty quickly) but we've never featured on HNQ, I suspect because we're too small --- so we'll never be discovered that way.

And on the gripping hand, I'm eternally grateful that HNQ introduces me to sites that I am very happy not to visit again, unless I'm having a spare ten-minutes when my eyebrows have already been combed! The questions might be "hot" but I'm old enough to mutter under my breath and move on... naming no sites (there's more than one).

What to do?

• First thing I'd do is abandon 'hot questions' and move to 'network-wide featured questions' or 'you might be interested in' -- let individual sites choose what they'd like to publicise within certain constraints (it would be nice to extend this to the Twitter feeds as well, and within sites as well -- don't just feature questions with a bonus). This is assuming (a big if?) that SE wants to bring attention to its lesser-known nooks and crannies. Yes, more work on the moderators -- or you could base it purely on question up-votes (within the context of individual site) rather than on answers....

• Then I'd look at the algorithm to ensure that the sites that already get plenty of traffic don't swamp the ones that really need the exposure. SO already gets the traffic -- does it need more? Will it really be disadvantaged if it doesn't 'feature'?

• Then I'd look at specific workplace concerns -- turn the list on by default but allow people to opt out.

This contribution is almost certainly too late to matter, but anyway...

Answer 8

I commented my visions for the Hot Network Questions sidebar during a previous post. There was a lot of consensus, but from the people with an actual say in the site, it sounded like that wasn't the time for addressing that feature. Here we are again, so please put some consideration into giving us an ability to filter out what sites we want to see on there. There are so many that people don't care about, and it isn't the same for any two users.

Answer 9

A lot of good suggestion has already been made here but I have others too, so here it goes:

_{For the record, "I found my favorite site through that list, you gotta preserve that somehow!" but that doesn't mean the HNQ list can't be improved.}

Several people have already suggested it, but I'm all for allowing trusted users (meaning, users with enough rep that understand the site policy) to vote for what question should be on HNQ (and, maybe, which one shouldn't).

But the important point here is that the user posting the question should also have the possibility to remove their question from HNQ at any point (I, myself, had several questions on HNQ and having the possibility to remove them from there would have been nice).

Also, from the beginning we are talking about good question but nominating a question for HNQ thanks to a good answer might also be an interesting idea (but maybe I'm just rambling here).

---

Also, regarding the issue of users seeing HNQ questions that they don't want to see, we have several solutions:

• By default, only show an HNQ list of related site. For example, when you are on Interpersonal Skills (IPS), you can see questions from The Workplace or Parenting but not from Stack Overflow. When you are on Stack Overflow, you can see Ask Ubuntu and Server Fault but not Christianity, etc...

  • With this suggestion, we might also want to adjust the number of HNQ slot a site get based on the "distance" the two site have between them. For example, if you are on Stack Overflow, you can only see one HNQ question from The Workplace but two from Ask Ubuntu, two from Server Fault, etc...

• Having a none default setting that allows the user to see HNQ question from every network site.

Edit, some (obviously awesome) new ideas

Related to the previous idea, we could also show questions (that won't have pop otherwise) based on the tag/key word they are using. Here are some examples:

• I'm on the Veganism & Vegetarianism site, I can see an IPS question about veganism (like, "How can I inform my futur host that I'm vegan?").

• I'm on The Workplace, I can see an IPS question that use a "workplace" tag.

• Etc.

Answer 10

As per the question:

We want some ideas on ways we can continue to meet our goals of getting the right sites in front of the right people who are likely to appreciate them and participate.

One idea is to Display other SE site links in the linked questions sidebar - when users link to other SE sites's questions.

The right tool is already in place, but for some reason it's limited to one particular site, instead of leveraging the existing network of sites.Let the community get the right questions in front of the right people, regardless of the site. This will help users discover other sites, not in a clickbait way, but in an organic way.

You can also possibly leverage this to show only HNQ from sites linked to the current question.

Answer 11

Not "hot" but "good".

I was happy to see your thoughts on HNQ reform in another answer. In particular, you mention review and more moderator control, which are essential if you're just going to pull current active questions like you do now, optimizing for controversy.

But your thoughts got me wondering if we should go one step farther. What, exactly, is the value of hotness?

What I hear from users of smaller sites, and even some larger ones, is that the HNQ list is important for site promotion. Even on The Workplace, where complaints about the HNQ have been loud and numerous in the past, it looks like the community as a whole would rather stay on the list (and pay the cleanup cost) than opt out.

But is the HNQ list the best way to promote sites? Supported by the answers and voting here, I'm gonna say no.

SE used to run single-question "ads" in the "community ad" slot, interspersed with the regular ads. These showed the title of a question, along with its site logo and name, and linked to the question. I haven't seen those in a few years, so I don't know if they faded away or were removed in favor of HNQ or were removed for some other reason. Why not bring that back, and feed it from a query of sites' good, recent questions instead of hotness? Make it easy for sites to advertise their best content instead of their most-controversial content and they can still draw in new users without all the downsides of HNQ.

Criteria will need to be developed. It'd be nice to know what they were for those ads; that'd be a good starting point. I think we are looking for significantly-upvoted questions with upvoted answers. "Significant" should be evaluated in the context of the site, not globally, or Astronomy will never be able to compete with the trilogy. To avoid seeing the same old questions all the time, there's probably also a time component -- qualifying questions asked within the last week (month?), maybe.

Answer 12

We should actively seek feedback from many more users than we're seeing here.

When the twitter/HNQ/IPS fiasco hit and it became clear how easy it would be to pull a site from HNQ, the RPGSE mod-team discussed the possibility. For us HNQ questions tend to get an influx of low-quality (read: untested, speculative, any-old-person-with-an-opinion) answers, strange voting patterns, friction with daily/established site-users, and tend to require mod-work to caretake. (And something like a third of our questions hit the HNQ--this is not a small issue.) I suspect we are not alone among sites' mod-teams in feeling that way about when a "hot" question rears its head.

So we posted a site-meta asking for community input on whether we should withdraw. One mod did a nice job laying out what we usually see when HNQs get problematic, and their initial "let's leave" answer gained some decent traction: +13/-4 the first day, then +4/-4 the second day.

Then, hoo-boy, did we see something interesting. The first "let's stay" answer came in later that day and ended the day at +8/-2. Next day: +15/-1, then +12/0.

But that's not what catches my eye. What catches my eye are the other answers and, especially, the comments on most of the "let's stay" answers: many of them are from users whose names I don't recognize. And they're telling stories about positive interactions with HNQ that, frankly, I'm not seeing anywhere else--not in the Teacher's Lounge, nor in the Tavern, nor in this Q/A.

---

There are answers, votes, and comments here from (SWAG, here) a few hundred SE/SO members. A lot of us posted angry in the first few days, reasonably so. But I'll also note that it's a lot of meta.SE regulars and mods/power users from other sites providing a lot of this commentary.

Tens of thousands (an even wilder SWAG, there) of users interact with the HNQ on a regular basis. I submit, respectfully, that asking on meta isn't a good way to find out what's good or bad about HNQ. HNQ requires well-designed UX research in order to understand how it's even being used right now, never mind how to make it better.

Answer 13

If a site is not an entertainment site, then the HNQ has no place there for the same reason that entertaining questions have no place on those sites.

HNQ questions may as well be featured questions. They are directly in the sidebar, almost exactly lining up with the first answer.

On sites where entertainment questions are absolutely banned, the HNQ should also be summarily banned as it serves no purpose. This has been true for a long time, and deviating from that just causes problems.

Disagree? Take it straight from the horse's mouth (Jeff Atwood), The Trouble With Popularity.

Popularity is a tough thing. I’m tempted to call it a curse, but what we try to do at Stack Exchange is make sure that questions and answers are popular for the right reasons — because they are amazing resources for learning from your peers. If you want to slip a few jokes in there with the learning, that’s fine, but when the question devolves into little more than entertainment, I hope you can understand why our community moderators are obliged to step in and protect the community from, well … itself.

And Atwood was correct. However, what we have today isn't entertainment questions, we have entertainment exchanges. They need to be placed in a separate place.

We know that closing the cookie jar is painful. We feel your pain. Nobody likes having their fun taken away. But it’s too addictive and too easy, and in the absence of any moderation, the community would do nothing but add and upvote the easy, fun stuff.

This is literally the situation being faced at present except on an exchange level. Even the exchange metas point this issue out themselves.

Closing the HNQ off altogether may not make much sense, but it should be separated into entertainment, and not entertainment.

Answer 14

Is this discussion only about the sidebar, or is it about HNQ in general?

I've been subscribed to HNQ's RSS feed for many years and it's probably my favourite. Every day I get to read high quality answers on a very wide variety of subjects. It is interesting, entertaining and educating. I would be very sad to see it go.

I have no opinion on the sidebar: I rarely use it or even notice it because I've seen all those questions in my RSS feed before.

Whatever happens to the sidebar, I really hope that the full HNQ list, along with its RSS feed, never goes away. In fact, I would love for it to be modified to have more entries with a faster turnaround. I would also suggest that any sort of keyword or site blocking be disabled on it. The sidebar may lack context while being very visible, but this doesn't apply to the full list nor the RSS feed.

Answer 15

Let's say we agree to take out the hot network questions entirely. But, we want something else in its place that continues to pull in some of the benefits, without being such a mindless pain in the neck.

I feel like this is entirely inappropriate. Taking it out all entirely.

Sometimes when I feel bored and feel like surfing some Q&A away from SO, HNQ is my way to do it. I even decided to join some other communities just because of that. Taking it away will reduce the traffic a lot and would kill some other SE's sites that I(we) never knew that they existed (somehow).

If the whole idea is to hide those inappropriate titles, here's a way to do it. Let the users do it themselves. Hopefully this isn't something that is already featured.

My suggestion is based on the famous phrase from SE: This site is moderated by you.

My idea is to let each one of us choose which Q&A's sites he would like to see in the HNQ list. Here's some scenarios:

• All current Q&As will be added to the list.
• A new Q&A will automatically be added to the list. Let the light shine!
• User will be able to filter them out from his profile settings.

The whole idea is because if 1000 people find something "rude" or "inappropriate", there might be less or even more people that don't, and would actually like to see such a title. Plus it gives each user the ability to choose which Q&A to follow without having 10+ tabs opened. For example, I might be interested in following the Puzzling Q&A, yet not the chemistry, despite that usually none of them have any bad titles, just, not interesting.

Update:

Lawrence made a good comment about offline users that I didn't think about. Here comes my suggestion about that, which, apparently, can also be used for online users.

Give a new privilege to all users at 3K (maybe?) reputation and moderators on each site to access a HNQ Review Queue. Voting will be simple, keep in HNQ or remove, while of course preserving the right to edit immediately.

What will be inside the queue? Just the questions that will already make it to the HNQ, and each time that a question is added to that queue, give it a RED FLAG to directly check it.. I mean that red circle we see when a queue is "full".

If removed, it won't be shown to offline people, and might as well be hidden from the online users even if they opted in for that particular Q&A.

Answer 16

Simply remove HNQs from the side-bar, leaving only a link.

This should be quick & easy to implement; SE developers surely have more urgent calls on their time than designing algorithms or voting systems for HNQs. (Completing the roll-out of new site themes, expanding responsive design to more pages, making the sites more accessible for people with disabilities, working on the "Ask a Question" wizard, ...)

Any kind of non-automatic way of deciding what goes in & out of the HNQ list will distract users from creating/curating/reading content. (And likely add another source of grievances.)

We can then browse HNQs if & when we want to, without having to worry whether clients or colleagues might be offended or disconcerted by seeing question titles like How can a person be kept alive while being periodically drained of blood? (example from today).

To some the HNQ side-bar is no more than intrusive advertising, & these are therefore going to be predisposed to annoyance at its content; sinking resources into trying to ensure this is quite unexceptionable just might not be worth the candle.

Answer 17

I like HNQ the way as it is now. I like some of the suggestions given here, especially those concerning breaking the positive feedback loop and diversifying the sites displayed. I also have a strange but strong feeling I've read about that very problem and community suggestions on this very site a while ago. Obviously, the burning hot problem is not how HNQ list appears to the community, so I won't write about that and hope it will get incrementally better in the following 6-8 weeks.

What is a problem is how it appears to outsiders. I'm going to align with one answer here which suggests hiding it for anyone not logged in is a proper solution. Put a "StackExchange network hosts answers to many more topics" link or something like that instead. Then, redirect it to main SE site, which by default shouldn't provide a list of hot questions, but rather something representative of good quality questions on the network. This can be a much more slow-moving list -- use a review queue or special tag to feature questions here. After all, primary audience for this are those not familiar with any site on SE who won't drop by more than once or twice, and not someone who would refresh the list waiting for something interesting to pop up.

For new users, set the HNQ to off by default and bury it somewhere in preferences. After they obtain some rep on the site (50, 100, whatever) and get familiar with how things work, show a tip somewhere, asking them whether they'd like to enjoy the obscenities that HNQ (occasionally) provides. That way, no outsiders or newbies get hurt by mentions of threesomes in the sidebar, and even if/when they decide to become a part of the community, it's optional. All that time, a representative sample of questions from all over the network is just a click away.

---

^{Venting box, feel free to ignore}

_{This is one of the most annoying posts here, dancing around "the site" for no apparent reason, which hurts comprehension and readability. Only thing more annoying than that is staff's "(Potential, but probably not given the anger) customer is always right" attitude on Twitter, with no intention to defend or at least explain the controversial titles in picture.}

---

Answer 18

One easy solution would be to give out a "nominate as great question" privilege at a certain amount of rep and/or tag participation. Something that works similarly to close votes, but instead in a positive way. With enough such votes, the question would then go to a pool of candidates that may eventually get their moment of fame in the HNQ.

That way we would pick questions based on content rather than the title. Moderation and appropriateness would already be handled by the privileged users prior to nomination, polished with edits if needed.

If this means we end up with questions that have more/less "click baitiness" appeal, or with questions which titles might appear weird out of context... then so be it! The spirit of the SE communities is that we only care about the quality of the post content.

Answer 19

Please allow the moderator team from any site(s) to control, or be able to ask to opt-out of the HNQ for the site they represent.

That visibility option should be decided by the site community itself.

Answer 20

Since HNQ is there to promote sites, not to attract attention to a particular question, I would suggest to:

• automatically protect the question once it goes HNQ. Presumably, it already received several good answers, so there's little chance that another answer from someone seeing the site for the first time will make a difference.

• ideally, also protect it from (up)-voting by new users. Since newcomers can mostly only upvote, that's what they do, often ignoring the site's quality standards. Even very poor answers (downvoted to a negative score) get their share of upvotes, making posting them worthwhile.

As I see it, "promoting" a site means that newcomers get to know what the site is about, then decide to contribute where their contribution is needed, which would be asking a new question or answering a recent unanswered question, not heating up the HNQ entry. That's what we should encourage them to do. Additionally, breaking the positive feedback loop (more popularity - more answers and upvotes - more popularity) should help with making controversial questions manageable.

Answer 21

"When in doubt, cut it out."

-- surgical aphorism

I propose eliminating the HNQ sidebar entirely and putting the HNQ somewhere else, possibly a tab on stackexchange.com because:

• It's distracting and doesn't add value to people who are visiting a non-related SE who would just as soon not disappear down a link rabbit hole.
• It is now being manipulated so that so it's not really HNQ but "HNQ that we think won't offend anybody." Best of luck with that.
• It takes up screen real estate that could be used for info relevant to the SE that the user is visiting.

Answer 22

Initial Response

I know we can't go back and change it, but I think it is important to discuss what should have been done. This absolutely should have been to temporarily remove HNQ from the sidebar, not remove IPS from the HNQ. Ideally, this action should have been taken after a review instead of immediately. The problem, for the most part, was where the information appeared and (as pointed out in the question) the design of HNQ itself. This is the response that would have addressed that issue.

Going Forward

As a long-time user of the HNQ, I hate to say it, but I'm tempted to say do away with the HNQ completely. That said, there are two main things that I would still like to see that I would lose without the HNQ:

1. I like seeing a summary of useful, fresh questions across the site.
2. I like the publicity sites get--though as pointed out by many, there are problems with the kind of publicity they currently get.

I don't really know how to address either. In regards to #1, I suggest we try different queries on each site to see if we can get a better list. Maybe something like "starred, recent questions"? That might even encourage more people to star.

Also, moderators and/or askers could be granted the ability to temporarily disable the star feature so if the question is controversial or otherwise problematic, it won't hit the list (hopefully). Even that may be too similar to what we have today, or it may break the star feature in some way I'm not thinking of, but I suggest we start thinking of alternatives like this.

In regards to #2, I like finding new sites without having to go to Area 51 or search for them manually. Maybe we could have a random "featured site" spot somewhere?

In Closing

I just want to say--I don't often get as riled as some with SE's handling of things, but this one really ticked me off. I respect Adam and Tim, but they both responded poorly here. It makes me fear for the future of this site. I'm tempted to say Adam's response seems knee-jerk and Tim's seems coordinated.

If true, the latter fact makes me worry if those doing the coordinating understand what happened here: I get not wanting to make it about IPS, but Adam made it about IPS by responding to the tweet in the way he did. You can see it didn't take long for the answers here to turn to that fact.

Answer 23

I notice a lot of people in various "answers" are suggesting keeping the list collapsed by default for new users. I simply wish to point out that there is a downside to that: it could very well reduce discoverability because the content is no longer in the new users' face.

If our goal is to help smaller sites be noticed, or good questions to be noticed, or anything else about noticing, collapsing the list may have detrimental effects.

I'm not saying it shouldn't be done, but that I do believe this should be factored into suggestions.

Answer 24

I don't see why you want to show "hot" questions.

I think that "hot" means, "this question is currently being answered -- perhaps you'd like to answer it?"

But, unfortunately, displaying "hot" questions apparently means that:

• It's impossible for humans to supervise
• It's computationally expensive to implement, you say
• It attracts answers from new users -- which may be a curse mixed blessing, requiring that the question be "protected" (since new users don't know what the site's policies are for what a good answer is).

I'd suggest that, instead, you select and display "good" questions -- i.e. topics where the site's community (users) agree that, "Yes, this is an exemplary topic, a good advertisement for our site."

A selection process could be something like:

1. Select some candidate topics

   • Automatically select candidates e.g. based on votes, date, etc.
   • And/or let the community (users) select (nominate) candidates manually

2. Candidate topics must be approved by a site moderator (this lets a moderator decide that a topic isn't suitable; you could e.g. ask all site moderators to exclude topics related to sexuality)

3. Approved topics are displayed as before.

You might have fewer topics (because it's more work), OTOH the quality will hopefully be higher.

The process would generate a pool of topics which can be displayed on the HQN space, also [re]used as a "Best of sitename" for any other advertising purposes -- for new/potential users, for Twitter, and so on...

Do you even need to find new candidate questions every day (which you're doing at the moment, by selecting "hot" questions)? Perhaps instead it's enough to have a pool of a few hundred exemplary questions per site, which might be recycled occasionally. If there were 300 questions per site, for example, you could display (on HNQ) a different question for each day of the year -- and don't necessarily display several questions from the same site at the same time (maybe one good question at a time is enough to advertise each site).

Answer 25

Weight high-rep users votes more to decide HNQ status

I know this can be a heavier burden to the system, but high-rep users usually know more about what constitutes a good question in their site (after all, their peers voted them up).

How to weight users rep must be tried and tested, but starting with the square-root of their total earned rep might be a good start. This also adds value to reputation.

Conversely, new users and associate bonus-only users votes shouldn't sway a question hotness.

Answer 26

I would like to see some degree of siloing in HNQ.
First, SE sites are already sorted into the following categories:

• Technology
• Culture / Recreation
• Life / Arts
• Science
• Professional
• Business

One simple solution would be that on any given site, the HNQ list is filtered to other sites in that category.
However, I think this would reduce the effectiveness of the discovery support goal.

I propose that each launched site has a mod-team-controlled blacklist of other SE sites which are excluded from the HNQ sidebar on that site. I propose this as a blacklist rather than a whitelist so that new sites can be automatically added as they come into being.

As a starting point, I would propose that blacklists would be empty for all sites, except the "Big 3" (SO, Server Fault, Super User) which would show only questions from sites in the "Technology" category. The links to related questions on the same site might be expanded, and mods can remove sites from the blacklist as they deem fit, probably after some community discussion.

I would also propose that sites in beta do not have such a blacklist as they are in the process of being integrated with other sites in the whole network, and often have less of a full elected mod team.

Also, when a question reaches HNQ, I would really like to see a record of this stored with the question data, like when a question is Tweeted or similar. It throws off so many metrics so far off that it's impossible to meaningfully do research about what makes a good question without accounting for the HNQ effect. More on that to come.

From a technical side, I also propose that instead of caching question titles, the long and expensive query cache only the site identifier and question number. A secondary cache can map question numbers to names and be updated when there is an edit to the question name. This secondary cache that stores only question names should be large enough to hold all the HNQ questions from that site, all the questions on the common front/load/landing pages, and a few more for buffer. Then if there's a problematic title, it can be edited and the edited version will propagate pretty quickly to additional user views.

Answer 27

Here are some of my ideas of how to "improve the HNQ"

Classify the 170 sites with metadata.

Professional, social, volatile topics, academia, etc...

Then have a HNQ list built up for each site depending on "related relevant sites".

IPS would get suggestions from Scifi, Workplace, etc...
Stack Overflow from Server Fault, Software Engineering, Blender, etc...
Academia from Latex, Mathematica, etc...

This way the "hot network questions" are relevant to the visitors of the website and encourage exploring. Software Engineering could lead to Workplace, Workplace could lead to IPS, IPS could lead to Scifi and that way when people start exploring and return to HNQ on relevant sites would explore more sites.

That way you free up competition space for hot questions by limiting the pool where to gather them from whilst still allowing browsing through the different topics by visiting more sites on the network, whilst making the question/topics also more relevant in my opinion.

I'm not sure if this is doable with the amount of data Stack Exchange processes, or wished, because part of the "exploring" will be harder to jump from one site to a totally new and fresh site. But it might encourage intersite exploration on the smaller sites.

edit

I removed the references to NSFW and SFW. Stop focussing on specific small part of my answer please, try to see the broader picture for categorising websites and showing related overlapping websites. And yes, it might be helpful for a person to see a relationship question on how to come out as gay, except if you're a Muslim coder working in a strong Muslim faith company where such things are seriously frowned upon.

For a professional, using a site with such risky questions can pose serious problems if a chief watches over your shoulder.

Answer 28

Showing the hotness score on the question has been suggested as something that would be useful but performance heavy. Assuming this query accurately reflects the hotness score, I'm not sure that's true. It requires the following data:

• Question ID. Already shown.
• Answer count. Already shown.
• Question score. Already shown.
• Average answer score for the top three questions.
  • Individual answer scores on that question. Already shown when ordered by answer score.
  • Number of answers. Already shown.
  • Rank. Easy enough to calculate if you have the answers. Limits answers to top three by score.
• Question age.
  • Time of question creation. Already shown.
  • Current time. Available in ECMAScript.

It also checks post type, but we know the type is question on the question page.

All the rest of this information is already available on the question with the possible exception of average answer score. We can approximate answer score by calculating the average for just the top three among the answers currently shown.

If the number is expensive to calculate on the back end, it is quite possible to calculate in JavaScript after the page is loaded from information already available on the page.

Only question age is continuously changing. Updating post scores and answer counts can be done when people vote, answer, or delete answers. Only the average answer score of the top three is not currently displayed.

Summary

What makes a hotness score expensive does not seem to be the data for the question but checking multiple questions at once. But to show the hotness score on a particular question, you don't need to know the score for other questions.

Recommendation

Show hotness score on the question, either calculate on the back end or in JavaScript. Modify the hotness score calculation if the current one is hard/expensive to calculate in terms of performance.

Answer 29

We have review queues for questions and answers of dubious quality. The same thing would be good for individual SE sites, where people could review hot questions and decide if they are problematic either for their titles or content, as an advertisement for SE.

However, if users were to review this queue on their own site, then they would likely remove from the review queue questions that they felt weren't good quality according to their personal predilections. That's not what we are trying to catch. So I would suggest trialling two things.

• Firstly, any hot question should be hot-question-flaggable for a set of specified reasons. (Not judged according to whether it is a good question for that site, does it show research etc). HNQ's would get a different flag list.

• Secondly, the reason that site-specific questions might be problematic taken out of context is something that users of that site are less likely to see. They tend to see the inherent usefulness of the question in context, and in the context of their own site. However, users of other SE sites will be able to judge pretty quickly whether certain question titles, or question content might be problematic to a wider non-specialist audience. Because there are a relatively small number of HNQ's—only a few HNQ's per site—all questions for certain sites that are known to be problematic could enter a review queue to be passed or rejected by high-ish rep users of other sites.

I think that reviewing HNQ questions from different sites would be extremely interesting for most people, as these question tend to be more inherently interesting in the first instance. It would be highly participated in. It would certainly be more interesting than reading hundreds of poor and uninteresting questions that one finds in close-vote queues.

It would also be a good way to introduce a new incentive for getting higher rep on users' own sites. This is because you could raise a new privilege, for say 30,000/40,000 rep users, of being able to review HNQ's for SE-wide sites (not including their own).

If all questions on such sites had to go through ten pass onto HNQ votes, then even if some great questions got eliminated, there would still be an HNQ list of interesting questions for all sites.

Answer 30

HNQ is neat, but I can see why some of the questions might be a little odd in the context of, say, Stack Overflow. And why some Stack Overflow questions might seem a little odd to nonprogrammers too.

So I suggest: Don't display all of the HNQ on each site. Instead, use another expensive query to compute site/pair relation strengths based on question migration and/or shared user records and then display the HNQs from the most related ten sites, for some value of ten. And perhaps offer a per-user option to include all, and perhaps include some extra-bland sites.