What is the Goal of "Hot Network Questions"?

Question

There has been a tug-of-war in the hot-questions list.

Community members like JonW seem to be unhappy with the traffic that it brings to their site:

'But we want to encourage people to post, that's the whole point of the HQ list!' I hear you cry. I disagree. We want to encourage people to the site not just to that question.

The SE Community Team seems to have a different opinion as Shog9 points out (emphasis mine):

the results have been... Not great so far: a significantly smaller number of people are clicking through to randomly-selected questions than to the top questions, which hints that the algorithm may've been doing a better job of identifying general-interest questions across topics than some expected.

^{Disclaimer: This should not be taken as a slight of the community team whatsoever, nor do I think this is some cause for revolt or a boxing match as the below prose may indicate. These are just poorly applied literary tools to emphasize the drastically different approaches to the same list between two groups.}

In the Red Corner, the Community Members

The goal of the hot questions should be to drive up interest in the site. The hot questions should be a lure to encourage SE network users to contribute to other content, not just do a drive-by on the hot question.

In the Blue Corner, the Community Team

The goal of the hot questions should be to drive traffic to general-interest questions. After all, the Hot Network Questions used to be more accurately named as "Popular Questions".

What is the Goal of Advertising Network Questions?

Before discussing how to calculate hotness, or how the list should be ordered, we need to come to an agreement on what the heck we are actually trying to achieve. Once we know what we are looking to accomplish, we can find the best way to do that.

The list of questions from a variety of sites is in a great location screen-wise, it is readily accessible and does get a lot of eyes on it. But as with any marketing, the goal isn't just to grab eyes, it's to grab the right eyes.*

^{* I have nothing against left eyes. Most of my friends have left eyes too. And they are awesome. But in the context right eyes are not a geospatial thing, but rather in the 'correct' sense.}

So what are the right eyes? What type of people do we want to attract to our site? What would we determine as 'success'? How can we measure that success?

Please do not limit yourself to the very narrowly scoped topic above. Think outside the box if you'd like. On every page across the network we have a nice piece of real estate for showing off the rest of the network. How can that space best be used if not on a list of questions picked by an arbitrary algorithm?

Answer 1

I actually have a slightly different opinion of what "hot" questions are good for: entertainment. When I'm bored, tired of doing actual work or waiting for something to finish running, they're almost always good for a quick - and ideally informative - diversion. This is what I've always used the homepage and per-site hot lists for, and now that network-wide hot questions are available on nearly every page it's what I use that for too.

Disclaimer: this is my own opinion, based on how I've actually used these sites for 5+ years now, and doesn't necessarily reflect the opinions or goals of SE, Inc. I don't think it's all that crazy though: let's face it, when you're actually trying to solve a problem you don't care about "hotness" - you want accuracy. So what could the point of such a feature be, if not some good ol' Reddit-style idle entertainment?

The hot questions should be a lure to encourage SE network users to contribute to other content, not just do a drive-by on the hot question.

That's a good goal, but... I can't help but think "hot" lists are a terrible way of going after it. Let's face it: these aren't tailored towards anyone's interests; the only way they're encouraging answers is by just shoving questions in front of so many eyeballs that the probability of someone with knowledge of the topic sees it by chance. Don't get me wrong: a big part of the SE 2.0 model for site creation relies on folks having a wide range of interests, and jumping at the chance to participate in sites dedicated to those interests when they emerge. I have no doubt that pervasive, network-wide "hot" lists help in this goal, but calling that the primary purpose is akin to saying the primary purpose of convenience stores is to fund The March of Dimes.

10 reasons why dumb content is popular (and one crazy trick for mitigating this problem)

One more thing... Some folks love to complain about how terrible questions often end up being popular, and how poorly they reflect on the sites they crop up on. I agree: popularity often indicates a certain "lowest common denominator" focus. We've known that for years: that's why new users can't vote, why voting and close-voting are separate systems with separate privilege levels, why we struggled for years to kick embarrassing "what's your favorite..." questions out of Stack Overflow's Greatest Hits.

Guess what: if your site is full of crappy questions, your site sucks - even if they're not highly-ranked by your own users, folks are finding them via Google, and that's where the vast majority of your readers are coming from. You can work to fix that - as painful as that process is - or you can bury your head in the sand and blame it on all of those stupid people from elsewhere. If you think "hot" questions are a serious problem for your site's quality, then you're already ignoring a much bigger problem. Blaming someone else is easy and fun for the whole family - but it doesn't fix anything.

Answer 2

1. That (significant) screen real-estate should be used to strengthen communities with like-minded users
2. Communities should be given more control over how that space can be used and what shows up on it
3. If there is a need for brainless entertainment, the harm to those communities from the entertainment should be minimized

Open Source Advertising

I don't think SE should aspire to being a reddit or the like. I don't think that entertainment should be promoted over the core of creating accessible high quality answers to questions.

Right now SE uses open source advertising because they realize the value of the SE audience, and how much of a positive impact they can have on open source projects. This is a noble goal, and I'd like to think it is continued because it's working.

The same should apply to the hot questions. Like open source projects, the communities that make up the SE network require committed users with interest who can make positive contributions. This has an added benefit over the open source advertising, because it strengthens the SE network and brings more eyes and adds more value over the long run.

Hot questions currently aren't doing that. As Shog says these questions appeal to the lowest common denominator and are being used for entertainment only. The conversion rates on these new users isn't great, and while it brings a lot of eyes (and added community moderation burden), it doesn't seem to bring much long-term value to the communities that the attention is brought to.

Give Communities More Control

Like open-source advertising, let communities decide how their site should be promoted. Give at least a part of that hot questions section to an open-source advertising-esque advert for a selected community that rotates every X weeks. If communities want to promote more serious questions, or tough unanswered questions, or the most useful questions -- let them. This will grow the value of the network.

SE is based on the concept that individual communities know best what they need. That's why we have per-site metas, and per-site mods, and per-site guidelines. Not all communities may want to be promoted in the same way, and giving us the option to choose how we want to promote ourselves (or not promote ourselves as the case may be) is a natural extension of community moderation.

Minimize the Harm of Entertainment-only Questions

If you want to leave a set of entertainment questions below those adverts, that's fine. I just ask that the harm from them is minimized. If the goal is entertainment, then:

1. The questions should be read-only (no posting answers without a bit of effort)
2. Communities should be allowed to remove questions from the list at will
3. The community moderation team should take a part in handling the added burden

Read-only

If the goal is entertainment, that's fine. Don't let people answer when they click through to that page. The real harm is in the horrid answer quality as people add their two cents without reading the other dozen answers, or actually giving any thought in to the quality of their post. If you create a natural barrier between reading for entertainment, and being able to post an answer, that will do a lot to mitigate the harm.

As an example, you can create a read-only version of questions that require people to click through to the main page and re-find the question to post an answer. Those looking for brainless entertainment are less likely to go through the effort, and some of the worst offenders (one-line answers) can possibly be avoided by adding a single hoop to jump through.

Allow Removal from the List

Give trusted users (10k or 20k+) the ability to remove posts from the hot questions list. For whatever reason, if a question shouldn't be on there, the community should be able to purge it.

I would suggest allowing a tag to be added to posts that prevents it from showing up on the list, like a mod tag, but accessible to trusted users too.

CMs should Share the Burden

If these questions are going to be left around for entertainment value, then the Community Moderation team should park themselves in the chat rooms of all sites with a hot question. Smaller communities do not have as many mods as the big three, and oftentimes mods are not available to handle the flood of flags these questions can cause. Regardless of if the other two options are implemented, community managers should be enticed to come and help handle the burden of moderation for something SE wants. A moderation SWAT team if you will.

I don't mind entertainment being a nice time-waster, but it shouldn't hurt the communities it is promoting. That's just pouring salt in the wound.

Answer 3

The ideal hot questions list would present the best posts of the network that are of interest to a wider population than only experts on that subject. The hot questions are shown to a much broader audience than the source site alone, showing them very specialized posts that they won't understand is not really useful.

I think drawing some attention to the larger SE network is one important role of the hot questions list, but I don't see recruitment of new users as its main purpose. It raises awareness of other sites that the users might not have noticed otherwise, and ideally it showcases some good content, and that might lead to some users staying on the site. But the whole mechanism is not targeted enough to be very effective in that regard, the hot questions list is shown to a large population where only a tiny part is likely an expert in the subject of one specific site. I think the best ways to recruit new users need to be targeted, e.g. recommendation from a colleague or a link to the site in a place frequented by experts on that topic.

The hot questions list becomes problematic when it identifies questions that are popular, but are not really of a high quality. When the algorithm identifies a question where the community of the source sites thinks the content is not very good and maybe even a bit embarrassing for the site, there is a problem. The questions should have popular appeal, but they should still be something even experts are not embarrassed to have as a showcase for their site. Often an excellent answer alone can raise a mediocre questions to something that deserves a place in the hot questions list.

So what the algorithm should identify are questions (and their answers) that are considered good by the source community, and of interest to a much broader audience. Looking at how many users click on entries in the hot questions list is a good way to judge the broader appeal, I think. But I also think that the quality aspect needs to be considered as well. The best way I can think of to examine this would be to look at the difference in voting between users active on the site and users that arrive at the question from the hot questions list. If the source community votes very differently than the "outsiders" looking at the question, this might indicate a problem.

Answer 4

TL;DR: What else are we expected to vote down and close at Workplace and Programmers?

After discovering that one of the answers reflects my own position quite closely I wasn't going to go out on a limb but since a suggestion was made...

...Guess what: if your site is full of crappy questions, your site sucks - even if they're not highly-ranked by your own users, folks are finding them via Google, and that's where the vast majority of your readers are coming from. You can work to fix that - as painful as that process is - or you can bury your head in the sand and blame it on all of those stupid people from elsewhere. If you think "hot" questions are a serious problem for your site's quality, then you're already ignoring a much bigger problem. Blaming someone else is easy and fun for the whole family - but it doesn't fix anything.

...I would like to address that.

I participate in two sites that suffer from overly hot questions: Programmers and Workplace. In my experience, communities at these sites close inappropriate questions pretty fast, which wipes them from hot list and makes it not an issue.

Questions that really suffer from senseless over-exposure are reasonable ones, those that match site quality requirements and don't deserve closure, nor even downvotes, neither from passers by nor from community regulars.

The only remaining way to "fix" these questions, if one takes above suggestion seriously, is to cast unfair down and close votes. However pragmatic it may sound, I believe that this is a fundamentally wrong way, as it suggests bending community morale as a workaround for feature that just poorly implements specification.

---

_{Frankly, I feel quite embarrassed to see this suggested by a community manager. It feels like being told by a police officer "hey it's your fault of being robbed because you did't carry AK-47".}

_{Oh and please spare me of extending this analogy to mumbling about dangerous districts, abandoned by law, where one would better enter armed. If a particular district of Stack Exchange City turns out broken and abandoned, there are known ways to stop the disorder and add a discipline, without letting the damage spread over and promote itself in the hot news.}

Answer 5

Well, I can tell you what I hoped for when I was a regular user on a tiny site: network-wide exposure. When Biblical Hermeneutics showed up in the hot question list, I vividly recall day-dreaming that everyone who had an interest in the Bible on the network would see our little site and have a burning desire to ask and answer hermeneutics questions. It was a good feeling to know that Jack's question and the answers it had already accumulated would get some exposure. It felt like our little community had earned a collective gold star.

Now I'll admit it was a bigger deal in my mind than it was in reality. As of this writing, the question has fewer than 600 views. It's not anywhere to be found on the first page of BH's Greatest Hits. Likely we didn't get even one new user out of the deal. And we certainly didn't have any crowd control problems. But you know what? I'm still proud of our little community and that question. There aren't a lot of achievements a site can earn from the time pro tem moderators are appointed until graduation, but having a "hot question" is one of them.

Back then, it wasn't as easy to find network-wide hot questions. Now that they are listed in the sidebar, it's not uncommon for me to wander over to a site I'd forgotten even existed. (Hello, Code Golf!) Ideally, I will find a question that isn't just entertaining to read, but also sparks my interest in writing something. There are some brilliant, eloquent, and thoughtful people on our sites, so I think some cross-pollination can strengthen the whole network.

The other day, I was looking into the RPG site and trying to figure out how it got a healthy bump in visitors since the beginning of the year. As I dug, it became clear that one factor has been direct traffic from other sites on the network. Digging a bit more, I turned up a series of "hot questions" such as Is 3d6 the same as 1d18? Since you don't have to be an expert in a particular RPG system to read, appreciate, or even answer this question, it attracted considerable attention (11,377 views so far). Site traffic also spiked noticeably the week the question was asked.

It was not without pain: two answers were deleted and the voting seems disproportionate. Arguably the question is not even on topic, yet it's tied for 26^th on the all-time best questions list. While the answers tend to be good, none of them demonstrate any particular RPG expertise. But the question also got two solid first answers (one from Eric Lippert and the other from someone who has provided an equally upvoted answer). All-in-all, the HQ list looks like a net positive on that site.

Speaking of rolling the dice, hot questions are a sort of gamble. They take advantage of the blockbuster business model. Every day there are dozens of new hot questions. Some of them blow up and others fizzle. Most sites consider it a great success to convert just one reader in a hundred (or even a thousand) into an occasional contributor. The hot question formula improves the odds by taking into account indicators of interest and quality signalled by regular users of the site. In other words, if lots of people on a site read, answer, and vote on posts, we can feel pretty confident that showing the question around will attract similar people.

Answer 6

I agree that increasing viewership to good questions and driving users to sites they may wish to join but might not naturally visit are both good goals. I would like to mention a third goal:

In the....other corner, random intermittent positive reinforcement

Studies have shown that random intermittent positive reinforcement can be more effective that purely positive reinforcement. This means that a reward that is given for good behaviour, but only sometime and to varying degrees, is more effective than a proportional reward. This is perhaps because people are always chasing the next "big score".

So what has this got to do with hot questions?

Hot questions are (one can only hope) good questions with a good answer since the upvotes are a major part of what drives them - So the positive part is taken care of.

However, it is rare to get one of your questions/answers on the "Hot questions" list, and I haven't been able to detect much of a pattern (except that the questions seem to be more easily understandable to a general audience and are high quality) - So the intermittent random portion is taken care of.

And I think we can all agree you are heavily rewarded for having a good answer/question in the hot questions list.

My experience

When I got my first answer on the hot questions list it was the most exciting Stack Exchange had been since the initial honeymoon period was over. These rare periodic large rewards are key to keeping long term expert users loving the site.

Conclusion

I'm not saying that random intermittent positive reinforcement should be the main goal of the hot questions list. But it is an additional positive effect to take into consideration.

Answer 7

Pity the poor community members of Workplace. They already have to deal with an endless supply of questions from the clueless who think that the rest of the world owes them a job and pats on the head. These are not necessarily bad questions, but they attract awful answers. Well, maybe they are not bad questions. They are made up of grammatical sentences. They seem to narrate reality as the OP sees it. Sometimes they even offer an opportunity for a really good answer, giving the OP a dose of much-needed reality. However, for every sensible answer offered by a person who has a clue, it seems like there are two or three from the orbit of Jupiter. Does this make them bad questions? Or does it just mean that a community that has taken on the job of TWP has a whole lot of work to do.

These questions are the reality TV of Stack Exchange. Broadcasting them creates work for the community, because it attracts an extra crop of drive-by dweeb answers. So I don't think that Shog9's notion that these questions are the entire fault of the community for failing to police questions is, in fact, entirely fair.

Answer 8

For the sake of completeness...

As of year 2020, the purpose was laid out as follows:

primary purpose of the HNQ List - to encourage movement around the Network and visibility of Network sites

---

...the original designer of the feature explained its purpose as follows:

the intent here is to explicitly expose you to the most interesting questions from across the network, whether you have accounts on those sites or not...

_{For the reference, search this URL to look for more details of how hot questions feature was originally designed and evolved: https://meta.stackexchange.com/search?q=user:1+[hot-questions].}

---

...when Stack Exchange main page (https://stackexchange.com/) was redesigned in 2014 to display hot questions list (https://stackexchange.com/questions?tab=hot), these were supposed to serve purpose described as follows:

• We want to talk to users or potential users, not reporters, investors, etc.
• We want to focus on showing them the kind of content they might find on our sites, with as short a summary as possible of what makes that content better.

Answer 9

I like the hot question feature! Although the algorithm might need finetuning. (it is hard to come up with good algorithms like this.)

I find this whole topic fascinating as far as the dichotomy of group voting vs individual opinion about quality. It seems on Stack Exchange,

votes determine the quality of questions!

This is exactly as is stated in the FAQ/help documentation across sites.

Now there seems to be a lot of attitude circulating that maybe there are cases where votes do not reflect question quality eg reflected in Mad Scientist's answer. But this is a bigger issue than the hot questions list to discuss elsewhere (and I am sure that would be a very interesting discussion/debate). Is there any relevant info in meta on this subject? (have not searched much here)?

The whole design of Stack Exchange is based on the idea that votes determine question quality and that concept is even referred to in the sites-wide help pages! It's a fascinating question to ask where there is a strong discrepancy between votes and quality. But then you have to define quality in some way. Is there a quantifiable way to do that? Suggest that anyone who thinks there is some definition of quality that is not the same as votes, try to quantify that eg using the data explorer. There very well may be other interesting metrics of quality, and it could be very useful to find/experiment with them.

So jmac says in the question "think outside the box" one of my favorite invitations of all! (but alas there is not much occasion to do so on SE, because, really, honestly, it has very many boxes.) My thinking here is, maybe just a good selection algorithm needs to be devised, so the ideas:

• Open the floor for experimentation and others to come up with some different metrics/algorithms for rating question quality. Maybe there could be some new consensus on a new algorithm. What if the different sites could choose different algorithms from some basic list to represent their sites hot questions? or maybe even decide on terminology eg "popular" vs "hot" or "notable" or "remarkable" or whatever! and of course [dataexplorer.se] is a very key/powerful tool/resource for that exercise.

• One of the big issues it seems to me is that sites are not using the mechanism to choose their own site-wide ads much. Is this working much? It seems to be half-dead on several sites that I frequent (eg muster a few votes but not the 6 quorum to pass the ad). This is a very cool feature of SE where sites can choose how to represent themselves site-wide, and it seems to be underutilized to me. If sites are not using this mechanism, and then turn around and complain about the hot question list, that seems dissonant there.