Increase the Hot Network Questions randomness (proposal for testing)

Question

Related: Ease up on the Hot Network Question Randomness.

^{This is a recent change which resulted in a 30% increase in engagement. All of this engagement came from repeated clicks... we do have improved algorithm changes coming...}

---

Some questions appear to stick in the hot list for quite a long time. As a result, site visitors complain about being repeatedly fed from sidebar with useless links to already visited questions (1, 2, 3):

every time I see the logo, I have to remember a) if I have already clicked it and b) if I have already decided not to read it. This adds some effort to the task, discouraging me from repeating it... Don't show the questions already visited....

Why hide a hot question? You've read it. You're not interested. You want to see more hotness...

I use iOS app mostly to read Hot Questions (95% of the time)... I use iOs app every day (important), and it is a nightmare to read same question title twice or trice...

Suggest to tame above issue by introducing more randomness and diversity in the selection for hot list.

Specifically, pick greater than 100 amount of "candidate" questions (150... 200... 300 etc) and then randomly pick "final 100" of these questions to get to hot network list (picking "finalists" could probably use pretty much the same algorithm that currently randomly "shuffles" hot questions to be displayed on sidebar).

Propose to test suggested change similar to how it was done with shuffling for sidebar.

Answer 1

It is interesting to observe how current design gradually reveals signs of its limited ability to scale.

One issue is related to growing number of sites in Stack Exchange network. There are more than 100 already so it's inevitable that some sites miss the chance to get into the list. And after the list-of-100 is formed and pushed to sidebar audience, "snowball" / "self-fulfilling prophecy" effects make it harder for questions from missing sites to get into the list.

Sidebar audience answers and upvotes increase score of "lucky" questions in a way that is really hard to compete against organically, especially in first 7 hours after question is posted, while hotness score is not compensated by aging factor.

• Increasing amount of candidate questions would naturally make it not an issue at all. No matter how many more sites enter the network, number of candidates can be increased to account for that, so that any site would have a chance to get their questions shuffled into the list.

---

Growing audience of the hot list poses another challenge to current design. The more people see hot questions at the sidebar, the less of a "dynamizing" effect shuffling has. Shuffling was supposed to spread audience attention more evenly among all 100 questions in the list, instead of focusing it on 5-10 that were on top (making these stick in there for too long)...

...And it still does that, every one of 100 questions is getting a fair share. Thing is though, increasing views on hot list questions (along with respective increase in answers and upvotes) make it so that all of 100 questions start getting "too much exposure" so that it becomes too hard for new questions from outside to push "old-timers" from the list.

^{In a sense, this is a return of an old known issue that was supposed to be addressed by shuffling. Only difference is that in the past questions tend to stick on the top of the list, while now they more and more stick somewhere in the list.}

The more people watch (and occasionally answer / upvote) questions in the list, the more static, sticky and boring it becomes, due to, again, snowball / self-fulfilling prophecy effects that prevent other questions from being exposed.

• This challenge also can be addressed by increasing number of candidates for the list. No matter how sidebar views increase, amount of candidates for shuffling could also increase to ensure that audience attention is spread as diverse as desired and snowball effects are under control (so that aging factor can compensate these).

It is even possible to dynamically adapt amount of candidate questions to views, to avoid the need for manual tweaking as views increase. Data needed for that is apparently available ("30% increase in engagement... came from repeated clicks").

    AmountOfCandidateQuestions = SomeConstant + (SomeOtherConstant * log(RepeatedClicks))

---

Granted, latter issue doesn't noticeably impact Stack Overflow, because of special adjustment that strongly pushes their questions from the list after 7 hours.

But on smaller sites one can see how potentially entertaining questions miss their chance to be exposed at the sidebar which is occupied by older, "snowballed" ones. And since questions from smaller sites make vast majority of the list, this makes an overall impression of it becoming less lively.

Answer 2

How about a different filter shuffle algorithm? For each time long time interval (minute/hour/whatever) each site contributes no more than ten questions to the big list using the current algorithm (or a close cousin).

The top question from each site and the top two hundred remaining questions from the big list get moved to the medium list which is then shuffled.

Every medium time interval (about 1/10 long interval) thereafter until the next long interval the three most popular hotlist questions (only counting clicks on the hotlist) are removed from the medium list and replaced with the top three questions from the long list.

Each short time interval (one second or less) a number of items are selected from the medium list and copied to the shortlist which is shuffled again. The first n entries on the shortlist are shown in the sidebar.

Answer 3

Yay! Let's see what's on the Hot Questions List... nearly the same thing.

Seemingly random.shuffle() doesn't work anymore nowadays. Seemingly a lot of the questions belong to graduated sites(which in turn have a lot of users and traffic). It can be rather hard to decide appropriate candidates to go on to be the 100 current questions selected. So seemingly, the problem is the definition of a hot network question. So what is a hot network question/what should it be?

Hot network questions should be questions that attracted more views than most other posts on their site. Lots of votes and comments and answers can make a question much more hot. Questions on high-traffic sites and new, young betas should be more likely to be featured. Or just use a random list of sites to choose from.

EDIT: Now instead, how about improving the algorithm through allowing newer questions higher chances of appearing in the list and removing old questions faster. After maybe 2 days (48 hours), remove the question and allow other questions to take the empty spots. This should allow a little more diversity with the hot questions. If possible, maybe allow the algorithm to adapt to number of views per question, allowing questions with less views to have a slightly higher chance of appearing.

Answer 4

Each logged in user sees hot list questions so firstly the server should maintain a log of questions that had been shown to that user and on that basis show new questions (that are not in the log) to them whenever the user refreshes or logs in again.

This log maintenance on server would be quite hectic as many users are present hence the log should be maintained on client maybe in form of cookies. But after a lot of site visits the cookie would become bulky for the user as it would require much of his device space and also would be useless if the user logs in through different devices. Hence maintaining the log on server is better approach from user point of view.

Say that the log is maintained on the server then server cannot have a fixed number of hot questions set from which some (say 10) are selected at random, instead server need to make sets of 100 questions from all questions arriving and sorting them on the basis of most relevant to the user.

And as the user exhausts a set a new set should take its place and also keeping in mind that the next set should contain newly arrived relevant questions not more than one next set should be prepared in advance.