44

Related: Ease up on the Hot Network Question Randomness.

This is a recent change which resulted in a 30% increase in engagement. All of this engagement came from repeated clicks... we do have improved algorithm changes coming...

Some questions appear to stick in the hot list for quite a long time. As a result, site visitors complain about being repeatedly fed from sidebar with useless links to already visited questions (1, 2, 3):

every time I see the logo, I have to remember a) if I have already clicked it and b) if I have already decided not to read it. This adds some effort to the task, discouraging me from repeating it... Don't show the questions already visited....

Why hide a hot question? You've read it. You're not interested. You want to see more hotness...

I use iOS app mostly to read Hot Questions (95% of the time)... I use iOs app every day (important), and it is a nightmare to read same question title twice or trice...

Suggest to tame above issue by introducing more randomness and diversity in the selection for hot list.

Specifically, pick greater than 100 amount of "candidate" questions (150... 200... 300 etc) and then randomly pick "final 100" of these questions to get to hot network list (picking "finalists" could probably use pretty much the same algorithm that currently randomly "shuffles" hot questions to be displayed on sidebar).

Propose to test suggested change similar to how it was done with shuffling for sidebar.

Improve this question
10
  • attentive readers may notice that effect similar to the suggested change could be achieved by just making hot list larger ("150... 200... 300 etc"). It took me a while to pick which approach to present in this request. Straight enlarging-the-list has a strong advantage of being simpler, howerer in the end I picked to present a "two-phase approach" because it seems to offer more diverse and lively hot list. If you think of it...
    – gnat
    Commented Jan 16, 2015 at 18:41
  • 3
    ..."candidate questions" that don't make it into final list are sort of "protected" from additional upvoting boost from network-wide audience (snowball effect) and this levels playing field for them to compete in hotness with other questions, which are also voted within site audience. As a result, next round of selecting candidates gets better chances for picking fresh questions that outscored prior candidates, making the list more diverse and lively
    – gnat
    Commented Jan 16, 2015 at 18:42
  • 1
    Regarding your last edit, I don't think it is related to the amount of time a question is "hot". From what I understand, OP there means that he see the same question when refreshing the list in the same day, even in same hour, because there is no check "Did I already see this question?", or "Did I visit it?" etc.
    – Shadow Wizard
    Commented Apr 19, 2016 at 12:54
  • @ShadowWizard if system would regularly shuffle 200-300 "candidate questions" to get to final list of 100, this would definitely solve issues mentioned by OP in that question. Not to mention that when questions (from smaller sites, ie vast majority in the list) stick in the list for several days as it is of now, no amount of refreshes would save OP of "nightmare to read same question title twice or trice"
    – gnat
    Commented Apr 19, 2016 at 12:58
  • ...thing is, hot list became sticky, static and boring (except for SO questions but these are minority) and various complaints show just that. If system would mark "visited" questions for the reader, they would soon discover that list is terribly repeatable. By the way because of that SE mgmt will likely ignore or decline such feature requests. As they love to say, "it would amount to little more than a tool to feed speculation and paranoia..."
    – gnat
    Commented Apr 19, 2016 at 13:05
  • 1
    I see, but I suspect the iOS app and android app Hot Questions are different than the "real" Hot Questions, i.e. those seen here. I will try to prove/disprove this soon.
    – Shadow Wizard
    Commented Apr 19, 2016 at 13:09
  • thanks @ShadowWizard - please ping me if you find that these are indeed different, I'll rollback the recent update then
    – gnat
    Commented Apr 19, 2016 at 13:13
  • 1
    Sure, will do. My assumption is the API can't really return the HNQ, so the clients are using their own formula.
    – Shadow Wizard
    Commented Apr 19, 2016 at 13:16
  • @ShadowWizard FWIW this recent answer suggests that mobile HNQ "are taken from http://stackexchange.com/hot-questions-for-mobile in JSON format"
    – gnat
    Commented Jun 20, 2016 at 11:49
  • 1
    Nice find, that indeed makes sense.
    – Shadow Wizard
    Commented Jun 20, 2016 at 15:06

4 Answers 4

Reset to default
15

It is interesting to observe how current design gradually reveals signs of its limited ability to scale.

One issue is related to growing number of sites in Stack Exchange network. There are more than 100 already so it's inevitable that some sites miss the chance to get into the list. And after the list-of-100 is formed and pushed to sidebar audience, "snowball" / "self-fulfilling prophecy" effects make it harder for questions from missing sites to get into the list.

Sidebar audience answers and upvotes increase score of "lucky" questions in a way that is really hard to compete against organically, especially in first 7 hours after question is posted, while hotness score is not compensated by aging factor.

  • Increasing amount of candidate questions would naturally make it not an issue at all. No matter how many more sites enter the network, number of candidates can be increased to account for that, so that any site would have a chance to get their questions shuffled into the list.

Growing audience of the hot list poses another challenge to current design. The more people see hot questions at the sidebar, the less of a "dynamizing" effect shuffling has. Shuffling was supposed to spread audience attention more evenly among all 100 questions in the list, instead of focusing it on 5-10 that were on top (making these stick in there for too long)...

...And it still does that, every one of 100 questions is getting a fair share. Thing is though, increasing views on hot list questions (along with respective increase in answers and upvotes) make it so that all of 100 questions start getting "too much exposure" so that it becomes too hard for new questions from outside to push "old-timers" from the list.

In a sense, this is a return of an old known issue that was supposed to be addressed by shuffling. Only difference is that in the past questions tend to stick on the top of the list, while now they more and more stick somewhere in the list.

The more people watch (and occasionally answer / upvote) questions in the list, the more static, sticky and boring it becomes, due to, again, snowball / self-fulfilling prophecy effects that prevent other questions from being exposed.

  • This challenge also can be addressed by increasing number of candidates for the list. No matter how sidebar views increase, amount of candidates for shuffling could also increase to ensure that audience attention is spread as diverse as desired and snowball effects are under control (so that aging factor can compensate these).

It is even possible to dynamically adapt amount of candidate questions to views, to avoid the need for manual tweaking as views increase. Data needed for that is apparently available ("30% increase in engagement... came from repeated clicks").

    AmountOfCandidateQuestions = SomeConstant + (SomeOtherConstant * log(RepeatedClicks))

Granted, latter issue doesn't noticeably impact Stack Overflow, because of special adjustment that strongly pushes their questions from the list after 7 hours.

But on smaller sites one can see how potentially entertaining questions miss their chance to be exposed at the sidebar which is occupied by older, "snowballed" ones. And since questions from smaller sites make vast majority of the list, this makes an overall impression of it becoming less lively.

Improve this answer
6
+400

How about a different filter shuffle algorithm? For each time long time interval (minute/hour/whatever) each site contributes no more than ten questions to the big list using the current algorithm (or a close cousin).

The top question from each site and the top two hundred remaining questions from the big list get moved to the medium list which is then shuffled.

Every medium time interval (about 1/10 long interval) thereafter until the next long interval the three most popular hotlist questions (only counting clicks on the hotlist) are removed from the medium list and replaced with the top three questions from the long list.

Each short time interval (one second or less) a number of items are selected from the medium list and copied to the shortlist which is shuffled again. The first n entries on the shortlist are shown in the sidebar.

Improve this answer
4
  • I doubt that every site is capable of maintaining as many as ten reasonable "candidates" to hot list but this is minor nitpick; there could be per site adjustment to take care of that. One thing for sure, this way looks capable to scale up smoothly as number of sites and readers grow
    – gnat
    Commented Jun 16, 2015 at 17:32
  • 1
    ..and, yeah, the way how top-three positions in medium and long lists swap could probably help mitigate "snowball effect", pretty interesting. Genuinely popular questions shuffled off this way would probably get back into the list in next round, but those that aren't would be pushed away by newer questions. This slightly resembles effect one can currently observe with Stack Overflow hot questions (due to their special, "merciless" aging factor) and have to admit, their dynamics looks more appealing than "turtle races" of hot questions from smaller sites
    – gnat
    Commented Jun 16, 2015 at 17:42
  • 2
    You will notice that I did not specify a minimum number each site contributes, just a max. Some sites may well just contribute one or two, or even none at all.
    – hildred
    Commented Jun 16, 2015 at 18:23
  • oh I see -"no more than ten" is there indeed. I missed that, sorry
    – gnat
    Commented Jun 16, 2015 at 18:28
3

Yay! Let's see what's on the Hot Questions List... nearly the same thing.

Seemingly random.shuffle() doesn't work anymore nowadays. Seemingly a lot of the questions belong to graduated sites(which in turn have a lot of users and traffic). It can be rather hard to decide appropriate candidates to go on to be the 100 current questions selected. So seemingly, the problem is the definition of a hot network question. So what is a hot network question/what should it be?

Hot network questions should be questions that attracted more views than most other posts on their site. Lots of votes and comments and answers can make a question much more hot. Questions on high-traffic sites and new, young betas should be more likely to be featured. Or just use a random list of sites to choose from.

EDIT: Now instead, how about improving the algorithm through allowing newer questions higher chances of appearing in the list and removing old questions faster. After maybe 2 days (48 hours), remove the question and allow other questions to take the empty spots. This should allow a little more diversity with the hot questions. If possible, maybe allow the algorithm to adapt to number of views per question, allowing questions with less views to have a slightly higher chance of appearing.

Improve this answer
1
  • for the sake of precision, there are 100 questions selected, not 24. And I recall times when shuffle was working just fine, it was just a bit over a year ago, when it was introduced. Back then, I didn't expect it to bump into scaling issues, at least not that fast :)
    – gnat
    Commented Jun 18, 2015 at 21:58
0

Each logged in user sees hot list questions so firstly the server should maintain a log of questions that had been shown to that user and on that basis show new questions (that are not in the log) to them whenever the user refreshes or logs in again.

This log maintenance on server would be quite hectic as many users are present hence the log should be maintained on client maybe in form of cookies. But after a lot of site visits the cookie would become bulky for the user as it would require much of his device space and also would be useless if the user logs in through different devices. Hence maintaining the log on server is better approach from user point of view.

Say that the log is maintained on the server then server cannot have a fixed number of hot questions set from which some (say 10) are selected at random, instead server need to make sets of 100 questions from all questions arriving and sorting them on the basis of most relevant to the user.

And as the user exhausts a set a new set should take its place and also keeping in mind that the next set should contain newly arrived relevant questions not more than one next set should be prepared in advance.

Improve this answer
1
  • Is my answer near to the answer you were looking for @gnat
    – Ashwin Kheta
    Commented Sep 19, 2016 at 9:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .