It is interesting to observe how current design gradually reveals signs of its limited ability to scale.
One issue is related to growing number of sites in Stack Exchange network. There are more than 100 already so it's inevitable that some sites miss the chance to get into the list. And after the list-of-100 is formed and pushed to sidebar audience, "snowball" / "self-fulfilling prophecy" effects make it harder for questions from missing sites to get into the list.
Sidebar audience answers and upvotes increase score of "lucky" questions in a way that is really hard to compete against organically, especially in first 7 hours after question is posted, while hotness score is not compensated by aging factor.
- Increasing amount of candidate questions would naturally make it not an issue at all. No matter how many more sites enter the network, number of candidates can be increased to account for that, so that any site would have a chance to get their questions shuffled into the list.
Growing audience of the hot list poses another challenge to current design. The more people see hot questions at the sidebar, the less of a "dynamizing" effect shuffling has. Shuffling was supposed to spread audience attention more evenly among all 100 questions in the list, instead of focusing it on 5-10 that were on top (making these stick in there for too long)...
...And it still does that, every one of 100 questions is getting a fair share. Thing is though, increasing views on hot list questions (along with respective increase in answers and upvotes) make it so that all of 100 questions start getting "too much exposure" so that it becomes too hard for new questions from outside to push "old-timers" from the list.
In a sense, this is a return of an old known issue that was supposed to be addressed by shuffling. Only difference is that in the past questions tend to stick on the top of the list, while now they more and more stick somewhere in the list.
The more people watch (and occasionally answer / upvote) questions in the list, the more static, sticky and boring it becomes, due to, again, snowball / self-fulfilling prophecy effects that prevent other questions from being exposed.
- This challenge also can be addressed by increasing number of candidates for the list. No matter how sidebar views increase, amount of candidates for shuffling could also increase to ensure that audience attention is spread as diverse as desired and snowball effects are under control (so that aging factor can compensate these).
It is even possible to dynamically adapt amount of candidate questions to views, to avoid the need for manual tweaking as views increase. Data needed for that is apparently available ("30% increase in engagement... came from repeated clicks").
AmountOfCandidateQuestions = SomeConstant + (SomeOtherConstant * log(RepeatedClicks))
Granted, latter issue doesn't noticeably impact Stack Overflow, because of special adjustment that strongly pushes their questions from the list after 7 hours.
But on smaller sites one can see how potentially entertaining questions miss their chance to be exposed at the sidebar which is occupied by older, "snowballed" ones. And since questions from smaller sites make vast majority of the list, this makes an overall impression of it becoming less lively.