211

Right now the front page Popular tab is fairly broken -- it's a simple descending sort by views. As Joel said in podcast #18, it is "a self-fulfilling prophecy." But this is not intentional, it's only because we haven't had time to improve it yet!

As I sit down to write a better algorithm, I thought you guys might have some insight into what would make a good formula to determine the "hot" questions -- over the last 24 hours, the last week, the last month, and all time.

Here are the variables we have to work with:

  • Votes [aka Score]
  • Views
  • Answers
  • Answer Votes [aka Score]
  • Whether the question has an accepted answer or not
  • Time question was originally asked
  • Time of last activity on question
  • Reputation of asker
  • Reputation(s) of answerers

Joel cited the reddit formula as an example, but I think our site is a little different than just a basic link submission site. Plus, we have more variables and data to work with.

I found a blog post describing a few formulas:

Y Combinator Hacker News

(p - 1) / (t + 2)^1.5

p = votes (points) from users
t = time since submission in hours

Reddit

t = (time of entry post) - (Dec 8, 2005)
x = upvotes - downvotes

y = {1 if x > 0, 0 if x = 0, -1 if x < 0)
z = {1 if x < 0, otherwise x}

log(z) + (y * t)/45000

Thoughts? Ideas? I don't want to overthink this; I just want the "hot" questions to be relevant and useful to us. So the simpler the algorithm the better!

Based on my analysis of the above and the comments so far, here's the second version of what I have implemented so far. This might suck. I don't know:

(log(Qviews)*4) + ((Qanswers * Qscore)/5) + sum(Ascores)
--------------------------------------------------------
((QageInHours+1) - ((QageInHours - Qupdated)/2)) ^ 1.5

Note that accepted answers weight not at all in hotness. This is intentional, as I feel accepted answers are a fine social contract, but not a good data point for question or answer quality.

This algorithm will heavily favor questions with LOTS of answers, as the sum(Ascores) are now included -- one assumes if there are lots of answers, there will be a lot more voting on the answers, too.

Update: Note that this formula is what is used on the hot tab linked from the homepage of each site. It is not the formula used to determine the network hot list. See What are the criteria for questions to be selected for Hot Network Questions?.

13
  • 3
    @Jeff Atwood: What units is the age in? It wouldn't matter if the age variables weren't mutated, so units become very important. I'm guessing days, but it could be anything less than a day (assuming that days are in integral units and rounded down?) assuming you don't want the age value to drop below 1 (so that when raised to a power, it doesn't lower itself).
    – casperOne
    Feb 25, 2010 at 3:05
  • 2
    @casper it's in hours May 2, 2010 at 9:09
  • 3
    @Jeff, I think an important factor to measure (which will lessen the effect of "self fulfilling prophecy"), is how long did a person stay in a discussion. I assume that not-so-interesting discussions might have many views (since they are considered hot, and were advertised in reddit), but people will close the tab short time after they read the first few lines. Jun 25, 2010 at 5:10
  • This question needs updating: meta.stackexchange.com/questions/60756/….
    – tshepang
    Jan 17, 2011 at 10:15
  • 2
    @tshe the SE.com algorithm is a bit different, this is documenting the hot algorithm on the sites themselves Jan 17, 2011 at 21:12
  • @JeffAtwood: is this still true?
    – Trufa
    Apr 1, 2011 at 3:32
  • @JeffAtwood:Qupdated mean is the last activity time to the current time?
    – Charlie
    Apr 2, 2011 at 1:51
  • Are negative Ascores zeroed or abs()'d, or does a bad answer decrease the hotness?
    – Bryan Agee
    Aug 12, 2011 at 22:45
  • 6
    The increase of hotness due to presence of answers is very counter-productive, since this basically turns questions that are probably too broad or over-answered into "hot questions" - questions like this one. You should consider both a lack of answers and an overpresence of these to be bad Aug 2, 2013 at 9:44
  • see also: What changed in the Hot Questions sidebar algorithm?
    – gnat
    Feb 26, 2014 at 11:22
  • 2
    would like to see an algorithm that is not highly unstable & changes every time browser page is reloaded as in current behavior! seems not to be selecting top questions by any deterministic algorithm in that case....
    – vzn
    Apr 2, 2014 at 16:00
  • 1
    @gnat I don't think so, based on the final paragraph of the question?
    – Glorfindel Mod
    Jul 23, 2021 at 13:25
  • @Glorfindel agree - retracted my vote
    – gnat
    Jul 23, 2021 at 15:21

32 Answers 32

124
+100

I don't think that reputation of participants should be considered. As the beta has clearly shown, "reputable" people can have irrelevant discussions.

I believe it would be better to just consider amount of activity, recentness of activity, and total upvotes minus downvotes in discussion.

4
  • 16
    Yeah, I agree. Relying too much on rep not only makes the equations messy, but it increases the elitism factor that every social network must grapple with. Let rep determine what people can do, but let voting determine what people find valuable.
    – Pollyanna
    Sep 19, 2008 at 2:26
  • 23
    I disagree with the minus down votes. The change in votes out to be just as important. A good but controversial topic might get lots of votes and still hover around zero.
    – David L Morris
    Sep 19, 2008 at 2:57
  • 2
    What about not using "raw" reputation counts but one of maybe 3 weight values? For example: 1 for low rep user, 5 for 1k+ user, 10 for 5k+ user. In this way very "reputable" person's vote, would still be only counted as high as another 5k+ rep user.
    – Vlad K.
    Aug 6, 2013 at 10:53
  • I don't like "hot questions" where has accepted answers (where most of them have), because the term "hot" represent that more users are talking to each other in that fairly good post and they are seeking for a good answer. Why I should click on "hot questions" tab? I can watch upvoted posts instead.
    – C.F.G
    Sep 3, 2020 at 9:12
63

Rather than jumping right to the formulas, I think it would be helpful to think about what kind of popularity you're trying to measure. Stackoverflow is a user-generated content site, so popularity should be largely governed by user actions. Most popularity driven sites (Digg, reddit) want their list to change regularly so people have a reason to come back to the site every day (or hour). They usually do this by incorporating some sort of temporal decay. Assuming that this is what you want when you say you want a popularity metric, you pretty much inevitably end up with a formula that is some variation on (user activity) / (time)

So how to measure user activity? Well you listed some possible inputs in the question.

  • Votes [aka Score] - Probably the best popularity metric available.
  • Views - A pretty poor substitute for votes. A view from someone who clicks on a link, spends ten seconds reading and decides this is boring gets counted just as much as a view from someone who reads the entire question and every answer.
  • Answers - What are you trying to accomplish with the popularity page? Do you want it to be a tool to call attention to unanswered questions, in which case questions with fewer answers should be rated higher? Or do you want to call attention to good answers, in which case more answers should be rated higher? Perhaps there should be a default popularity (no answer-based weighting) with tabs for most popular unanswered questions and most popular questions with accepted answers.
  • Whether the question has an accepted answer or not - Same concerns as answers.
  • Time question was originally asked - If you incorporate the time originally asked in the denominator, the popular list will incorporate only recently asked questions. It will be all but impossible for an old question which sees a burst of activity to show up.
  • Time of last activity on question - Using the time since the last activity makes it possible for old questions to show up again, but it also would make it possible for someone to game the system and keep a question they like "popular" for a long time, they just have to keep doing minor things to keep the time since last activity low.
  • Reputation of asker - Given the huge disparities in reputation that have arisin just in the beta period, it seems like using this will mean that newbies will never appear in the most popular list, regardless of how good their question is or how much interest it attracts.
  • Reputation(s) of answerers - Similar concerns to the asker's reputation, a question wouldn't show up on the list until someone with a lot of rep answered it.

I think number of votes is pretty clearly the best popularity metric. Number of answers/accepted answer could be incorporated depending on what you want to emphasize (I think a separate tab for popular unanswered questions would be a great addition).

I'm not really enamored with either of the time alternatives you listed. Instead, I'm thinking the best way to incorporate time would be to use it as a sampling window, rather than dividing popularity by some time measure. Look at the number of votes/views/answers in the last hour/day/week rather than dividing by some sort of time measure. This disadvantages questions that have just been posted, of course, since they haven't had a full hour/day/week to accumulate votes. You could weight the votes based on how long it's been since the question was posted up to the sampling window.

So, I would propose something like:

Popularity = (Number of Votes in last 24 hours) / (Hours since question was posted or 24, whichever is less)

2
  • 2
    A question with no answers in my opinion isn't "hot", and neither is one with an accepted answer because there's no more discussion, it's resolved.
    – mk12
    Oct 6, 2009 at 2:33
  • 21
    Just because the question author has accepted an answer doesn't necessarily mean that the discussion is resolved..
    – Nico Burns
    Aug 28, 2012 at 1:02
24

As far as I can tell, substantial part of Qanswers in current formula is problematic:

(log(Qviews)*4) + ((Qanswers * Qscore)/5) + sum(Ascores)

About 1/3 of the answers studied here (83 of total 254) have score less than 1/100 of top voted post in respective question. Given the high amount of views and votes on studied questions, it looks like when sufficient evidence is obtained for the question, time comes to re-check (Qanswers * Qscore) part of the algorithm to make sure that it still reflects the underlying assumption:

one assumes if there are lots of answers, there will be a lot more voting on the answers, too

The observed score difference ("less than 1/100 of top voted post") clearly indicates that not all the answers satisfy above assumption. Algorithm assumes voting on the answers but the evidence strongly indicates readers don't vote on some of these; thus (Qanswers * Qscore) part becomes fake.


Given that questions checked were ones with tens thousands views, insultingly low score indicates that assuming these answers to be popular wouldn't even be in the ballpark. Still, the formula pumps these into Qanswers value, as if it is something everyone would be happy to read (hint: it isn't).

Consider tuning the formula to make it closer match observed voting evidence (when it becomes sufficient to learn from) with the initial assumption of "a lot more voting on the answers".

When question and answers gain a lot of votes, begin ignoring answers with low / non-positive score. Or better yet, ignore answers scored less than some reasonable fraction (eg 1/10) of the top one.

  • For the sake of completeness, another option would be to keep current algorithm and instead modify "feature specification" to better match the formula. Although I honestly can not imagine how apparently useless answers can be explained / specified as contributing to "hotness score" in a meaningful way.

Note there is a feature request at Prog.SE meta to test a less 'klingonic' modification of the formula:


Yet another indication of issues with current formula is how easy it is to manipulate hotness score. In my recent experiments with a particular typical hot question, it looked like changing direction of a single question vote (up->down->up) has been causing changes in score by 10-15 points. This is because any question vote is leveraged by amount of answers. Consolidated, 2-3 voters can "swing" the score by 30-50 (for comparison, current top screen at collider shows me 6 questions scored from 77 to 38). Similarly, adding / removing an answer (any answer) in a highly voted question offers even more possibilities for cheating, because of it being leveraged by question score. If it was a game, I would call this somewhat boring because of poor game balance.

Actually, one can say that sort of unconscious coordinated manipulation already happens quite regularly, when hotness lemmings attracted by titles shown on top of collider land on the questions and begin posting multiple zero-effort braindumps, making formula pump the score even higher, which attracts more lemmings that break things further and so on. "My particular concern is the poisonous effect these mis-answers have on questions, making interesting and well presented problems look the same as non-constructive popularity contests." (quote source).


For the record, in August 2010 algorithm has been tweaked as follows:

  • Succeeding questions from the same site are penalized by increasing amounts. So, the first question from SO in the list gets multiplied by 1.0, the second by 0.98, the third by 0.96, etc)

  • Community wiki questions are penalized, to keep the entire home page from being Poll-type questions

  • The benefit of many answers is capped at 10, and we only look at the score of the top 3 answers

  • We only degrade based on question age, and not the last update date on a question, so questions don't pop back up to the top every time they're edited

  • Views are not counted towards the score

The core of the formula (without the site-based degrading or traffic scaling) is:

   (MIN(AnswerCount, 10) * QScore) / 5 + AnswerScore
   -------------------------------------------------
            MAX(QAgeInHours + 1, 6) ^ 1.4

Yeah "benefit of many answers is capped at 10", how cute.

Even with tweaked formula, stuffing 8 useless, zero-score answers into +50 question would have the same effect as giving 80 upvotes to answers. At +200 question, this would be like giving 320 (over three hundreds!) upvotes to answers.

No wonder than even with above tweak, some questions stick to the top of the hot questions list forever.

13
  • 11
    +1 And it's really frustrating when a good question with a single authoritative answer fails to rank high because it doesn't have enough crap answers to bump it up.
    – Mysticial
    Jan 30, 2013 at 8:43
  • Why don't you make a formal feature request to get rid of Qanswers from the formula completely. I think we have enough evidence already.
    – Mysticial
    Jan 30, 2013 at 8:47
  • 1
    @Mysticial upon discussing these matters, I am not convinced that straightforward throwing of Qanswers altogether is a good idea. At early stages it appears to be doing quite a good magic, "...it's more sensitive to potentially-hot questions at early stage, when there's not yet enough votes to judge better. Frankly, this is the part I really like in current formula..."
    – gnat
    Jan 30, 2013 at 9:29
  • 1
    ...at later stages, totally ignoring Qanswers would be "unfair" to mediocre questions that were made hot by great answers. Don't get me wrong I dislike some things about current formula but have to admit is does some things good (handling of early stage and moderately hot questions looks good to me for example)
    – gnat
    Jan 30, 2013 at 9:31
  • I think that's more of an issue of what defines a hot question. Is a question with a single good answer considered a "hot question"? The only time I've seen questions with few answers become popular are the ones that get the help of reddit. Never have I seen a question with fewer than 5 answers go +500 from just the multicollider - thanks to the equation putting them at a huge disadvantage. (The branch predictor question, with 2 answers, probably could have done it without Reddit. But that's an extreme case.)
    – Mysticial
    Jan 30, 2013 at 9:33
  • @Mysticial an example: single answer from John Carmack made the question really hot (hundreds votes, 100K+ views). If collider wouldn't be able to reflect this, that would be wrong would it?
    – gnat
    Jan 30, 2013 at 9:41
  • 1
    That John Carmack answer was linked in just about every place you could imagine. So it got just about as much external help as it could possibly get. The multicollider equation wouldn't have mattered at all. I do remember it getting up to 600+ on the multicollider with only 3 answers. But that's because of all the votes pouring in from outside.
    – Mysticial
    Jan 30, 2013 at 9:43
  • 2
    In other words, questions with few answers become popular not because of the multicollider, but because they get linked. 3 of my top answers fall into this category. If they get high on the multicollider, it's because of the question getting linked early - as was the case on 2 of my top 5 answers.
    – Mysticial
    Jan 30, 2013 at 9:45
  • @Mysticial I see, interesting. So the idea is to get rid of Qanswers at all, right? How would that work at early stages, when Qviews and Ascores are still low?
    – gnat
    Jan 30, 2013 at 9:52
  • 2
    It won't have an effect. Let the pure sum of votes be the primary factor. If there's only a single good answer it will collect a lot of votes, so let it get on the multicollider. By throwing out Qanswers, bad answers have no effect on the question's hotness.
    – Mysticial
    Jan 30, 2013 at 9:55
  • let us continue this discussion in chat
    – gnat
    Jan 30, 2013 at 9:58
  • Does AnswerScore = sum(Ascores)? Jan 24, 2023 at 20:30
  • 1
    @FranckDernoncourt until amount of answers reaches 10, yes. After that, it changes to sum of 10 top-scored answer scores
    – gnat
    Jan 24, 2023 at 20:36
13

@Matt
You wrote:

The most important factors are the total number of votes on the question itself and whether the question has an accepted answer.

and then this formula.

(votesOnQuestion ^ 2) * (answered ? 2 : 1) / timeSinceLastActivity

This would keep the questions with accepted answer on the top of the list. I think that it should be the opposite of that. When the question has been answered it doesn't need to be discussed anymore and can be archived. If somone has similiar question they will find the answer using search.

12
  1. Every 15 minutes, look at the questions asked in the last 24 hours that have not been promoted to the home page.
  2. Put the question with the most views at the top of the home page.
  3. Put the question with the most votes at the top of the home page.

That spounds good to me, but maybe do it for the top two/three of each cat rather than just one. Six new additions are better than 2 - even if it is every 15 minutes.

I'd definitely like the current setup as a seperate tab however - I like how you don't have to spam your link to everyone to get it on the frontpage (where it has any real chance of being seen by the masses) a la Digg. Perhaps rename it to recent - it's recent activity (questions/comments) rather than new questions.

1
  • Or switch to every five minutes. That way you're getting six additions in 15 minutes, and updates every five minutes. You could also post one question at a time every 2.5 minutes, alternating between votes and views. Oct 12, 2009 at 21:21
9

How about this?

  1. Every 15 minutes, look at the questions asked in the last 24 hours that have not been promoted to the home page.

  2. Put the question with the most views at the top of the home page.

  3. Put the question with the most votes at the top of the home page.

Simple, right? You essentially have a list of the "hot" questions from the last 24 hours. There's no self-fulfilling prophesy. And you could use the same algorithm to select "hot" questions over the last week, the last month, and all time.

7

The first item in the list of variables to work with is "votes", but I assume that's the number of votes that the question has received. Is it possible to count the number of votes it's answers have received?

To avoid the pitfall of the questions that historically got the most views, being the questions that get the most views in the future could you define popularity based on the amount of interaction questions are getting once someone has viewed them.

Ideally you'd want some kind of running average of the votes for the question, the votes for it's answers and the frequency of answers. So that when the a good solution is found and new answers stop coming in and discussion slows down it will drop down the list of popular topics even if it is still getting many page views.

7

The formula needs to control for the average votes per question on a site, as well as the average answers per question.

Instead of (in Atwood's notation) saying Qscore, use Qscore/Ascore (average score) in the formula (and adjust the formula accordingly).

This controls for sites whose vote frequencies vary. The same thing could also apply to answer. This would also even the distribution between questions on sites; it would stop sites that accumulate too many votes from gravitating into the Hot Questions list too often.

1
  • 1
    This is a very important part I think.
    – danijar
    Aug 9, 2013 at 23:29
6

Edit again: Micha(l) was right, I had the "answered" bit the wrong way round.

The most important factors are the total number of votes on the question itself and whether the question has an accepted answer. As @Blorgbeard says, reputation doesn't really affect it.

Age definitely has to be a factor, otherwise a highly voted question could stay around long after the technology it refers to is superceded. Perhaps last activity is all that needs to be factored in here, rather than when the question was asked.

Maybe something like:

((totalVotesOnQuestion + 1) ^ 2) * (answered ? 1 : 10) / timeSinceLastActivity

Where the denominator is scaled appropriately.

Edit: This would rely on people up-modding questions they want the answer to. I don't know if some extra helpful text would help this, prompting people to vote for the question if they, too, want to know the answer. This would tie in nicely with the suggestion on UserVoice to make the votes against a question count as "bounty".

1
  • Hi, what do you mean by (answered ? 1:10) ? What could be the values here?
    – Syed Priom
    Mar 26, 2015 at 5:36
6

Have you looked at the mouseover text on the voting arrows?

"This was helpful"
"This was not helpful"

...you've demonstrated exactly why voting is useful, and it works. :)

Jeff here mentions again how important he believes voting should be where popularity is concerned. But as Inigo once said, "Why you keep using that word? I do not think it means what you think means."

Topics and threads (because that's what the hottest "questions" have become) don't have to be helpful to be popular, so I might suggest that Jeff is as brutally misusing this metric as the guy he's chastising in this quote. To be fair, so are the vast majority of the people on the "hot" questions.

Solution : Have "Hot Discussions" and "Hot Questions" be separate items. Allow high-rep users to take a normal question that is clearly a discussion piece, and make it "Go Wiki" (Can't editors do that already?), and by doing so have it no longer count as a "Hot Question". Now you can apply almost any formula you like to the questions because any open-ended question that gets to the "Hot" board will be ripe to be instantly shoved aside. Almost all the things that you cite in your formula like views, votes, and time since the last update will be more relevant, because spammers, flamers, and trolls will gravitate to the "discussion" questions.

Making it easy to get into flamewars, and easy to segregate them, ala Wikipedia discussion page, has a pretty good track record in returning relevant results, no matter the formula.

5

One additional thought about ranking hot questions. I think the hot questions ranking should potentially be calculated or weighted differently for answered and un-answered questions. It seems that questions that are viewed lots, but not answered are potentially more interesting than questions that are already asked and answered.

5

Have a single popularity field for each question.

When user votes question or one of its answer up, increase popularity by fixed amount. Decrease with down votes.

If user cancels up vote, decrease questions popularity with same fixed amount as up vote would increase it.

Then decrease popularity periodically (once an hour, once a day) with this formula:

popularity = popularity * X, where X is carefully selected and 0 < X < 1, 
                             for example X=0.95

Then just order questions by popularity. The question with highest value in popularity field is the hottest.

This would need some figuring out what the coefficient X should be and how often formula should be applied to questions.

This would give more weight for recent votes, and high popularity spikes would eventually fade away. This way the questions that recently got a lot of up votes would get to the top. This would work even if the question was very old but for some reason got lot of a activity recently.

5

My suggestion, which has been rejected by numerous site frameworks but I continue to make... Store another variable, "Views originating from any list sorted by most views", and don't count those.

5
+1000

I agree with Blorgbeard.. I don't think rep should play a major part in determining a question's popularity..

I certainly don't think it should be based on just the number of views or answers posted (both could obviously be gamed easily).

However, I believe it should be mainly based on score (?) (upvotes - downvotes). This should weight the up/down votes on the question higher than those for the answers, but those for the answers should obviously have an impact..

This would probably be made easier if the question score was automatically partially controlled by the answer content (i.e. 3 upmods on answers = +1 to question).

1
  • 1
    Posted 5 (five!) years ago, this answer presents ideas that read surprisingly relevant now: "I certainly don't think it should be based on just the number of views or answers posted (both could obviously be gamed easily). However, I believe it should be mainly based on score... (upvotes - downvotes)."
    – gnat
    Aug 6, 2013 at 11:08
5

If you'll permit me to get philosophical, what is the purpose of the 'popular' or 'hot' view? I think for a Q&A site it should be to attract people to interesting but unanswered questions, but there are other possibilities. Is it to show off how an interesting question was answered? Is it just to suggest, hey, lots of other people were interested in this question, you may be as well?

I think the score should be based on the total number of votes, number of views, (relatively weak) number of answers, and the time since posting, because each of those is a measure of engagement. So you have something like (A * views) + (B * votes) + (C * number of answers) with C less than A or B. Votes is total number of votes (up or down) on answers plus net positive votes on the question. Then divide by some rapidly increasing age factor, (age of the post ^ D + age of the post ^ E), so that the hot topics are always changing. Sadly all these constants need to be found empirically, but I think this will do it.

5

Maybe tags could be taken into account somewhere along the way. Especially since high reputation people can re-tag other's people questions, avoiding mis-tagged questions.

Just a thought...

3

I don't see anyone doing this. But how about using amount of time spent on reading this question and it's answers? In fact it is possible to get where on the screen the user is and whether or not the window is in focus. Using these parameters to determine whether a quesion (and which of it's answers) is most "spent time on".

And I guess "hotness" can be a function of some of the parameters you mentioned along with the one above.

3

Personally I don't see the benefit of "hot" questions on stackeoverflow. It's probably just bias but I come here for hard answers to specific questions not to lurk or partake in topical discussions.

3

Basically, A user post's a question just to get a proper answer.

As a user, I just want to get a list of related question's suggestion that have been already answered and have been accepted by the OP , better if the answer has few upvotes but that doesn't matter that much because the user who have posted the answer might be an Unsung Hero

Also, As you pointed out the variables to be considred like :

Reputation of asker
Reputation(s) of answerers

Among these two , i think 'Reputation of asker' should be given the least importance as that is far more irrelevant. Consider this: Joshua_Bloch has just created an account on stackoverflow and asked a very good question with a reputation of 1, don't you show his question in suggestion to other's ?

2

I think that for a site like this, the hot question would be a function of what is being discussed the most. Now, if we leave it at that only, then an irrelevant discussion would may also become hot.

In that case, we should take into account the reputation of the people answering (hence contributing to the discussion).

2

Usually, the best way to do these things is to keep it simple so that people can understand and maintain any coefficients in the algorithm. I guess they will require some tweaking as the masses flood the gates.

I am quite surprised how good the Firefox frecency algorithm works. It can't be translated directly to this, but switch some variables around and you have a pretty lightweight formula to order your posts.

2

I understand that using a simple reputation metric in the calculation for which topics should be "Hot" questions would lead to a situation where a new user would have a low chance of having his question make it to the list.

How about calculating a Net Reputation metric based upon the total number of points a user has obtained over the age of his account on Stack Overflow? The age variable itself may be based upon the number of days where a user has logged into the site for at least 10 minutes.

netReputation = (reputationPoints / numDaysUserVisits);

That should allow a user who has more reputation points, and happens to be active on the site to gain a higher metric.

2

What I am suggesting here is not a solution but a general approach.

The problem at hand is generally referred to as the 'ranking problem' in machine learning. Given a set of examples, each of which is a vector of real numbers, induce a function to rank unseen examples. This problem is quite well studied.

What I am suggesting is that the creators of Stack Overflow extract a small subset of their database, manually rank it and put it online for researchers to download and work with (anonymize it if necessary). The UCI Machine Learning Repository would be a good venue. People who work in this area are bound to jump on it, use it to benchmark their algorithms and ideas. Soon you will see publications will real results on a subset of your database. You can get this at almost no cost to you. Many organizations have tried this approach, Netflix to name one.

2

Random question. Maybe unrelated, pardon me in that case.

In our ratings, do we take into account toppers bias. i.e. people tend to support the widely acclaimed opinion. In our case, since the responses are sorted by votes, such a bias would be aggravated. e.g. in a substantial likelihood, this response will be ignored ;)

2

Have you looked into the "Best" algorithm that Reddit added? It's pretty nifty. Here's Randall Munroe explaining it.

1
2

I am starting to think that

  • Votes per view and/or answers per view
  • And to a lesser extent, views per person that saw the question near the top of the given list (this will be different for each tag the question has).

Are good measurement to use.

As it may predict the outcome of highlighting the question, we are after all trying to get good answers and engagement (voting) of users.

If a lot of people that see the title of a question decide to view the questions, it is likely that showing the question title to more people will get more engagement.

I.e the questions that get most benefit from being in the “hot” list should be put in the “hot” list.

2

Help me look at this hot algorithm is correct?! php code

function hot($Qviews, $Qanswers, $Qscore, $Ascores, $date_ask, $date_active)
{
    $Qage = (time() - strtotime(gmdate("Y-m-d H:i:s",strtotime($date_ask)))) / 3600;
    $Qage = round($Qage, 1);
    $Qupdated = (time() - strtotime(gmdate("Y-m-d H:i:s",strtotime($date_active)))) / 3600;
    $Qupdated = round($Qupdated, 1);

    $dividend = (log10($Qviews)*4) + (($Qanswers * $Qscore)/5) + $Ascores;
    $divisor = pow((($Qage + 1) - ($Qage - $Qupdated)/2), 1.5);
    echo $dividend/$divisor . "\n";
}

test result:

hot(1265, 11, 80, 121, '2011-04-01 00:50:02Z', '2011-04-01 04:53:35Z');
hot(85, 4, 0, 6, '2011-04-01 04:05:30Z', '2011-04-01 04:24:02Z');
hot(15, 2, 0, 0, '2011-04-01 05:02:21Z', '2011-04-01 05:14:34Z');
hot(33, 3, 0, 7, '2011-04-01 04:25:43Z', '2011-04-01 05:06:01Z');
hot(12, 0, 0, 0, '2011-04-01 05:00:45Z', '2011-04-01 05:00:45Z');
hot(12, 0, 0, 0, '2011-04-01 05:02:05Z', '2011-04-01 05:02:05Z');
hot(11, 1, 0, 0, '2011-04-01 05:01:04Z', '2011-04-01 05:11:56Z');

5.97048822863
0.308542463568
0.119073201261
0.314512900506
0.10786376822
0.10786376822
0.104757620731

I collected a few data from the SO of the tab hot, but the results on the ranking value is inconsistent with the SO, this makes me very confused

3
  • 2
    This will be very difficult to tell. Why don't you try running it on some real data from the data dump?
    – Pekka
    Apr 1, 2011 at 7:46
  • I collected a few data from the SO of the tab hot, but the results are inconsistent with the SO's hot sort, so to ask where the mistake
    – Charlie
    Apr 1, 2011 at 7:53
  • hmmm, I see. This is a bit difficult, I'm not entirely sure whether it's on-topic here on this site, but it might be. Not sure what to say
    – Pekka
    Apr 1, 2011 at 7:58
1

Are you planning to make any exceptions in your alogorithm for meta-questions? You joked around in one podcast about building a system that is being used to discuss the system, which of course isn't really what's happening when you count up the actual number of meta-questions, but when you look at the really popular questions it seems that way.

The general line of thinking is that once the site goes public, these posts will eventually fall by the wayside, but I'm not so sure that's true. There will always be interest in discussing how the site works, especially among your target audience.

Maybe just a slight subtraction for tag=stackoverflow after the beta is done. This would, of course, require some diligence with re-tagging.

1

If it were me, I'd say weighted activity per minimum period through a linear filter: http://en.wikipedia.org/wiki/Linear_filter

1

I think that you have to factor churn rate as this is an issue for most here - fantastic resource heavliy used IMHO.

So I would go for

(number views/number of answers)     *     (dateTimeQuestions/dateTimeLastAnswer)  

and then a simple ranking of those.

View v Answers is a churn rate to answer rate (should be good as there have been lots viewing it and lots answering it) then add the Time Factor of the Question, that is, the speed of the question to the speed of the answers.

Note that this is simple problem solving algorthim not a real one.

UPDATE

Having thought about this for more than a nanosecond there may be more of an issue.

What is definition of hot topic? Views to Answers or Time from Question to Number of Answers.

I think what I was trying to do was combine these to perspectives on a hot topic then provide a quick rank of the multiplication of those two routines...

UPDATE 2

Having just read this again - I still haven't put reputation and voting in...
On reputation (this is from people that have put time in) my mind thinks that a person coming with a rep of 10 may still provide the "best answer" so discount reputation out of the "hot topic"...

So the last piece would be to

(  (number views/number of answers)     *     (dateTimeQuestions/dateTimeLastAnswer) ) * (number of up votes/number of downvotes )

Humm b*tch of a question = good luck =>

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .