What stats should I track in an HNQ list monitor?

Question

This is a question for the community at large.

I've been fascinated by the HNQ list for a while now and this question has pushed me over the edge to create an HNQ list monitoring and logging tool (for purely academic purposes).

Currently for each question on the list at each sample I'm planning on capturing:

• The obvious stuff:
  • Question ID, site, etc.
  • The official hotness value (it's reported in the tool tips on the master list).
  • First and last time seen on the list (implicit in the recording of sample timestamps).
• The parameters of the core formula:
  • Question Score
  • Answer Count
  • Total Answer Score
  • Age in Hours (/ Creation Date)

Additionally I'd also like to capture some extra data for the purpose of running simulations of alternate algorithms (with limits, of course, since I'm only sampling questions that actually are on the HNQ list). Right now I'm thinking:

• Up/down vote count per question.
• Individual up/down vote count per answer.
• View count.
• Some suggestions from chat:
  • Number of close votes.
  • Close vote times and question closure time (if it ends up closed) (this does require me to track questions for some amount of time after they leave the list).
  • Protection status.
  • Question comment count.
  • Per-answer comment count.

All of the above are captured per sample to allow an easy look at changes over time. We also have the benefit of some additional data being recreatable through SEDE queries if needed, which isn't ideal but can fill in gaps if certain things are missed that we want to see later.

My question for everybody here is: What else should I capture? I'm taking requests. What might we care about for simulating alternate score algorithms? What might we care about just because we're curious? All suggestions welcome.

Answer 1

These are all great suggestions for properties of the hot questions themselves. Can we also capture useful information about the sites from which the hot questions come? I have two suggestions:

• Total number of questions asked "recently" and how many answers they have. I'm not sure what counts as recent, but question age is a factor in hotness and my goal here is to understand what proportion of a site's questions are coming to HNQ and how close the "near misses" are. (This tells you whether small changes in the algorithm, including throttling, are likely to produce significant effects on that site.)

• Disposition of previously-hot questions. Did they (or their answers) get closed, protected, downvoted (unlikely), or deleted? This might tell us whether something intrinsic to the question caused it to lose hotness or if it just aged out.

Answer 2

Related to my own feature request Encourage active users to edit Hot Questions, especially titles, I think it'd be good to record editing stats: number and extent of edits relative to other events. E.g.:

• do questions get some touch up after their first answer comes in?
• do mods delete a bunch of answers, add protection, and then rewrite the question?
• do edit wars ensue over titles?

And so on.

Answer 3

To a certain extent who is asking those questions/answers might be useful. There have been a few cases where specific users ask ... somewhat contrived questions to game the HNQ systems, and I for one would love to know what proportion of the HNQ lottery I or other users are winning ;)

For similar reasons, being able to filter per site would be nice, though it is something that's probably should be core functionality.