41

Another meta question identified a bug that was marked as designed. To summarize the bug, the little number that appears next the Review link in the header shows an incorrect value most of the time.

Wrong posts number on SO Review section

I would like to propose that the feature be dropped. Honestly, I would prefer it be fixed, but the response to the question above gives me little faith that it will be. Half of the time, I click that little brown box and find out that I was lied to. There's nothing to review.

The little number no longer does the job it was created to do.

Oh sweet! Three things I can review!

Just kidding...

  • 1
    I wouldn't get my hopes up on SE fixing this: meta.history.stackexchange.com/a/916/739 – yannis Aug 22 '14 at 21:58
  • 1
    So far every meta post I can find has been a bug report. We know by now that it's a bug and that it was designed that way. This is a request to change that design. If it gets enough support, maybe they will remove it. – Rainbolt Aug 22 '14 at 22:00
  • 1
    I would argue that it does what it intends to do: make people go to the /review page. I think this is more intended to be a "hey, go review!" indicator vs. an accurate count of the number of review tasks. – hichris123 Aug 22 '14 at 22:09
  • 5
    @hichris123 No, it learns people to build a resistance and ignore the number. – yo' Aug 23 '14 at 22:07
  • Worth noting, btw, that the new top bar on SO does not display the number on the bar itself. Although, there is this, which may or may not be partially related to said change. – Jason C May 30 '17 at 18:13
7

Well, here it is, three years later... Three years of complaints, bug reports, stalled discussions and abortive attempts to make that number more accurate. And I have to say...

...this is done!

Which is to say, we lost the review count. It is no more. No matter how we approached it, calculating an accurate count of reviews - or even just an accurate timestamp for the newest task - for the current user on every page load was simply not something that could happen without a complete redesign of review.

Upon that sad realization, we revisited the original rationale for having a counter for review:

I was wondering if there wasn't something that could be done to help direct everyone possible to review (or at least be aware of it - there was a comment at one point of a 10k user who wasn't aware of anything in the /review link after clicking it as a new user and not seeing anything in it that could be done by him)

Note that the specific suggestion there was never implemented (too expensive) - we punted and threw the total number of pending reviews up for users with 10K+ rep, thus fulfilling the prophecy contained in that same request:

This should be something that you can get to 0 every day. Making it something that is an unmanageable number makes it something people will ignore, and that's not what this is suggesting.

...as well as generating endless confusion as to what the number even meant.

But there was a small side benefit to it, annoying though it was: if you were the only active reviewer on a site, you couldn't ignore the fact that your peers weren't pulling their weight - if no one else was reviewing, the number would just keep going up.

These were the base requirements then:

  1. Notify people that their assistance was needed in review
  2. Don't display a number that doesn't correspond to anything the reviewer can actually do
  3. Indicate when the site is unhealthy due to a lack of active reviewers

Now... There are some previous use-cases that aren't in those requirements. For example: if you happen to be a moderator and the only reviewer on a small site, the number worked perfectly for you and its replacement is not going to work as well. This was never really a goal for review though.

So with those requirements in mind, we replaced the number with a simple indicator, triggered based on one or more queues exceeding a per-queue (and per-site) threshold. When chosen correctly, those thresholds should represent a level at which there are an insufficient number of active reviewers - thus fulfilling requirements #1 and #3. The thresholds should also be sufficiently high that discrepancies between the cached task counts and what the current reviewer can actually review don't matter - thus fulfilling requirement #2.

Needless to say, this isn't the conceptually-elegant, trivially-scalable system we wanted: it's likely going to take a lot of tweaking of those thresholds to make it work effectively. But it has a distinct advantage over all those other ideas we've been batting around for the past 3 years: it's actually been implemented and hasn't brought any site to a grinding halt.

26

The number indeed serves little purpose now. I suggest highlighting the link when reviews are waiting in one or more queues:

Example

  • ... having a coloring like that would be a) barely recognizable, and b) what would its purpose be? Maybe on smaller sites, sure, but any graduated site... – hichris123 Aug 22 '14 at 22:11
  • @hichris123 so maybe background color instead of text to make it more visible. Purpose is draw attention to the link without showing a wrong number. – Shadow The Curly Braced Wizard Aug 22 '14 at 22:14
  • 1
    I like this answer because it saves what functionality is left of the current feature without providing inaccurate information. Plus, it even reduces clutter. This is a win-win in my opinion. – Rainbolt Aug 22 '14 at 22:28
  • 10
    @hichris123 We can make it blink! /I'll show myself out. – yannis Aug 22 '14 at 22:33
  • @Yannis plant a bomb is Stack Exchange headquarters, that would be more elegant way to close the sites down. :-D – Shadow The Curly Braced Wizard Aug 22 '14 at 22:36
  • Yes please! And bring back the suggested edit indicator for us >10k folks. – ɥʇǝS Aug 29 '14 at 19:18
22

I really think it is important to have that number broadly accessible to the most trusted users on the site. If it's hanging there for too long, there is a problem.

That said, I agree that there are issues with how it's displayed right now. I recommend the following:

  1. Maintain a "last viewed" timestamp per reviewer, updating it upon visiting /review.
  2. Maintain a "last review task created" timestamp per site, updating it when the queues are synchronized (roughly every 5 minutes) when a new review task is created.
  3. If #1 is < #2, display as today (for 10K users). If #1 is > #2, don't highlight the number:

This would then continue to allow you to keep tabs on the state of the queue, while removing the need to keep clicking through to see if you'd missed something. It would not necessarily guarantee that you'd only visit review when there was something for you to do - caching, other reviewers, and various eligibility requirements could conspire to still highlight the number when there is nothing to review - however, this should happen less frequently and less persistently.

Not sure yet if this is feasible, but... I'll find out.

Related:

  • 3
    I should start charging for architecture consultancy… “If it's hanging there for too long, there is a problem.” No, not necessarily. It's a problem if the same reviews are getting no attention for a long time. It's not a problem if they're new reviews each time. – Gilles Aug 25 '14 at 12:31
  • The indicator is still useless. Why not actually make it useful? Why do I need to know there are 478 things needing review vs 477? Show me something that I can actually take action on, like suggested edits. I know there will always be stuff to review when I have time to review it. What's the point of an indicator if it only tells me stuff I already know? – ɥʇǝS Aug 29 '14 at 19:20
  • 1
    What's the status on this? I still see a false number on PuzzlingSE almost all the time. – CodeNewbie Jul 13 '15 at 11:35
  • 1
    This keeps confusing people. What's your timeline for fixing this? 6–8 years or 6–8 decades? – Gilles Dec 13 '16 at 23:22
  • Well, top bar's getting redesigned again so... Maybe after that – Shog9 Dec 14 '16 at 0:17
  • And again. Yay, top bar redesign. Or boo, top bar redesign. Are you actually going to improve it this time? Finally fix the regressions introduced by the previous change, at least? (E.g. name, votes went missing) – Gilles Dec 15 '16 at 11:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .