Modulate the review notification's color to reflect changes, or make it dismissible

Question

Following another feature request that was completed yesterday (Notify users of possible reviews on toolbar), users of several sites have noticed a persistent notification in the top bar for flags that need reviewing. Previously this notification pertained only to suggested edits, but it now displays the full count of flags of all kinds (except close votes on Stack Overflow) that are eligible for review whenever this number ≥ 10. This includes flags that I cannot act on personally. I.e., the number reflects the overall community's review queue, not my personal queue, and will continue to display a number even after I have completed the maximum number of reviews for a day, or reviewed all flags that need reviewing. This number is effectively the rest of the community's job to manage after I've reviewed everything I can for the day.

As I understand it, the idea behind showing me this notification anyway is to motivate continued action beyond personal completion of reviews. Arguably, I should raise heck on the per-site meta about all the social loafing going on.^* If I understand correctly, that little orange box is supposed to provoke me to first perform as many reviews as I can, and then at least go upvote the existing meta-rant (assuming someone beat me to it, as is usually the case on high-traffic sites) about the collective need to kill the review queue with fire (the preferred method, AFAIK).

If this works as I think the SE folks intend it to, hopefully that little notification will go away quickly on all healthy sites, or at least stay very low. Still, while it's there, it's a bit of a misfit to the extent it notifies me of things I can't do anything about. Assume for the sake of argument that I've already reviewed all I can, and done my part on meta to mobilize a little collective cleanup effort, and waited patiently to see some results (TBH, only the first of these three is true in my case on Cognitive Sciences). What more can I do about this notification besides trying to pray it away? As the original feature request argued:

^{This should be something that you can get to 0 every day. Making it something that is an unmanageable number makes it something people will ignore, and that's not what this is suggesting.}

However, I note Shog9's comment on Anna Lear's answer to the original feature request:

^{This is everything in review, even stuff that you, personally, can't review. Super-expensive to personalize this - so the alternative is simply not having an indicator. Think of it as... A reminder to encourage your peers to review.}

Frankly, this "reminder to encourage your peers" is not subtle (nor clear). Hence I worry about what ends my more enthusiastic and notification-sensitive peers might go to in encouraging the rest of my peers, who I suppose have created the problem by not minding the review queue already, despite plenty of badge-based encouragement. For those of us in the middle who pay attention and do our part, but aren't the first to sound the alarm and don't care to constantly hear others sounding it, might there be a better way to tweak this notification? Hopefully something programmatically simpler than personalizing the count displayed (which Shog9's comment suggests is unfeasible), but something that still provides some presently useful information in the notification, or no notification at all.

I recognize the current workaround is to remember the flag number when I last checked the review queue, and check it again when it goes up, or do a little celebratory jig when it goes down. This isn't terrible, but I'd prefer to keep that space in my working memory free for other mental operations. I'd even prefer to have a flashier read on the pulse of the community than this personally, but I suspect others would prefer to have no notification there at all (after all, this is what we've been accustomed to until now). Therefore I propose two general options that need not be mutually exclusive:

1. Modulate the notification's color based on its direction of recent change, if any. E.g.,

   • If the count has gone down in the past half-hour, make it yellow-orange, or more transparent.
   • If it's gone up in the same time frame, make it red-orange, or give it a white border.
   • If it's been steady during whatever time frame we like best, leave it as is, depicted here:

   
   ^{[This classic freehand circle was brought to you by Shog9]}

   A simpler alternative would be to display the white number in an orange bordered box without the orange background fill. The fill could kick back in if a new flag is raised. If the flag count drops, only the white number on the black background would need to change inside the orange border.

   I don't want to ask too much of the programmers when it's already been stated that personalizing this would be difficult, but might it be more feasible to just make the color, fill, or transparency personalized to reflect the state of a binary variable rather than a count of flags to review? This variable would instead be "A review task is/not available for you presently". If a task is available for me personally, light that mothasucka up! If not, let me become comfortably numb.

2. Dismiss the notification after it's been clicked:

   • until its number changes,
   • until I log in next,
   • or at least until tomorrow.

   This borrows an idea from Richard Tingle's feature request (In 10k mod notifications show notifications only for unacted on flags), which seems to have languished in obscurity since December '13. Hopefully this bit of coding would be no more difficult than it was to dismiss new achievement notifications. If the first idea is implemented, notification dismissal could also mean returning the notification to its default no-recent-change state. This would provide that subtle visual hint that the notification has been managed (and is safe to ignore if one is so inclined), even if it doesn't go away completely (and you don't want it to).

An advantage of option 1 would be a slight improvement in the intuitiveness of the notification. Most notifications I have are personally relevant, not about the community as a whole. If this notification changes in some way that appears good without my intervention, or only lights up in a way that appears to demand corrective attention when there might be something new I can do about it, it should be easier for me to intuit that this notification is slightly different from the others. Maybe this will help control the spreading bug infestation (see 1, [2], [3], [4], and probably more to come over time)...And of course, option 2 ought to help in this regard too, at least by giving people less to worry about. I appreciate that most of us should probably worry more about reviewing, and if they do, this problem will diminish...but again, you also don't want to goad the more spastic among us into worrying even more about it than they already do with an indelible notification.

Thus IMO, a little extra tweaking of this design could go a long way. To that end, I invite comments on how to tweak this proposal, especially including:

• Choices of hue, transparency value, or border and fill options
• Optimization of the time window in which to detect changes in flag count
• Options for when and how to dismiss the notification, whether partially or completely

Of course, a third option exists and has just been requested. If showing me the community's total flag count doesn't accomplish the net good it's supposed to in the end, we could just roll back the change. I.e., "Bring back the suggested edits notifications to the top bar . . . again"...Useful comments there also suggest lowering the threshold for a notification below 10 so it regains the functionality of the old suggested edits notification: immediate notification of new flags on sites where queues are ordinarily empty. That would be nice on Cross Validated, where the review queue clears notoriously, competitively fast. It would also make the notification even more omnipresent, and tweaking it even more helpful, if it can be done right.

Edit: I see that the old-school suggested edits notification still appears for users without the mod tools privilege unlocked:

This eases my concern about confusing new users, but gives me yet another idea. Could this previously implemented notification remain active at all privilege levels and take priority over the new community-wide review notification? If a suggested edit (or whatever else you can notify us about on a personal basis) is available to review, produce this notification. Otherwise produce the other notification and apply different appearance settings to it, such as any combination of those I proposed originally.

---

^{Footnote
*Hopefully in doing so I avoid the temptation to make an overly big deal about how annoying the little orange box is for me personally, or how holier-than-thou I am for doing my moral duty as a reviewer amidst the abundantly evident decay of personal responsibility and family values, [insert other foolish rant options here], etc.}

Answer 1

Dismissing the notifications when clicking on them, i.e. resetting the notification counter to zero, solves the problem of irrelevant notifications (false positives): The notifications will only show up again when there new items to review. Assuming that I always click on the notification when I start reviewing, I will certainly not have reviewed the items that I am notified of.

Obviously this comes at the price of more false negatives, i.e. there are things I still could review, but I am not notified. However this only happens when I stopped reviewing before having seen everything. IMHO this is not a big issue because I can always remember myself (e.g. by leaving the review tab open) that there are still things I wanted to review. I don't need the system to remember this for me.

And the best thing about this: It is easy to implement!

---

Implementation proposal for dismissable review notifications

For every item in the review queues, store the timestamp of when the item entered the queue.

Maintain a histogram of open review items by entry time, which is updated by events: When an item enters the review queue, increment the counter for the corresponding bin (which is e.g. one minute wide). When an item leaves the review queue, decrease the corresponding counter.

For every user, store the timestamp of when she last clicked on the notification. Determine which bin this timestamp corresponds to. To compute the number shown in the notification, sum up the counters starting with the next bin all the way to the latest bin.

Given that most review items are handled within four days, it should be enough to cover that timespan. With bins which are one minute wide, this would only add up to approx. 6k bins. If summing up these bins is a concern, you could also keep a second histogram, e.g. with one hour bins, so that you need to sum up at most in the order of 59 + 4*24 + 59 bins.

---

With this implementation, there are a few review items that I would not be notified of: The ones which are created after I clicked on the notification and before the next minute starts. This is probably not a big deal because I am likely to spend more than one minute in review and will have seen these items anyway.

Answer 2

Address with the October 2017 rollout of the new top bar review indicators: pending task counts replaced with colored indicators for queues most in need of attention.