This has been fixed. All the duplicate badges have been deleted. If you are still receiving extra Steward badges at this point, it is because you earned them.
This was caused by a temporary issue related to the rollout of Visual design changes to the review queues.
As part of that project, we are changing steward badges from being awarded once per queue at 1000 to be awarded per queue every 1000, so users with 5000 reviews would get 5 badges.
The problem we ran into is that badges are expressed as unique tuples of
(UserId, BadgeId, ReasonId).
ReasonId is an abstract object which could represent a post ID, a review queue, or really anything. In our case, it was simply the review queue ID, making it possible to award the steward badge exactly once per queue.
I originally had the idea of having the badge be awarded as
ReasonId = (count(*)/1000 - 1) * 1000 + ReviewQueueId, so your first badge at 1000 would be
ReviewQueueId, your second at 2000 would be
1000 + ReviewQueueId and so on, but it was rightfully pointed out that this would be confusing.
Instead, I decided to award badges with
ReasonId = (count(*)/1000) * 1000 + ReviewQueueId, so your first badge is
1000 + ReviewQueueId, your second is
2000 + ReviewQueueId, and so on. This required a migration to increment all of the ids by 1000. Both solutions required that any code that got the review queue from the badge would have to do
ReviewQueueId = ReasonId % 1000
During the time between when the migration ran and when the code deployed, I expected two things to happen:
- We'd experience a brief window where code that read the review queue id from the reason ID would fail. These all had fallbacks, like the "for 1000 reviews, over 200 actioned in the old review system" message you saw, so I wasn't terribly concerned.
- If the SQL job ran, some users might get the Steward badge awarded with the low reason ID. I figured this would be rare and easily remedied with manual deletions.
Unfortunately, I was way off base with #2. The code ran on at least 30 sites and re-awarded all the steward badges there.
The fix here will be to run a query deleting all Steward badges with
ReasonId < 1000 as I've confirmed that they are all erroneous.