# I'm not getting reputation notifications

I haven't been getting reputation notifications since recently.

(I reloaded several times, closed and reopened my browser, and restarted my computer)

New rep is still shown in the profile page Reputation tab:

• I can reproduce: i.stack.imgur.com/XUy39.jpg Jul 29, 2021 at 4:36
• Hah! I'm not getting notifications for the upvotes to this question :) Jul 29, 2021 at 4:36
• Could have something to do with today's maintenance Jul 29, 2021 at 4:48
• Heck, I'm not getting any... I have some followed questions and those notifications aren't coming in, either.
– Catija StaffMod
Jul 29, 2021 at 5:01
• @Catija I'm getting comment notifications and chat pings, at least. And I don't have enough followed posts to check 'em. Jul 29, 2021 at 5:02
• Yeah, I got the one for here but there's definitely others that aren't going out.
– Catija StaffMod
Jul 29, 2021 at 5:03
• I'm not getting followed post notifications or reputation notifications either. Jul 29, 2021 at 5:38
• same here ...... Jul 29, 2021 at 6:09
– Glorfindel Mod
Jul 29, 2021 at 6:42
• On mobile I did see my updated rep bumber up top Jul 29, 2021 at 7:18
• Regression of No notifications for followed posts? Jul 29, 2021 at 8:04
• I'm only getting reputation notifications for a Teams post, nothing for the main sites.
– Tinkeringbell Mod
Jul 29, 2021 at 8:52
• Same here. No notifications. But I can see the reputation change in my Profile under Reputation Tab.
– Ram
Jul 29, 2021 at 9:12
• I noticed it because I got notified of a “Nice Answer” badge, but got no notification of an upvote. Jul 29, 2021 at 9:41
• I'm getting some rep notifications now, but it hasn't backfilled fully yet. At least I assume that's what happening - I'm getting little bursts of rep notifications when I know I've repcapped overnight with this Q. Jul 29, 2021 at 12:36

Couple of things blew up here and it took a little while to isolate and repair the fallout, so sorry for the delay.

We had a (successful!) SQL failover in the wee hours of this morning but, for the brief period of time that the application was failing over, a number of duplicate events were written to a service that we call the "aggregator". This service is responsible for taking events from the various sites across the Stack Exchange network and aggregating them to denormalized tables so that we can get performant queries for things that cross the whole network - things like global inbox messages, rep, achievements, etc.

Usually this would be a non-issue - we have de-duplication logic in those code paths and it has functioned just fine for years. However, there were a specific set of events that had, clearly, never been de-duped - they're relatively rare (happen once per user per day). De-duping takes the form of sorting and eliminating the duplicates using equality checks - unfortunately the code for checking equality in one of these events looked like this:

public bool Equals(object obj)
{
if (!EqualityBase(obj, out VisitEvent other)) return false;

return other.Equals(this) && other.UserHistoryId == UserHistoryId;
}


That call to other.Equals(this) was very rarely hit, but it clearly recurses infinitely, leading to a StackOverflowException. That exception kills the process and because of the way the aggregator service runs, it meant that whichever server picked up the work would then die off when it tried to pick up the work.

We have some poison message handling around this but it didn't kick in effectively here and we have a large backlog of events to get through now the bug is fixed. It's getting through them now though and we should be all set in the next hour or so.

UPDATE Annnnnd it’s done. Thanks for your patience folks!

• So Stack Overflow's rep problem was due to a Stack Overflow? (Someone had to say it :D) Jul 29, 2021 at 12:41
• @bobble Frankly I'm only glad the site was up enough for me to find the right procdump incantation to capture a dump of a StackOverflowException :) Jul 29, 2021 at 12:42
• In TeX it would be \def\recurse{\recurse}\recurse ;-) Good job! Jul 29, 2021 at 13:15
• Reminds me of this... Jul 29, 2021 at 13:32
• @egreg Hate to be a nitpicker, but this infinite loop will not lead to a stack overflow, as TeX’s macro expansion routines handle tail recursion gracefully. You’d need something like \def\recurse{\recurse x}\recurse. Jul 29, 2021 at 13:37
• @EmilJeřábek Indeed, it does even worse than causing an overflow! ;-) Jul 29, 2021 at 14:01
• Hey Dean, its still not 100% up to speed for me. Jul 29, 2021 at 15:57

As Dean estimated things would be back to normal by now, I have to say things are not lining up for me yet.

See this screenshot for example:

My achievements box shows 130 rep for the day, while my profile shows that I'm rep-capped at 200.

• Same here - my inbox only shows 172 meta rep for today. Jul 29, 2021 at 17:23
• It’s quite possible that those were dropped on the floor; I’m out for a few days now but somebody else on the team will continue digging here to see what’s up Jul 29, 2021 at 17:30
• Enjoy your time @DeanWard.l! Jul 29, 2021 at 17:37
• Today it appears to work fine. However there is still rep dropped from yesterday in the top. Jul 30, 2021 at 5:10