30

In the new top-bar I see from time to time the indicator kind of resetting itself. It does not know anymore that I already "confirmed"/"noticed" the new rep.

Then I see something like this for instance:

rep indicator

or smaller numbers. But I am sure I did not gain any or so much rep since the last time I clicked on that number.

This happens whenever I delete a post of mine.

BTW my internet connection is excellent and the websocket connection is absolutely stable.

  • 10
    +1 Confirming, I've seen it happen once so far. – Stijn Nov 22 '13 at 15:44
  • 2
    Happens to me often. A few times today. – juergen d Nov 22 '13 at 15:44
  • 4
    We've got a number of these edge case bugs, and the dev team has decided to do a minor re-factor of the JavaScript which should take care of them. – Jeremy T Nov 22 '13 at 16:41
  • @JeremyTunnell: If it helps: It seems to be resetting when I lose rep from deleting. Whenever I delete an upvoted answer of mine it gets resetted. – juergen d Nov 25 '13 at 16:15
  • 5
    It is getting quite annoying. – qwertynl Nov 25 '13 at 16:24
  • Maybe related, but it happened when I earned a badge today. The "+X" indicator came up. When I checked what the new rep was, it was just a badge notification. The number reflected my total rep gain for the day. – Geobits Nov 25 '13 at 17:22
  • per my reading of comments from SE developer, it was intended to show only increments compared to last time you checked: "The big green number is how much rep you've gained since the last time you checked." – gnat Nov 29 '13 at 11:36
  • @gnat: What has that to do with my question? – juergen d Nov 29 '13 at 11:43
  • @juergend if you see 430 once, next time indicator blinks this value is expected to be excluded from the number. Whatever is shown when you delete your answer, should somehow follow similar reasoning, and not reset your rep gains to the start of the day – gnat Nov 29 '13 at 11:49
  • 1
    This also happened to me, but with comment notification! (I had a comment notification on SO, clicked it, read the comment. Then I went to MetaSO, and after a while the notification popped out again.) – BartoszKP Dec 11 '13 at 11:27
23
+50

Tracking this down was driving me absolutely insane, but a fix is rolling out now.

Here's the fix:

//GetCache().Set(key, DateTime.UtcNow, OneMonthInSeconds);
GetCache().Set(key, DateTime.UtcNow, OneMonthInSeconds, broadcastRefresh: true);

I'll try and explain what was happening since this took so long to get a fix out for. The cause didn't occur to me until reading the code for the 200th time just now. Here's what happens when you open the achievements dropdown:

  • Your browser hits /topbar/achievements
  • That route sets 3 values:
    • Integer: Unread Rep to 0
    • Integer: Unread Badges to 0
    • DateTime?: Your last read date, now: DateTime.UtcNow

Here's what happens when your rep is recalculated (pretty much this only happens when a post you possibly had any stake in is deleted or undeleted):

  • Your reputation is recalculated on the site that triggered a recalc
  • Your RepHistory is updated on that site
  • An aggregator event is added to a redis queue for a network recalc to happen
  • An aggregator instance (whichever web server currently won the election) processes that item
  • Your NetworkRepHistory (aggregate from all sites) is synced up by doing a diff with the site that triggered the recalc
  • Your "last read" date is fetched from cache
  • A query is run to total up your actual network rep change between that date and now
  • We update your total unread rep and send it realtime

Ok, so here's where that went south. In our caching layer, integer gets, sets, and increments are atomic and go directly to redis. The DateTime? fetch does not, it goes through L1 cache which is HTTP Runtime cache on that web server. That means you hit one web server & app pool to set your "last read" date on achievements while the process that was fetching your total after a recalc was hitting another, almost guaranteed to be out of date (up to a month, ouch).

I pushed the change above (broadcastRefresh: true) which invokes a cache clearing mechanism we have for these edge cases that are very apparent to the user when cache decay occurs. It uses the redis pub/sub mechanisms to inform other applications that they need to clear their L1 cache of that key. This means when the aggregator runs it'll get a fresh brand-spankin-new date for the total query that is, most importantly, accurate.

I really am sorry this bug lingered so long, it was subtle and not apparent to a few of us trying to figure this issue out. Still, it was my bug from the first time that code was written.

  • 1
    Wow, you're fixing a lot of things today. – hichris123 Dec 14 '13 at 3:02
  • Well played sir ! – ʞunɥdɐpɐɥd Dec 14 '13 at 4:09
  • 1
    quite a pity that this is perfectly on-topic here at MSO. Otherwise, I'd request its migration to Programmers.SE - stuff like this terrific analysis is much wanted there! – gnat Dec 15 '13 at 11:39
  • If you ever plan to write a biography, this would make a perfect part of it! :-) – Shadow The Princess Wizard Dec 15 '13 at 12:21
  • Nick, could you please take a look here? Oded mentioned there is a chance that spurious "resets" in user profile may be related to this change – gnat Jan 17 '14 at 7:12
  • 2
    @gnat this change was for the indicator, and does nothing to affect the actual rep...this is at the bold steps above, the reset bug (somehow) is happening at step 1. A user's rep is loading as 0 then committing in Linq2SQL (I think), but I have yet to track it down. – Nick Craver Jan 17 '14 at 13:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .