What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 129 Stack Exchange communities.

Tag scores update is supposed to happen at UTC 3:00 everyday. Tag scores are updated by a scheduled job that is time consuming and often times out (see for example the answers for Tag info not updated regularly on profile? and Tag scores have not updated; script did not run).

In the past month or two, the issue with not updating the scores has gotten much worse. Tag scores are now regularly not updated for 2-3 days in a row. The job fails more often than it runs which obviously lowers usability and appeal of tag badges (which are not awarded when the script fails) and scores to some extent.

I can also guess that if a job fails on consecutive days, it only makes it harder to make it work as it has more and more tags and answers to recount.

This issue happened in the past but not to the extend it's happening now.

Please do not close it as duplicate. I'm creating a separate post as I've been asked to do it by Anna Lear in a comment for the answer for the first of the linked questions. Other posts reporting this issue have accepted answers which makes them difficult to track for the SE team.

UPDATE

I've been writing down whether the scores were updated or not every day in April.

The results are:

Day      1   2   3   4   5   6   7   8   9   10  11  12  13  14  15  
Updated  -   -   -   ✓   -   ✓  -   -   -   -   -   ✓   -   -   - 

Day      16  17  18  19  20  21  22  23  24  25  26  27  28  29  30
Updated  ✓   -   ✓   -   ✓   -   -   ✓  ✓   -   ✓  -   ✓   -   -

The update succeeded on 10 days, failed on 20. The situation got somewhat better in the second half of the month. Has something been done about it?

share|improve this question
    
Question is, how important are the tag scores? Won't it be better to ditch the whole thing, now that the user profile is going under a major change anyway? –  Shadow Wizard Mar 25 at 9:29
19  
@ShadowWizard For me, it is important - it's like detailed reputation for specific areas of knowledge. Also, see how Careers website features your score in tags. –  Szymon Mar 25 at 9:31
1  
I'll keep track of how the tag count update goes in April and update the question with the results at the end of the month. So far it failed on the first 3 days. –  Szymon Apr 3 at 3:48
5  
My tags score has not been updated for more than 4 days: stackoverflow.com/users/114029/leniel-macaferi?tab=tags –  Leniel Macaferi Apr 3 at 16:10
    
It has been like that for a couple of days now again on Stack Overflow. Reading that this issue comes up every 2-3 months at least, why is it not fixed for "once and all"? OK, bugs can always happen, but it is a bit too frequent IMHO. –  lpapp Apr 9 at 5:48
3  
Yes, so far in April it worked on 2 days only: 4th and 6th of April. –  Szymon Apr 9 at 5:50
1  
Same problem for top-users pages. –  false Apr 10 at 18:05
1  
@ShadowWizard: Also top-users pages are not updated for maybe a week or so. –  false Apr 10 at 18:20
1  
@ShadowWizard I'm pretty sure I've earned the gold matlab badge, but my tag scores aren't recomputing to give me the badge. It's very irritating. Are the tag badges not relying on the tag scores to be updated? –  chappjc Apr 11 at 20:08
1  
@chappjc Tag badges are based on tag scores so they are not given when the tag score update script fails. –  Szymon Apr 11 at 21:10
1  
@Szymon That's my point. My comment was a rhetorical question. Shadow Wizard questioned the importance of tag scores altogether and my response is, "I want my gold badge, pretty please update the tag scores." –  chappjc Apr 11 at 21:15
3  
@chappjc and Szymon guess you're right, didn't think of the badges. Guess the problem has bigger impact than I thought then. –  Shadow Wizard Apr 11 at 21:43
1  
@ShadowWizard Just got my badge. :D I guess someone ran the script manually... we'll see if it goes every day now. –  chappjc Apr 12 at 3:36
    
@chappjc the problem isn't with the script being automatic, it's failing due to heavy load on the database. So even when run manually it has same chances of failure/success. Glad it worked this time! :) –  Shadow Wizard Apr 12 at 9:51
2  
Wait... maybe it's forming a secret morse code that send us some message? :D –  Shadow Wizard Apr 30 at 9:31

1 Answer 1

Now the Review-button is extremely slow, if it responds at all. Think you have reached the max of MS technology.


You are some sports. Downvoting without explanation. The new style? The SO style? I am able to delete this answer you know. But please let me know why it is so bad in the first place. Going from +3 to -3 when it got your attention.


I really cannot see the problem. Or the arguments.

  • The job fails more often than it runs which obviously lowers usability and appeal of tag badges and counts.

How? Tag badges is given for 100, 400 and 1000 upvotes. It is very unlikely that anyone "misses" a lot of tags just because the scheduled job (successfully) runs not every day, but say every 2th or 3rd day. Worse case scenario is that some people receive a badge with a small delay - small compared to the time they have spent gaining it. And it is even more unlikely that people should loose interest of tag badges just because they have to wait 2 days before their calculated tag count increases from 99 to 100. What "usability" should be compromised?

  • For me, it is important - it's like detailed reputation for specific areas of knowledge. Also, see how Careers website features your score in tags

Again. Yes, it is important - a nice feature. But is that feature gone? You still have "detailed reputation for specific areas of knowledge" havent you? Unless you are learning a completely new skill every 2nd or 3rd day, and receives an overwhelming number of upvotes for those new skills each time, the calculated tag counts is more or less the same after 2 or 3 days. And really doubt it spoils anyones carreer opportunities tag counts not is updated a day or two :)


How I think SO should handle this issue

I read this answer :

There are some database timeouts happening in the scheduled jobs that recalculate those scores starting on Jan 22 2014. This job is one of the most expensive things that are done on the site, we're trying to optimize it even further. I'll let you know when the fix will be pushed out.

This is obviously a hard nut to crack. So why not change the rules for recalculating tag scores to UTC 3:00 every day, but only if a user has achieved 50 reps or more since last recalculation? I assume this could lower the stress on the databases to a more acceptable level, and for most users it will not make any difference. People who get a lot of reps on daily basis will be updated every day, people who gain less reps will just have to wait a day or two, maybe more, before their perhaps 5 new upvotes become visible in the tag counts.

share|improve this answer
    
I agree with you to some extent. You took my comment out of context - I was replying to the suggestion of removing this whole feature. Also, I like your suggestion - if it can improve performance of the update job, it's a good one. –  Szymon Mar 26 at 22:16
    
@Szymon, you are right. It was not quite what you write in the first place. –  davidkonrad Mar 27 at 13:10
    
I guess it depends on how exactly the query is run whether or not this idea is even useful. Your check, however, seems like too much effort. With that kind of restriction you'd need to calculate their recent reputation activity relative to the last update time, which just adds more complicated calculations to replace the ones you're trying to avoid. In order for it to be effective, you'd have to simplify that a lot. Something like "did this user have any events at all for today?" would suffice in eliminating any users who haven't changed since yesterday, if that's where the problem lies. –  animuson Apr 2 at 22:49
1  
@animuson, "did this user have any events at all for today?", no - what should that help? Most people, I guess, are not gaining any rep per daily basis. Would it be overcomplicated to add a "lastCalculatedRepValue" field to whatever table, then only run the calculation where current rep differs from "lastCalculatedRepValue" with +50 or more, and if the calculation is runned, update the "lastCalculatedRepValue" with current rep? I think not. Sounds like a very simple and effective way to reduce the number of accounts that needs recalculation. –  davidkonrad Apr 3 at 1:59
1  
tag count recalculation has now been erratic for +3 months, right? –  davidkonrad Apr 3 at 2:00
    
That doesn't really sound simple. And by "no events today" I mean, there's nothing listed in their reputation history today, which implies they gained and lost nothing today. If there were no changes, there's no point in recalculating their tag scores. As well, with your method, you wouldn't catch scenarios where they lost reputation. Someone who got 10 downvotes on an answer, or even had an answer deleted, would keep their tag score until their reputation caught back up to +50. There's just too many variables to think about with that, which makes it complicated. –  animuson Apr 3 at 2:02
    
@animuson, downvoting has only importance for the tag count calculation if, and only if, the answer prior is voted up and has been included in a prior tag count calculation. 10 or 1000 downvotes does not matter in this context for an answer with no upvotes, those inflict on overall rep only. And it is not the normal lifecycle of an regular answer that it get a lot of upvotes, and a few days after is downvoted 10 times, without further upvotes. And if that is a big worry, check if lastCalculatedRepValue is in the span -50/+50. Same deal. –  davidkonrad Apr 3 at 6:19
    
SO certainly not like this :) from +3 to -2 after they got attention to it. No explanation for the downvotes what so ever ;) –  davidkonrad Apr 3 at 16:33
    
And it continues. Keep it on! 158 downvotes to go. All without explanation, if you please. –  davidkonrad Apr 3 at 18:33
7  
@davidkonrad People downvote on meta when they disagree. Downvotes simply mean that. Don't worry about downvotes here, you get a lot up and a lot down. –  Szymon Apr 3 at 19:01
1  
@Szymon, yeah I know. The funny thing is, that the downvoting started when someone from the SO-team interfered. They can downvote me from here and to eternity. They still have not solved the initial problem, and my solution would defently take the overhead of the tag count recalculation. I guess the SO-person answered this is not a skilled database-person. I have dealed with databases where there were 5 billion records per month. –  davidkonrad Apr 3 at 19:24
    
I dont think SO is at that level, with comments, questions and answers totally. Or maybe SO intend to do all thing "perfect", calculate for those who not need recalculating. –  davidkonrad Apr 3 at 19:25
1  
It costs nothing to add add a new field you can compare to. It is essentially database theory. It is a number, costs 2xbyte and could save for a long search. –  davidkonrad Apr 3 at 19:29
5  
@davidkonrad: someone from the SO-team interfered? Are you referring to animuson giving a comment? You do realize that he's a community moderator, right? A volunteer elected by the community, not a Stack Exchange employee? He was just sharing his personal opinion, not 'interfering'. –  Martijn Pieters Apr 10 at 7:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .