13

For some reason, chat searches are not returning any results newer than July 8 at about 2:05 PM UTC. This example search pinpoints the issue.

The problem appears to be caused by the recent datacenter transfers. The time cutoff is apparently defined by the time the failover to Colorado occurred, and search began returning missing results after the failback to New York. It seems as if the search engine is running on outdated indexes that were not updated since the Colorado failover.

Can someone look into this?


Edit: This issue appears to be isolated to the chat.stackexchange.com server. The other two chat servers are unaffected.

1
  • Just FYI: Chat search indexes are up to date on SO and MSE – rene Jul 22 '17 at 6:16
8

We had an issue with our scheduler service crashing due to the failover last night. The scheduler has been restarted and your example search appears to have recent results now.

4
  • Any chance to add some alert to notify you when such crash occurs, so that you might restart it before it cause such problems? – Shadow The Vaccinated Wizard Jul 22 '17 at 20:02
  • already added to our followup items. Its our biggest pile of tech debt right now, needs a full replacement as it keeps causing issues. – Greg Bray Jul 22 '17 at 20:04
  • But it worked for years without a problem. What changed? – Shadow The Vaccinated Wizard Jul 22 '17 at 21:36
  • Also, do you have the power to mark this bug report as status-completed? – Shadow The Vaccinated Wizard Jul 22 '17 at 21:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .