20

As we all know the community eagerly awaits the weekly SEDE refresh with all our posted content so we can analyze, share-alike or train our large language models (LLMs).

The script that runs the refresh has seen some love over the past year and while it should have alerted SE staff by now, I will tell you that the refresh script isn't going to finish today.

My refresh status query shows this:

SEDE refresh rows with Raspberrypi_temp database as restoring since 5:30

As you can tell, the Raspberry Pi site database is currently being restored, but it has been that way since 5:30 UTC. Also note that we have done 264 databases with 100 left to go. That is way too late by this time, as it should be restoring Stack Overflow by now.

Given that accessing the Raspberry Pi database returns this error:

Database 'StackExchange.Raspberrypi' is already open and can only have one user at a time.

I'm pretty sure we hit the same condition earlier, but I can't find the exact post or comment where this was confirmed, but the gist was that the script won't recover on its own from this error state.

Consider this post as a public service announcement for those wondering what's up with SEDE.

Now swallow that Raspberry.

7
  • 12
    SEDE really needs to learn to chew first Commented Dec 31, 2023 at 9:24
  • @Elements In Space: Isn't it about the Stack Exchange site, not the single-board computer? Otherwise, for a cheeky title, it should be the common noun. Commented Dec 31, 2023 at 13:56
  • 6
    @This_is_NOT_a_forum yeah but... humor. Please don't take it away. Commented Dec 31, 2023 at 13:58
  • 3
    @This That’s more than clear from the context. Commented Dec 31, 2023 at 14:00
  • refresh seems to have completed now. at least for SO.
    – starball
    Commented Jan 1 at 11:38
  • @starball it has indeed. I wonder if it was restarted by the on-call or that an automatic monitor noticed the failure and rescheduled the job.
    – rene
    Commented Jan 1 at 12:21
  • @rene well Aaron did say they plan to have auto-fixing in place, maybe they actually added such thing and didn't update the answer. Commented Jan 1 at 14:03

2 Answers 2

9

This was fixed, and we shouldn't see this again(probably). The team responsible has been implementing several improvements to make SEDE more resilient and prevent this type of thing. Some more work is also planned for SEDE to help make it more reliable and faster in the future.

6

This has been magically resolved now, as noticed by starball.

It looks the job restarted at 2023-12-31 15:17:03 and ended at 2024-01-01 02:18:52 completing all 364 databases. Here is a link to a query for a breakdown of start time, end time and duration for each database.

Start: enter image description here

Finish: enter image description here

The query was slightly adopted from How long does the Data Explorer Refresh take for each database?

3
  • 3
    No magic. Obviously, Santa arrived and used his powers to repair the infected raspberry. Commented Jan 1 at 14:02
  • @ShadowWizardLoveZelda wouldn't a joke more consistent with the question analogy be something like a heimlich?
    – starball
    Commented Jan 2 at 6:00
  • @starball the few brain cells assigned to the task weren't able to reach that far. ;) Commented Jan 2 at 7:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .