12

I'm having issues decompressing the October/2022 stackoverflow.com-Posts.7z published at https://archive.org/download/stackexchange.

The command works for older dumps, but it fails for this. Is there something wrong with my commands or with the dump?

>7z e ./stackoverflow.com-Posts.7z

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_GB.UTF-8,Utf16=on,HugeFiles=on,64 bits,4 CPUs LE)

Scanning the drive for archives:
1 file, 19430749220 bytes (19 GiB)

Extracting archive: ./stackoverflow.com-Posts.7z
ERROR: ./stackoverflow.com-Posts.7z
./stackoverflow.com-Posts.7z
Open ERROR: Can not open the file as [7z] archive
5

2 Answers 2

9

I am happy to report that the issue was identified on Friday, and since then we have been monitoring a re-upload of all the files. This upload has now completed. All of the compressed dumps should now be available. There is an issue which remains that prevents the view contents link from properly displaying the file content from within Archive.org. The first file is display as expected, but other files are not. We will continue to investigate the cause of this.

Thank you for your patience while we got this straightened out!

We look forward to a smoother process next time.


Update: We worked with the folks from Archive.org to track down the issue responsible for the "View Contents" link not working, and have tested and deployed a fix which will prevent this issue during the next upload.

3
  • 1
    It is completed, the outstanding problem will still be worked on, but it's not really the same issue. Oct 11, 2022 at 15:27
  • But what went wrong, if we may know? Oct 13, 2022 at 20:34
  • @ShadowTheKidWizard For the upload issue, just extra content added to the file inadvertently (basically double-headering). For the displaying only one file issue, I think that's on Archive.org. Patrick is out but we'll be sure to address it before the next quarterly upload.
    – Aaron Bertrand Staff
    Oct 14, 2022 at 15:00
9

Checking the raw file with Notepad, it looks like the culprit is some extra info pushed to the file before the actual contents (click for the full image):

Source of corrupted file

After manually editing the file and saving it without the extra content, it decompressed properly.

Example for extra content:

--2b46d2c8-35ba-4d55-b8ea-ea60b249a695
Content-Disposition: form-data; name=file; filename=stats.meta.stackexchange.com.7z; filename*=utf-8''stats.meta.stackexchange.com.7z

And one blank line, so I am pretty sure that just removing the first three lines from the data dump compressed files will let you decompress them.

14
  • Could you clarify the specific content you're removing to get it to decompress correctly?
    – Spevacus
    Oct 6, 2022 at 14:54
  • 1
    @Spevacus edited to add example, it's visible when editing the compressed file. Oct 6, 2022 at 14:58
  • 4
    We're looking. Source files (read: compressed files immediately prior to upload) don't contain that extra info. Hopeful they can repair in place vs. start over.
    – Aaron Bertrand Staff
    Oct 6, 2022 at 15:54
  • @AaronBertrand so it's something on archive.org's side? Oct 6, 2022 at 17:21
  • 3
    It might be a bit of both. The content-disposition was from us, but even removing that, we still get the boundary GUIDs and the blank line, which must be coming from their side. We've already engaged an engineer over there and I'll update here as soon as I know any more.
    – Aaron Bertrand Staff
    Oct 6, 2022 at 17:27
  • 2
    We received some late-breaking info with a few more details but the TL;DR is we'll have to re-upload the files. Because of the darned clock on the wall and personal out-of-hours commitments, we'll pick this upload back up tomorrow. I know this isn't great news and I'd love to have a better answer but that's the best I can do for now.
    – Aaron Bertrand Staff
    Oct 6, 2022 at 22:57
  • @AaronBertrand "pick this upload back up tomorrow" is great news, could as well be next week, next month, or just abort. So thanks again for the time, it's not something you have to do, but yet you do it anyway. :) Oct 7, 2022 at 5:25
  • 2
    Thanks, @AaronBertrand! I used sed -e '1,3d' stackoverflow.com-Posts.7z > stackoverflow.com-Posts-Fixed.7z to remove 3 first lines from the file, and things seem to work.
    – andreban
    Oct 7, 2022 at 8:32
  • 2
    This isn't something that affects folks ability to use the site and isn't completely time critical. That y'all know what the issue is and its fixable feels like 'good enough' :D. It can wait for a saner hour Oct 7, 2022 at 9:40
  • @Journeyman it's Aaron we're talking about, he's more dedicated than any developer I've ever known. :) Oct 7, 2022 at 11:35
  • Doubly true then :D Oct 7, 2022 at 11:53
  • 3
    This is well on its way again. It's in good hands and Patrick should be posting updates over the weekend.
    – Aaron Bertrand Staff
    Oct 7, 2022 at 22:54
  • Stackoverflow's Comments, PostHistory, Posts, Users and Votes are still corrupt. All the other files seem good from stackoverflow. It also seems unix.stackexchange.com.7z is still corrupt. I acknowledge it's very likely these are being updated as I type, so this is more a "current status report" than a complaint. Oct 9, 2022 at 12:58
  • 3
    @JamesRisner yes the upload speeds are not great, any file you see in the listing with a timestamp from before Oct 7 is still in the queue to be replaced.
    – Aaron Bertrand Staff
    Oct 10, 2022 at 15:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .