4

I am working on a project where I am doing some NLP analyses on various Stack Exchange sites, and I downloaded the data dump made public by Stack Exchange at this link.

When I downloaded and extracted the 7 different Stack Overflow files, I got some COM files. I thought that they would be xml types, as was the case with other files for other sites like superuser.com. Any idea if there is some error in Stack Overflow files? If not, then how can I read the contents of COM files using Python so as to create database contents?

**UPDATE:**These are the files that I am getting after extracting from zip files

pic

1 Answer 1

5

The "problem" is that your OS is hiding the real extension which is ".7z" from you by default.

Assuming you're using Windows 7, you can fix it by following those steps:

  1. In Windows Explorer, press ALT --> Tools --> Folder options

  2. View tab --> Untick "Hide extensions for known file types"

  3. Click OK

Now you should see the real extensions.

You have to extract (aka unzip) the files, then you should have raw XML.

Edit: As Anna Lear says, you might have stumbled into a real problem:

For all other sites, we produce normally-named XML files within a single .7z file. For Stack Overflow, we split the data dump into several files because it's huge, and there we run into file-naming issues. Feel free to rename the files after unzipping to append .xml and whatever other programs you're using should pick it up after that. The contents are still valid XML

Good luck! :)

13
  • 2
    And specifically for .7z, 7-zip is the way to go.
    – Adam Lear StaffMod
    May 1, 2014 at 22:45
  • @Anna true, but since it's hidden it means it's "known" i.e. he already got it installed. :) May 1, 2014 at 22:48
  • I think you misunderstood my problem. The COM files that I am getting are after extracting from zip files. I have uploaded a pic of the files that I am getting after unzip. May 1, 2014 at 22:48
  • @user2966197 Looks like the problem is the same, just at a different stage. Windows is "helpfully" interpreting the ".com-WHATEVER" part of the file name as an extension. Those are XML files in reality.
    – Adam Lear StaffMod
    May 1, 2014 at 22:49
  • @user2966197 so those are likely the XML files, Windows will also hide their extension. Have you followed the steps in the answer? May 1, 2014 at 22:50
  • Yes I did but they still remained same.If they are xml files then why my program is throwing error when I am trying to read these files saying that .xml files does not exist? May 1, 2014 at 22:52
  • 3
    @user2966197 I see the problem. For all other sites, we produce normally-named XML files within a single .7z file. For Stack Overflow, we split the data dump into several files because it's huge, and there we run into file-naming issues. Feel free to rename the files after unzipping to append .xml and whatever other programs you're using should pick it up after that. The contents are still valid XML.
    – Adam Lear StaffMod
    May 1, 2014 at 22:54
  • @Anna now that's worth a new answer, me thinks. :) May 1, 2014 at 22:57
  • @ShadowWizard Nah. Feel free to edit it into your own. Free solutions! ;)
    – Adam Lear StaffMod
    May 1, 2014 at 22:58
  • 1
    @Anna so generous of you! Done, but I wonder if you can fix this and have the .xml extension in there? May 1, 2014 at 23:03
  • @ShadowWizard I'm checking with Oded, since he's usually the one who takes care of the data dump these days.
    – Adam Lear StaffMod
    May 1, 2014 at 23:03
  • @Anna cheers, say "dash" from me. (he'll know what I mean ;-)) May 1, 2014 at 23:07
  • 4
    Ha. We do handle the SO files differently. Looks like the file extension fell through the cracks. I'm on it.
    – Oded
    May 2, 2014 at 8:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .