4

I am working on a project where I am doing some NLP analyses on various Stack Exchange sites, and I downloaded the data dump made public by Stack Exchange at this link.

When I downloaded and extracted the 7 different Stack Overflow files, I got some COM files. I thought that they would be xml types, as was the case with other files for other sites like superuser.com. Any idea if there is some error in Stack Overflow files? If not, then how can I read the contents of COM files using Python so as to create database contents?

**UPDATE:**These are the files that I am getting after extracting from zip files

pic

5

The "problem" is that your OS is hiding the real extension which is ".7z" from you by default.

Assuming you're using Windows 7, you can fix it by following those steps:

  1. In Windows Explorer, press ALT --> Tools --> Folder options

  2. View tab --> Untick "Hide extensions for known file types"

  3. Click OK

Now you should see the real extensions.

You have to extract (aka unzip) the files, then you should have raw XML.

Edit: As Anna Lear says, you might have stumbled into a real problem:

For all other sites, we produce normally-named XML files within a single .7z file. For Stack Overflow, we split the data dump into several files because it's huge, and there we run into file-naming issues. Feel free to rename the files after unzipping to append .xml and whatever other programs you're using should pick it up after that. The contents are still valid XML

Good luck! :)

| improve this answer | |
  • 2
    And specifically for .7z, 7-zip is the way to go. – Adam Lear May 1 '14 at 22:45
  • @Anna true, but since it's hidden it means it's "known" i.e. he already got it installed. :) – Shadow Wizard Wearing Mask May 1 '14 at 22:48
  • I think you misunderstood my problem. The COM files that I am getting are after extracting from zip files. I have uploaded a pic of the files that I am getting after unzip. – user2966197 May 1 '14 at 22:48
  • @user2966197 Looks like the problem is the same, just at a different stage. Windows is "helpfully" interpreting the ".com-WHATEVER" part of the file name as an extension. Those are XML files in reality. – Adam Lear May 1 '14 at 22:49
  • @user2966197 so those are likely the XML files, Windows will also hide their extension. Have you followed the steps in the answer? – Shadow Wizard Wearing Mask May 1 '14 at 22:50
  • Yes I did but they still remained same.If they are xml files then why my program is throwing error when I am trying to read these files saying that .xml files does not exist? – user2966197 May 1 '14 at 22:52
  • 3
    @user2966197 I see the problem. For all other sites, we produce normally-named XML files within a single .7z file. For Stack Overflow, we split the data dump into several files because it's huge, and there we run into file-naming issues. Feel free to rename the files after unzipping to append .xml and whatever other programs you're using should pick it up after that. The contents are still valid XML. – Adam Lear May 1 '14 at 22:54
  • @Anna now that's worth a new answer, me thinks. :) – Shadow Wizard Wearing Mask May 1 '14 at 22:57
  • @ShadowWizard Nah. Feel free to edit it into your own. Free solutions! ;) – Adam Lear May 1 '14 at 22:58
  • 1
    @Anna so generous of you! Done, but I wonder if you can fix this and have the .xml extension in there? – Shadow Wizard Wearing Mask May 1 '14 at 23:03
  • @ShadowWizard I'm checking with Oded, since he's usually the one who takes care of the data dump these days. – Adam Lear May 1 '14 at 23:03
  • @Anna cheers, say "dash" from me. (he'll know what I mean ;-)) – Shadow Wizard Wearing Mask May 1 '14 at 23:07
  • 4
    Ha. We do handle the SO files differently. Looks like the file extension fell through the cracks. I'm on it. – Oded May 2 '14 at 8:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .