17

This issue was brought up, with regard to ServerFault, nearly three months ago. At that time, Jeff responded:

We will do this, but I agree with Michael -- there's not enough data yet. Plus, we'd like to "perfect" the process with the SO data dump before extending it to the other sites.

Check back in a few months...

I'm not sure what "enough" data is precisely, but I think that the need and rationale given for creating the data dumps does not depend on the quantity of said data. And given the fact that there don't seem to be any major issues that have cropped up, I'd say that the process seems to be more or less "perfected".

Finally, now that it's almost 3 months later, I would like to formally request that the September (and all future) dumps contain data from all four sites (SO, SF, SU, Meta).

One thing that does come to mind to address when creating the dump for all the sites is to provide migration information - perhaps additional fields in the posts table: MigratedToSiteId, MigratedToPostId for the origin post and MigratedFromSiteId, MigratedFromPostId for the target post? Is there anything else that needs to be tweaked/added?

edit: Just remembered something else that should go in the dump - account association information. This will help keep track of users' activity across sites, and since it's already public info I don't think that there are any privacy issues. However, it's not clear to me what the ideal format would be, especially if the sites will be in their own separate dumps. What makes most sense to be is to have a table with the user IDs on the different sites as the fields i.e.:

<row StackOverflowUserId="658" ServerFaultUserId="548" SuperUserUserId="6" MetaUserId="658" />

for all users with accounts on 2 or more sites.

2
  • 4
    We'll get it in six to eight weeks. Sep 26, 2009 at 22:03
  • Imagine how much of the Meta data will be posts about skirting the one time "bug" of posting less than 15 characters.
    – random
    Sep 27, 2009 at 0:16

3 Answers 3

1

The new data dump will be out soon and each site will have its own .7z file. Each .7z file will contain both the site and the corresponding meta if there is one.

8

This is probably obvious, but please do not have the dump for all sites in the same .7z file, so everybody can choose which site(s) to download.

3

So from the blog:

oh, and if there are any remaining issues with the data dump PLEASE let us know now. We’re planning to dump Super User and Server Fault next month as well, and I want this to be the final data format...

No mention of Meta. Too many drop-shadowed waffles riding atop ponies jumping through freehand circles for its own good?

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .