Also, since it could probably by done by Jeff, should Jeff provide the data dump in other formats - SQL Server, SQLite, and XML instead of just XML? I'm not sure how intensive it is, but it would probably make it easier for people looking to use or mine the dumps if it was in a multiple formats.
I'm not sure how intensive it is
It would be three times as intensive.. The issue with the data-dump is the file-size - releasing the data in three formats (basically) triples the file-size, and spreads out seeds over three torrents..
I don't think it's worth the hassle, and I think a better solution would be to include a few scripts to convert the XML data into other formats, the concentrate the seeds on one torrent (the XML distribution)
Hm, it might be a good idea to start a cwiki'd question to collect a few Python or Ruby scripts to convert to SQLite/MySQL/Postgres, C# code to convert to MSSQL etc etc.. Then the data-dump readme could simply link to that..
Edit: We shall see if it's a good idea: Scripts to convert data-dump to other formats
We are programmers, we have
XML schemas for breakfast and have data dumps at night. We sometimes get indigested, but that's what scripting languages where made for:
Anatomy of a data dump (useful for your reference)
Both of the last are currently "empty".