2

Are there any scripts available to import the data dump into databases other then SQL Server? I don't have an SQL Server license, and the data dump will probably out grow the SQL Server Express 4 gig limit if it hasn't already.

Improve this question
1
  • fyi it has not outgrown the 4 gig size yet.
    – waffles
    Commented Aug 25, 2009 at 22:08

3 Answers 3

Reset to default
2

I've created a script that uses Groovy and Hibernate to at least in theory, import the data dump into any database Hibernate supports.

I have gotten it to work with MySQL and PostgreSQL but it doesn’t work on HSQLDB, those are the only three I’ve tested it on. The code isn’t optimized and takes about an hour to import all five files, but I plan on improving it when I get a chance. The Git repository can be found on Github, I hope it's useful to others.

Improve this answer
4
  • @Jared, I load it up on SQL in about 8 minutes, including tag splitting and indexing, make sure you have a look through my code for some of the tricks used, granted the fast inserter is not available with hibernate github.com/SamSaffron/So-Slow/tree/master
    – waffles
    Commented Aug 25, 2009 at 22:10
  • I'll take a look at your code when I get a chance. This was actually the first time I've used Hibernate so I'm sure there's room for improvement.
    – Jared
    Commented Aug 25, 2009 at 23:21
  • It looks like the code is gone now.
    – carson
    Commented Oct 22, 2009 at 19:06
  • I found some problems with the code so took it down. I haven't gotten around to fixing it yet. Since there's a ruby script to import the data into mysql it's not a high priority of myne now.
    – Jared
    Commented Nov 18, 2009 at 17:53
0

Here is a windows application that will import to MySQL, MS SQL Server and SQLite:

https://meta.stackexchange.com/questions/45333/fast-multi-platform-data-dump-import-sql-2000-05-08-sqlite-mysql

Improve this answer
0

There's been an SQLite version of the data dump for past versions... I'll dig up a couple of URLs.

Here's the June dump - I don't believe July and August are available, unfortunately.

I have some code to convert it into Protocol Buffer files, but that's probably a bit of a niche market :)

There's also the Data Dump Sandbox if you're happy to use a web interface :)

Improve this answer
2
  • No, they're not, I've been meaning to get to them, but I wanted to tweak my import script first. I don't know if it was SQLite or my script, but I was experiencing an exponential slowdown in the number of records I was processing. June took half an hour on my machine, and I fear that July and August would take even longer...
    – Kyle Cronin
    Commented Aug 11, 2009 at 19:44
  • All props to Ian for the Sandbox. It's MySQL, btw.
    – Eric
    Commented Aug 11, 2009 at 19:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .