Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [data-dump]

This tag is about the quarterly creative commons data dumps of all public data in the Stack Exchange network Q&A sites.

0
votes
1answer
42 views

How do I know who made a post Community Wiki from the Stack Overflow data dump?

Is it possible to determine who made a post closed or community wiki and why from the Stack Overflow data dump?
5
votes
1answer
67 views

Is a changed nickname reflected in a user’s previous questions in the data dump?

A user is allowed to change his nickname. If a user submits a question “A” using nickname “userA” and then later on changes his nick to “userB” and submits another question “B”, in the data dump will ...
5
votes
1answer
74 views

Does archival interfere with attribution removal?

There was a question asking about CC BY-SA 3.0's requirement to remove attribute on request. It was brought up that automatic archives may still preserve historical information, including attribution ...
1
vote
2answers
122 views

Total reputation (sum of all reputations) of all Stack Overflow users?

I wanted to know if it is possible to get the total reputation of all Stack Overflow users - i.e. the sum of ALL reputation scores of ALL Stack Overflow users. I am aware that this would be a large ...
4
votes
0answers
29 views

Is it possible to estimate the active reviewer population using the SEDE data?

My main site has, I think, a problem with a shrinking reviewer population, and I would like to use the SEDE data to estimate whether this is indeed the case and, if so, how bad it is and how it has ...
1
vote
1answer
49 views

What is Views of users in the database schema documentation?

In the Database Schema Documentation, what is the Views field for Users? It does not match the (estimated) "number of people reached" displayed for users on their profile page.
1
vote
0answers
65 views

Is Stack Exchange working on providing a GraphQL based API endpoint?

GitHub currently offers both REST API and a GraphQL endpoint to query its entire database. I was wondering if Stack Exchange (SE) is working on offering a GraphQL endpoint sometime soon. As a ...
1
vote
1answer
88 views

Can we redact sensitive information from the data dump?

When we forget to delete sensitive information from our post, we can redact them with the help of moderators. But also, Stack Exchange sites are archived periodically, and saved to the Internet ...
1
vote
0answers
36 views

Related questions under Creative Commons?

Stack Exchange releases its user-generated content in a data dump under a Creative Commons license—this is an incredible resource for researchers like myself, and as far as I know no other site has ...
0
votes
0answers
27 views

Is it possible to obtain data on the evolution of questions' views over time? [duplicate]

In the SE data dump we can obtain a snapshot of the number of views over time. In particular, it is in the Posts table. I am looking for data on the evolution of views over time. My question is ...
5
votes
1answer
61 views

The Reputation in Stack Exchange Data Dump is of SO

I was going through the data dump at https://archive.org/details/stackexchange for Meta Stack Overflow. The reputation field in Users.xml seems to return the reputation of the User on Stack Overflow. ...
3
votes
1answer
36 views

Difference between Votes, PostFeedback and PostNotices

I was curious to know the database schema for the public data dump and came across this thread. Does anyone know the difference between the following tables: Votes PostFeedback PostNotices
3
votes
1answer
48 views

How can one determine one's OwnerUserId on SE?

In the archives of shelved Stack Exchange sites, the authors of a Q or A are identified by a field "OwnerUserId" in XML files. How can one determine one's OwnerUserId? Is it unique across SE sites? ...
4
votes
0answers
82 views

Archival annoyance: Amazon URLs in data dump are rewritten to go through the old Stack Overflow affiliate redirect domain

I was recently generating some pages from data dump content, and noticed something problematic: links to Amazon books/products in the data dump HTML are still rewritten to use Stack Overflow's ...
2
votes
0answers
30 views

View count history [duplicate]

I am using the Stack Overflow data dump (located at https://archive.org/details/stackexchange) for research that involves looking at the popularity history of posts, and I have a question regarding ...
2
votes
0answers
40 views

Invalid Tags in Data Dump

I noticed a minor inconsistency in the 2017-12 Stack Exchange data dump. In the Posts.xml files, the tags for posts are stored in the Tags attribute, each surrounded by <>: <row Id="307517" ...
10
votes
1answer
157 views

Internet Archive dump of Stack Exchange access restricted

The data dump on the Internet Archive says that signing in is required to access the files; however, even after signing in to the site, I only have access to 5 files (mostly metadata files), while the ...
3
votes
1answer
375 views

How can I download the Stack Exchange data dump from archive.org through the command line?

I am trying to download the Stack Exchange data dump from archive.org through the command line using the internetarchive python package developed by archive.org, as follows: pip install ...
8
votes
1answer
214 views

Hosting all Stack Exchange data dumps on archive.org

Currently Stack Exchange only hosts the most recent dump on archive.org (Where are the Stack Exchange data dumps?). The previous dumps are only available through torrents: All Stack Exchange data ...
7
votes
1answer
165 views

Why are links to social media and GitHub not appearing in the SE data dump?

I am looking in the Users table for Stack Overflow in the SE data dump and have noticed something strange. There is a field for the website, namely WebsiteUrl. However, if a user has filled this ...
12
votes
0answers
78 views

Consider donating to the Internet Archive as part of “Stack Overflow Gives Back”

As part of "Stack Exchange/Overflow Gives Back", "some of the tools and organizations that make what we do possible" receive donations of USD 1000. For 2017, this is the list: LetsEncrypt Python ...
3
votes
1answer
57 views

How to get data (question details) of a specific Stack Exchange site?

I am working on a data science project for which I need data (questions and corresponding tags) of a specific Stack Exchange site, Academia SE for example. From this blog we know that SE data is ...
2
votes
2answers
106 views

Checksums for Sept 2017 data dump please

Is anyone facing issues with the data dump (web download) from archive.org? I've tried downloading the files multiple times over the last few days and I keep running into problems with the xml. (...
24
votes
1answer
320 views

Stack Exchange Data Dump no longer available via BitTorrent (since June 2017)

The Data Dump on Archive.org was last updated a month ago, but the torrent version is not available. In fact, no torrent has been made available for any release since June (see my index), when the ...
7
votes
1answer
111 views

Please update the readme.txt in the Data Dump

In the Readme.txt of the SE Data Dump found on archive.org the contents of each site's 7-Zip file is documented. The fact that the tags.xml is available isn't reflected in the readme.txt. Please add ...
1
vote
2answers
46 views

Data on who offered bounties

Is there a way to know who offered bounties on each question on SE? We are working with the SO datadump, and there is data there on bounties in the Votes.xml file. However, there is no information on ...
-8
votes
3answers
67 views

How to get data on who viewed questions and profiles?

I am working with the Stack Overflow datadump and have a general question. I would like to obtain more detailed data on the "views" attribute, pertaining both to question views and profile views. ...
6
votes
1answer
126 views

Choosing best resource for crawling data from Stack Exchange sites

For the purpose of an academic research project, I would like to obtain detailed data on questions, answers, tags, users, etc. That is, I seek historical data, as detailed as possible. I have seen ...
2
votes
0answers
35 views

How to get “people reached” info from Users Table? [duplicate]

How can I get "people reached" data from the Users table for Stack Overflow? I didn't find any fields related to the people reached state in the Users table/schema. Is it possible to get that data ...
7
votes
0answers
69 views

Deleted (Spam) Questions from Stack Exchange (Deep Learning) [duplicate]

I've been trying to implement a Machine-Learning based version of the "Smoke Detector" system by Charcoal, which uses human-crafted regular expressions to currently detect spam. It's already a great ...
1
vote
0answers
34 views

Torrents for latest database dumps [duplicate]

Are the data dumps available as torrents? The official location for the database dumps appears to be here, but they are direct HTTP links, and some of the files are quite large (over 10 GB for ...
19
votes
0answers
117 views

Please add the individual close-as-off-topic reason in the Data Explorer

A couple of previous questions (How to find the actual close reason for a question? and How to determine why a question was closed from the data dump/data explorer) document the fact that it is ...
0
votes
0answers
42 views

Information about moderators in the data dump [duplicate]

I was trying to look for information about the moderators of Stack Exchange Community, but I didn't find any. So can anyone please tell if there is any information available about the moderators in ...
3
votes
1answer
214 views

Does stackoverflow.com-Posts.7 in March 2017 Stack Exchange data dump contain posts from Jan-Mar'17?

The md5sums of stackoverflow.com-Posts.7z files in Mar'17 Stack Exchange data dump and Dec'16 Stack Exchange data dump are identical (776cab94cd0c2244a9dbe3047e74decc). The files are of the same size ...
8
votes
0answers
56 views

Download my voting data

For my own data analysis reasons, it would be useful to have a dump of votes that I've cast on a particular site. I would normally use the SEDE and/or the data dumps, but voting data has been ...
14
votes
3answers
565 views

Data dump, Programmers and Software Engineering

I've removed the poll style answers in favour of open discussion. I realise now this isn't the best way to have a proper discussion about this issue - sorry to anyone who commented/voted. This is ...
2
votes
1answer
61 views

BountyAmount is missing in latest Stack Exchange Data Dump (14-Mar-2017)

We are doing a research project on Stack Overflow data. So I have downloaded the latest Stack Overflow data dump from the Stack Exchange archive (14-Mar-2017). But the table Votes (or any other table) ...
8
votes
1answer
114 views

Why no softwareengineering.stackexchange data in Stack Exchange Data Dump (March 14, 2017)

I can't find softwareengineering.stackexchange data in latest Stack Exchange Data Dump (March 14). I don't know how to report this 'bug', but found a related post about missing Stack Overflow in ...
1
vote
1answer
77 views

Any plans to refresh the Stack Overflow dataset?

The latest dataset hosted by the Internet Archive was created back in 2014. A lot of things have changed in 3 years. Any plans to refresh this dataset?
1
vote
1answer
46 views

How to download DataExplorer queries?

I'm trying to collect (SQL query, description) pairs for a project and the queries here http://data.stackexchange.com/stackoverflow/queries seem very useful. Is there any way to automatically retrieve ...
5
votes
1answer
97 views

What do the Posttype values in the schema of Stack Exchange data dump mean?

I was exploring stack exchange database and came to know there can be various types of posts. Among them, there were many post types that I did not understand. The names of those post types are ...
2
votes
2answers
180 views

Stack Overflow Data Dump: Why does Posts.Body contain HTML elements and PostHistory.Text not?

My goal is to explore the evolution of code snippets on Stack Overflow. Therefore, I want to use data from the official Stack Overflow data dump. I noticed that table Posts.Body contains HTML elements ...
4
votes
1answer
97 views

How often is the Stack Overflow's public data dump updated?

I'd like to automate the creation of activity reports on Stack Overflow. I'm hoping to use the public data dump that Stack Overflow has made available. Can you please tell me how often it's updated?
-2
votes
1answer
109 views

Offer questions from closed SE sites to migration review queue of open ones

Rather than leave the questions/answers from closed SE sites to be accessed in data dumps, could they be filtered and offered to other SE sites based (the most relevant and popular first)? I offer ...
2
votes
0answers
93 views

Can you switch to a better compression algorithm for the Stack Exchange dump files?

This question has already been asked in one form or another but the problem still exists at least for me. I have been working with some of the largest dump files recently e.g. stackoverflow.com-Posts....
3
votes
1answer
65 views

How to get the tag-synonym data dump of Stack Overflow?

I want a data dump of the information at https://stackoverflow.com/tags/synonyms. How can I get that?
1
vote
1answer
351 views

Download stack overflow dump related to specific tag

I know we can download the entire stack overflow database from the web Archive Is there any way to query stack overflow database archieve. Entire database is around 30 gb and I don't want all. I want ...
15
votes
0answers
122 views

Why does the SEDE ReviewTaskResults table not include user data, which is all public anyway?

I wanted to do some data mining on the SEDE, and I ran across a limitation which I cannot understand. The Data Explorer table ReviewTaskResults currently contains the following information: Id ...
4
votes
2answers
260 views

Offline version of Stack Exchange for specific interest, for example, Python programming

I have no intent to post a duplicated thread here. I'm interested in an offline version of Stack Overflow where I can study on the run as my job is an interstate courier that does not always have ...
4
votes
1answer
79 views

How to get information about deleted users in Stack Exchange data dump?

I was looking at Stack Exchange database and I wanted to know how to get the information about the deleted users. I didn't find any such information in schema of data dump. I specifically wanted to ...