Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [data-dump]

This tag is about the quarterly creative commons data dumps of all public data in the Stack Exchange network Q&A sites.

4
votes
1answer
65 views

SEDE query to find how many questions remain unanswered related to a particular tag

For example, there are N questions which have a tag named X (there might be other tags also) and M of the questions remain unanswered. I require the number M associated with all tags in Stack Overflow....
2
votes
1answer
68 views

How can I get a notification when a new Stack Exchange dump is made available?

How can I get a notification when a new Stack Exchange dump is made available?
5
votes
1answer
93 views

Retrieve list of top 100 user on stackexchange for a specific tag sorted by posted content?

I am trying to replicate the top user page with user sorted by number of post for a specific tag. Suppose there are two users for tag. User 'A' with 5608 post for tag. And User 'B' with 2627 post. ...
2
votes
0answers
30 views

Does LastActivityDate for a question from Posts.xml consider answers, comments, votes? [duplicate]

From some answers regarding LastActivityDate, I know that LastActivityDate of a question considers: a) Posting a question, b) Answering a question, c) Edits to either question or answer(s) I want ...
1
vote
1answer
93 views

How can I find all the StackExchange websites that I'm part of? - Queries on Data Explorer

How can I find all the StackExchange websites that I'm part of as well as I'm not part of using the Data Explorer..? In addition, if it's possible, I would like to use this query to do additional data ...
0
votes
1answer
41 views

How do I link the Stack Exchange data together? [duplicate]

I was trying to link the Stack Exchange data together from the archive.org data dump as my college database subject project, but I can't understand how to link the Post and its answer. I tried to link ...
6
votes
1answer
46 views

How to find the current top questions by number of views? Or the quarterly number of views per question?

Stack Overflow provides the total number of pageviews per question - but it doesn't tell you if those views are historical or if the question is still current. How can I find the current top questions?...
3
votes
1answer
51 views

How can I get the summary of questions a user ask from the Stack Overflow data dump?

I need information about how many questions a user asks, how many of them are resolved, and how many of them remain unanswered. Is there any way to get this information from the SO data dump?
3
votes
0answers
54 views

Use of downloaded data from data.stackexchange.com

I'm curious about data provided publicly by data.stackexchange.com. It is also available at archive.org. Can I use it for professional purpose? I would like to use only tag names.
1
vote
1answer
70 views

How do I know who made a post Community Wiki from the Stack Overflow data dump?

Is it possible to determine who made a post closed or community wiki and why from the Stack Overflow data dump?
6
votes
1answer
76 views

Is a changed nickname reflected in a user’s previous questions in the data dump?

A user is allowed to change his nickname. If a user submits a question “A” using nickname “userA” and then later on changes his nick to “userB” and submits another question “B”, in the data dump will ...
7
votes
1answer
165 views

Does archival interfere with attribution removal?

There was a question asking about CC BY-SA 3.0's requirement to remove attribute on request. It was brought up that automatic archives may still preserve historical information, including attribution ...
1
vote
2answers
157 views

Total reputation (sum of all reputations) of all Stack Overflow users?

I wanted to know if it is possible to get the total reputation of all Stack Overflow users - i.e. the sum of ALL reputation scores of ALL Stack Overflow users. I am aware that this would be a large ...
4
votes
0answers
34 views

Is it possible to estimate the active reviewer population using the SEDE data?

My main site has, I think, a problem with a shrinking reviewer population, and I would like to use the SEDE data to estimate whether this is indeed the case and, if so, how bad it is and how it has ...
2
votes
1answer
78 views

What is Views of users in the database schema documentation?

In the Database Schema Documentation, what is the Views field for Users? It does not match the (estimated) "number of people reached" displayed for users on their profile page.
2
votes
0answers
136 views

Is Stack Exchange working on providing a GraphQL based API endpoint?

GitHub currently offers both REST API and a GraphQL endpoint to query its entire database. I was wondering if Stack Exchange (SE) is working on offering a GraphQL endpoint sometime soon. As a ...
1
vote
1answer
99 views

Can we redact sensitive information from the data dump?

When we forget to delete sensitive information from our post, we can redact them with the help of moderators. But also, Stack Exchange sites are archived periodically, and saved to the Internet ...
1
vote
0answers
40 views

Related questions under Creative Commons?

Stack Exchange releases its user-generated content in a data dump under a Creative Commons license—this is an incredible resource for researchers like myself, and as far as I know no other site has ...
1
vote
0answers
28 views

Is it possible to obtain data on the evolution of questions' views over time? [duplicate]

In the SE data dump we can obtain a snapshot of the number of views over time. In particular, it is in the Posts table. I am looking for data on the evolution of views over time. My question is ...
5
votes
1answer
66 views

The Reputation in Stack Exchange Data Dump is of SO

I was going through the data dump at https://archive.org/details/stackexchange for Meta Stack Overflow. The reputation field in Users.xml seems to return the reputation of the User on Stack Overflow. ...
4
votes
1answer
45 views

Difference between Votes, PostFeedback and PostNotices

I was curious to know the database schema for the public data dump and came across this thread. Does anyone know the difference between the following tables: Votes PostFeedback PostNotices
3
votes
1answer
53 views

How can one determine one's OwnerUserId on SE?

In the archives of shelved Stack Exchange sites, the authors of a Q or A are identified by a field "OwnerUserId" in XML files. How can one determine one's OwnerUserId? Is it unique across SE sites? ...
5
votes
0answers
91 views

Archival annoyance: Amazon URLs in data dump are rewritten to go through the old Stack Overflow affiliate redirect domain

I was recently generating some pages from data dump content, and noticed something problematic: links to Amazon books/products in the data dump HTML are still rewritten to use Stack Overflow's ...
3
votes
0answers
35 views

View count history [duplicate]

I am using the Stack Overflow data dump (located at https://archive.org/details/stackexchange) for research that involves looking at the popularity history of posts, and I have a question regarding ...
4
votes
0answers
57 views

Invalid Tags in Data Dump

I noticed a minor inconsistency in the 2017-12 Stack Exchange data dump. In the Posts.xml files, the tags for posts are stored in the Tags attribute, each surrounded by <>: <row Id="307517" ...
10
votes
1answer
190 views

Internet Archive dump of Stack Exchange access restricted

The data dump on the Internet Archive says that signing in is required to access the files; however, even after signing in to the site, I only have access to 5 files (mostly metadata files), while the ...
3
votes
1answer
491 views

How can I download the Stack Exchange data dump from archive.org through the command line?

I am trying to download the Stack Exchange data dump from archive.org through the command line using the internetarchive python package developed by archive.org, as follows: pip install ...
13
votes
1answer
346 views

Hosting all Stack Exchange data dumps on archive.org

Currently Stack Exchange only hosts the most recent dump on archive.org (Where are the Stack Exchange data dumps?). The previous dumps are only available through torrents: All Stack Exchange data ...
7
votes
1answer
174 views

Why are links to social media and GitHub not appearing in the SE data dump?

I am looking in the Users table for Stack Overflow in the SE data dump and have noticed something strange. There is a field for the website, namely WebsiteUrl. However, if a user has filled this ...
12
votes
0answers
78 views

Consider donating to the Internet Archive as part of “Stack Overflow Gives Back”

As part of "Stack Exchange/Overflow Gives Back", "some of the tools and organizations that make what we do possible" receive donations of USD 1000. For 2017, this is the list: LetsEncrypt Python ...
3
votes
1answer
66 views

How to get data (question details) of a specific Stack Exchange site?

I am working on a data science project for which I need data (questions and corresponding tags) of a specific Stack Exchange site, Academia SE for example. From this blog we know that SE data is ...
2
votes
2answers
112 views

Checksums for Sept 2017 data dump please

Is anyone facing issues with the data dump (web download) from archive.org? I've tried downloading the files multiple times over the last few days and I keep running into problems with the xml. (...
24
votes
1answer
401 views

Stack Exchange Data Dump no longer available via BitTorrent (since June 2017)

The Data Dump on Archive.org was last updated a month ago, but the torrent version is not available. In fact, no torrent has been made available for any release since June (see my index), when the ...
7
votes
1answer
119 views

Please update the readme.txt in the Data Dump

In the Readme.txt of the SE Data Dump found on archive.org the contents of each site's 7-Zip file is documented. The fact that the tags.xml is available isn't reflected in the readme.txt. Please add ...
1
vote
2answers
50 views

Data on who offered bounties

Is there a way to know who offered bounties on each question on SE? We are working with the SO datadump, and there is data there on bounties in the Votes.xml file. However, there is no information on ...
-9
votes
3answers
70 views

How to get data on who viewed questions and profiles?

I am working with the Stack Overflow datadump and have a general question. I would like to obtain more detailed data on the "views" attribute, pertaining both to question views and profile views. ...
6
votes
1answer
148 views

Choosing best resource for crawling data from Stack Exchange sites

For the purpose of an academic research project, I would like to obtain detailed data on questions, answers, tags, users, etc. That is, I seek historical data, as detailed as possible. I have seen ...
2
votes
0answers
35 views

How to get “people reached” info from Users Table? [duplicate]

How can I get "people reached" data from the Users table for Stack Overflow? I didn't find any fields related to the people reached state in the Users table/schema. Is it possible to get that data ...
7
votes
0answers
71 views

Deleted (Spam) Questions from Stack Exchange (Deep Learning) [duplicate]

I've been trying to implement a Machine-Learning based version of the "Smoke Detector" system by Charcoal, which uses human-crafted regular expressions to currently detect spam. It's already a great ...
1
vote
0answers
61 views

Torrents for latest database dumps [duplicate]

Are the data dumps available as torrents? The official location for the database dumps appears to be here, but they are direct HTTP links, and some of the files are quite large (over 10 GB for ...
19
votes
0answers
120 views

Please add the individual close-as-off-topic reason in the Data Explorer

A couple of previous questions (How to find the actual close reason for a question? and How to determine why a question was closed from the data dump/data explorer) document the fact that it is ...
0
votes
0answers
42 views

Information about moderators in the data dump [duplicate]

I was trying to look for information about the moderators of Stack Exchange Community, but I didn't find any. So can anyone please tell if there is any information available about the moderators in ...
3
votes
1answer
245 views

Does stackoverflow.com-Posts.7 in March 2017 Stack Exchange data dump contain posts from Jan-Mar'17?

The md5sums of stackoverflow.com-Posts.7z files in Mar'17 Stack Exchange data dump and Dec'16 Stack Exchange data dump are identical (776cab94cd0c2244a9dbe3047e74decc). The files are of the same size ...
10
votes
0answers
63 views

Download my voting data

For my own data analysis reasons, it would be useful to have a dump of votes that I've cast on a particular site. I would normally use the SEDE and/or the data dumps, but voting data has been ...
14
votes
3answers
585 views

Data dump, Programmers and Software Engineering

I've removed the poll style answers in favour of open discussion. I realise now this isn't the best way to have a proper discussion about this issue - sorry to anyone who commented/voted. This is ...
2
votes
1answer
64 views

BountyAmount is missing in latest Stack Exchange Data Dump (14-Mar-2017)

We are doing a research project on Stack Overflow data. So I have downloaded the latest Stack Overflow data dump from the Stack Exchange archive (14-Mar-2017). But the table Votes (or any other table) ...
8
votes
1answer
115 views

Why no softwareengineering.stackexchange data in Stack Exchange Data Dump (March 14, 2017)

I can't find softwareengineering.stackexchange data in latest Stack Exchange Data Dump (March 14). I don't know how to report this 'bug', but found a related post about missing Stack Overflow in ...
1
vote
1answer
81 views

Any plans to refresh the Stack Overflow dataset?

The latest dataset hosted by the Internet Archive was created back in 2014. A lot of things have changed in 3 years. Any plans to refresh this dataset?
1
vote
1answer
53 views

How to download DataExplorer queries?

I'm trying to collect (SQL query, description) pairs for a project and the queries here http://data.stackexchange.com/stackoverflow/queries seem very useful. Is there any way to automatically retrieve ...
5
votes
1answer
122 views

What do the Posttype values in the schema of Stack Exchange data dump mean?

I was exploring stack exchange database and came to know there can be various types of posts. Among them, there were many post types that I did not understand. The names of those post types are ...