Over the past year, a lot of meta data has been changed or added to the system:
- Review tasks and results
- Duplicate links (separate from close votes and ad-hoc editing conventions)
- Close votes as a distinct type of flag (previously they were stored as votes)
However, none of this is in the public data dump. There are bits and pieces - suggested edits predate the new review system, and duplicate links can be obtained from PostHistory with a bit of trouble - but not enough for any serious analysis.
Therefore, I recommend the following information be added to the public data dumps and data explorer:
Close and reopen flags
As with close votes, this information should have userIDs and exact times stripped before being made public. Suggested schema:
Id FlagTypeId (Recommend Close: 13, Close: 14, Reopen: 15) PostId CreationDate (date only - no exact time) CloseReasonId (requires the CloseReasons table) CloseAsOffTopicReasonId (requires the CloseAsOffTopicReasons table) DuplicateOfQuestionId BelongsOnBaseHostAddress ResultDate (this and ResultTypeId are only needed if deleted flags should be included) ResultTypeId
The question of whether or not deleted votes/flags should be included is a big one here: IMHO, it's very useful for doing historical analysis... But right now it's explicitly not included. If you can think of a good reason not to make this information public, post it below.
The PostLinks table is quite useful: it contains not only duplicate links, but also ad-hoc links and even related links (the two sidebar lists next to most questions). I would like to have all of this information made available, but if this isn't feasible for some reason then ad-hoc links and duplicate links would be the most useful. Suggested schema:
Id CreationDate PostId RelatedPostId LinkTypeId (there is no LinkTypes table, but maybe there should be one) Rank
Since all of this information is already public, I don't foresee any privacy issues with publishing it in the dump.
This covers everything available under /review. Nothing in this table should really be private, but I would recommend excluding information on audits for space and simplicity's sake. Suggested schema:
Id ReviewTaskTypeId (requires the ReviewTaskTypes table) CreationDate DeletionDate ReviewTaskState (requires the ReviewTaskStates table) PostId SuggestedEditId (this table already exists in the public data)
Note that "deleted" review tasks are still publicly visible - DeletionDate is really just the data on which the task was either completed or invalidated.
Review task results
Id ReviewTaskId ReviewTaskResultTypeId (requires ReviewTaskResultTypes table) CreationDate (date only - no exact time) RejectionReasonId (requires ReviewRejectionReasons table) Comment (custom rejection reasons for suggested edits) CompletedReviewTask (bit) (this doesn't actually exist internally, but I recommend calculating it for convenience - particularly if timestamps are omitted)
This is a tricky one... Most of this information is already public in some fashion, but making it easily accessible is another matter. In particular, First Post and Late Answer reviews can involve actions that are not made public: voting and flagging. Making it too easy to correlate a FP/LA review with specific events on a post could lead to trouble. Therefore, I recommend the following:
- As with votes, omit the ID of the user reviewing.
- As with votes, omit the exact time that the review took place
- Omit any detailed information on the actions that contributed to the review for First Posts and Late Answers reviews.
- Omit audit information (if it is left out of ReviewTasks)
I may be over-cautious here... For instance, it's possible that including #1 would be fine as long as #3 was omitted. Post your thoughts below.
Account ID on Users rows
This is actually a very old request, but Tim Stone reminded me of it again tonight. This is public information already; at one point, the ability to disassociate accounts from your network profile may have made it problematic, but given we don't allow that anymore... I don't see any issue here.
It's another integer in each row in the Users table that corresponds to your network ID (mine is 620) and is identical on each site linked to your network account.
Please review the above suggestions. In particular, if you can envision a way in which the release of such data could be used to compromise information that is normally considered private (such as voting data), call it out now - much better to be safe than sorry.