Recently, Stack Overflow posted a blog post about their Providence system (see also a 2015 blog post by Kevin Montrose about this same subject). This is related to the Personalized Prediction Data.

It appears that Stack Exchange keeps a large amount of personally identifiable information for an indefinite period of time and doesn't have a clear data retention policy documented in the Privacy Policy.

The Personalized Prediction Data tool lets me delete and stop collection of my personalized prediction data, but will hinder question suggestions. This means that I either allow Stack Exchange to gather large amounts of data and have access to tools that let me find interesting and relevant content or I disable it and don't. There should be a middle ground - identify a period of time, after which data is no longer relevant, and purge it.

Following European Union laws and commonly accepted best practices and ethical behavior, Stack Exchange should update their functionality and associated privacy policy to identify a data retention policy and delete/destroy data that is no longer relevant.

We now have sites about the workplace, politics, various religions, health, and law. Consider a worse-case scenario - a person in a repressive regime viewing content deemed "bad" or that can be used to single them out for some kind of retribution. Actors, through legal or illegal means, should not be allowed to obtain data that can tie what types of content is viewed to an individual.

This isn't necessarily a far-fetched thought. Internet freedom is on the decline. In order to protect individuals, the amount of data collected, inferred, and stored by services (especially the data that can be tied to a particular individual) should be minimized while still maintaining the performance of the service. If the data is not collected or stored, it cannot (as easily) be obtained by third parties through legal (or illegal) action.

  • 5
    I wouldn't say that site activity (things like question views) is entirely useless, even if it's not obviously used now. I can imagine plenty of uses: building user interest models and using them to provide a view of suggested questions to read, or refining view count/influence metrics over time, doing analysis of activity later to try to learn what works and doesn't work for sites, etc. Anyways, maybe there's room for improvement in these policies, but it might be better to make fewer assumptions as part of the question, and distinguish PII from site activity? – Cascabel Nov 29 '16 at 20:15
  • 1
    @Jefromi If there's no use right here, right now for the data, then it should not be gathered now (for example, the phrase "adequate, relevant and not excessive" is used in the EU laws). However, when it is needed for a particular purpose, then it is gathered, there should be specific data retention policies. I'm not sure what assumptions I'm making in this question, other than the fact that SE does not specify their data retention policies and "forever" (especially for any data tied to a particular individual) is generally not considered acceptable without a rationale. – Thomas Owens Nov 29 '16 at 20:20
  • 8
    The critical part of this, I think, is described on the personalized prediction data page - which depends on the privacy policy for details on handling, but provides its own description of how the data will be used, how it can be purged, and most importantly what is actually tracked. IOW, you control the parameters for 4 & 5 as they apply to you. – Shog9 Nov 29 '16 at 20:47
  • @Shog9 The question mentions page views (used as inputs to the prediction, I think?) and IP addresses (unrelated) - I think the idea is to add retention policies for those, not just the predictions. If I understand right, disabling predictions clears the predictions but leaves the existing saved activity alone. – Cascabel Nov 29 '16 at 20:53
  • 1
    @ThomasOwens You're assuming that information is being collected that isn't useful indefinitely. While I understand the general principle there, I don't think it's really quite that black and white. StackExchange doesn't have to spell out every feature of the site and each piece of information that's necessary to provide those features. If they have a current or planned feature that requires some kind of information. They don't have to build the "detect sock puppets based on question views" feature before they start recording question views. "review and document" - yes. "discard" - maybe. – Cascabel Nov 29 '16 at 20:56
  • 2
    @Shog9 The personalized prediction page gives me an option to delete my data and disable (which it says degrades functionality). There should be a middle ground between what appears to be "keep lots of stuff tied to me as an individual forever" and "delete and stop collecting everything". What I'm asking is for this middle ground to be created (and documented). Some stuff needs to last forever - tying a post to an account, for example. But other stuff, like the fact IP address Y asked or viewed a particular question Z, not so much. – Thomas Owens Nov 29 '16 at 21:08
  • Then you're down to talking about server logs, @Jefromi. What we know about you as an individual is attached to your profile (and covered under the privacy policy and/or prediction page), but there is of course a tremendous amount of source material that your account, and your individual site profiles, tie together. We could probably get a lawyer to phrase this in a way that'd be suitable for the privacy policy, but as far as the core concern here (inference) goes I think it's looking at the wrong end. – Shog9 Nov 29 '16 at 21:10
  • The functionality it refers to is that described in the paragraphs above: ad targeting / job recommendations, @Thomas - maybe question recommendations, although you have much more control over that. Those systems have access to the data you can download from your profile (and, crucially, not the source material, the logs, that it's based on). So you can determine whether that's something you're concerned about and purge as-needed. – Shog9 Nov 29 '16 at 21:13
  • 4
    @Shog9 That's the first time I've heard that I can opt out of some tracking. Good to know. But why isn't that prediction data page linked from the privacy policy or other relevant places? Any links I could find are on Meta or on 3rd party sites. – amon Nov 29 '16 at 21:13
  • 1
    @Jefromi Some stuff is useful forever. For example, tying an account to a post. The system wouldn't work otherwise. A feature doesn't have to exist for data to be retained. There simply needs to be a need. If I know I need data X and Y to implement a function, I should retain X and Y, even if I haven't implemented the function yet, since I can explicitly tie the data to a need. All I'm asking for is that SE takes a hard look at what data they are collecting and storing and stop storing data for what appears to always be an indefinite length of time. – Thomas Owens Nov 29 '16 at 21:15
  • @Shog9 The personal prediction page says "this might mean showing you suggested questions that are more likely to be of use to you, based on the tags and technologies you usually browse". I may be wrong, but I read this as if I'm logging in and I view a question X on a site Y, the fact that I viewed that question is forever stored. As a user, I want to see suggested questions that are more likely to be of use to me. But let's take Health. Do you need to always store the fact that I browsed a bunch of questions about a specific disease or illness forever? Or is a one year window sufficient? – Thomas Owens Nov 29 '16 at 21:19
  • 2
    Did you look at the data, @Thomas? It's not a list of questions you've viewed; it's essentially a set of weights attached to tags. It's derived from access logs, but it doesn't actually contain that data. – Shog9 Nov 29 '16 at 21:30
  • 1
    @Shog9 It's still identifiable. You can tell that, at some point, I'm interested in something that may be seen as bad by a hostile entity, which would put me at risk. – Thomas Owens Nov 29 '16 at 21:46
  • 2
    @Shog9 I shouldn't have to make the choice. Keep enough data around for long enough to be useful, but short enough to keep me safer (for example, throw away tags that I haven't viewed in the last....90 days?). – Thomas Owens Nov 29 '16 at 21:50
  • 1
    @Shog9, My prediction data has a field named QuestionViews and a list of 33,944 question ids. – Andy Nov 30 '16 at 4:41

You must log in to answer this question.

Browse other questions tagged .