40

Suppose I'm doing an academic paper, journal article, or other professional presentation. Would you say that it would be possible to use Stack Overflow as a reference?

Of course I'm pretty sure that the answer here would be "yes."

So here's the follow up: When, if ever, would it be a good idea to cite a Stack Overflow question as a reference in an professional paper/presentation of some kind?

2
  • One thing that springs to mind are unofficial "polls." Like for example, you could cite "Is C more easily affected by buffer overflows?" (hypothetical topic) and use it as an anecdotal example to illustrate your point. Nov 11, 2008 at 15:29
  • 4
    in any case your abstract should start with the words "like, suppose..." :)
    – Epaga
    Nov 11, 2008 at 15:41

11 Answers 11

54

In an academic paper, you should be careful with using unofficial references. Stack Overflow is an example of this. Anyone can write anything, and the only guarantee you have as to the quality of the writing is the reputation of the author and the number of up-votes the answer received. Publications such as professional magazines and articles are subject to known peer-reviews and are checked for accuracy and authenticity. While answers on Stack Overflow are also subject to peer-review, the standards of that review are not well known, if they even exist.

As such, while you may want to use Stack Overflow to provide direction for a paper, you would be wise to double check any answers you get, and see if you can find an established paper/article that supports that information. Then you can reference the official source, and give credit to Stack Overflow in general for helping with your research.

5
  • 4
    I would say in many cases the problem is more institutional than philosophical. There are a lot of excellent, rigorous online communities, and many parts of peer review can be just as much a popularity contest. However, if you're going to participate in academia, you have to play by their rules.
    – Robert Elwell
    Nov 11, 2008 at 16:06
  • 1
    This answer makes a very good point. This is why linking to references and standards is so important when answering SO questions, even if the answer is obvious and can be tested within seconds.
    – Pekka
    Jan 31, 2010 at 13:06
  • In other words: please do use StackOverflow as a resource when writing your paper. But not as a reference. Use stackoverflow to help point you to more authoritative locations. Jan 31, 2010 at 15:00
  • 9
    To be fair, I don't think peer review process of low-quality publications is really better than SO.
    – mmx
    Feb 1, 2010 at 16:21
  • "However, if you're going to participate in academia, you have to play by their rules." Yeah, when I had to write my thesis at college, they explicitly prohibited citing any online resources... They allowed to use it when doing a research, but they required the students to trace back everything to a printed book or journal.
    – Calmarius
    Sep 26, 2013 at 14:27
9

Added to the excellent points above, bear in mind that the community-editable nature of the site means that any references you make could well be out-of-date / deleted / modified by the time your paper is published. There are mechanisms for looking back at earlier versions of a question or an answer, but please bear in mind that it is your academic reputation that's at stake, not that of the community.

1
  • Reputations are at stake! I'd prefer to make an ethical attempt at crediting people for their work than to unethically use that work without citing it.
    – Richard
    Feb 11 at 23:03
7

There is one set of circumstances where I believe Stack Overflow should be cited:

If your primary source of information is Stack Overflow, and that information is unavailable elsewhere, and you can verify that information is correct (by testing), then I think it would be very bad manners not to cite Stack Overflow.

This is particularly important if the information sourced on Stack Overflow is central to your thesis.

Examples might be:

A discussion about an undocumented API reference that someone has discovered, and that you can verify by checking a compiled binary.

A programming technique that is unpublished elsewhere, but that can be demonstrated in practice.

2
  • Hi, so this is exactly the situation I'm in. In my paper I am describing a programming technique that as far as I can tell is unpublished elsewhere; or at the very least not any places that I can find. In my paper's Discussions section at the end I am talking about the relative infrequency of the technique, and I want to cite that the only place I've seen it was on StackOverflow. My question is, how do I cite an author? His username is "John".. Should I put the author as "John" in my bibliography?
    – NHDaly
    Dec 1, 2013 at 18:56
  • That sounds reasonable to me.
    – Kramii
    Dec 1, 2013 at 22:19
6

For academic reference purposes, treat Stack Overflow the same as you would treat Wikipedia - somewhere to start, but not definitive.

The exception to this is when you are quoting someone.

5

I take a contrary point of view, in at least one case. If you are benefiting substantially from code proffered by someone on SO, then you can verify that it works and public scrutiny via the up/down votes similarly subjects it to a form of lightly managed peer view. As a result, you are encouraged to cite the poster or the SO post for the source of code that you use.

There are cases where code is posted to SO that is quite a feat, and proper attribution is the professional thing to do.

As for opinions and whether or not SO is a "reliable source", that is immaterial in the case of code. For other SE sites, such as the math site, proofs may be given that are novel and not commonly known. For that, it is also reasonable to cite the author.

The tricky part is that successive edits and refinements may make attribution murky. I have no answer for that, yet.

4

If you are trying to show what problems (or questions or issues, etc.) programmers have in real-life then a set of Stack Overflow questions may be a good source. (Think of using Stack Overflow as a sort of survey when you don’t need to connect the people that are taking part in your experiment.)

However if you are trying to say how an API works, then you better to point to the API documentation.

2

I've seen more answers with wrong things (including posted by myself) than all right on Stack Overflow. You should be very careful with citing Stack Overflow in academic papers.

2
  • 2
    Unless of course the point of your academic paper is that working programmers don't know what the hell they are talking about.
    – Chris Morley
    Nov 11, 2008 at 15:46
  • Thanks to the replication crisis every psych paper I ever read has turned out to be wrong or questionable. StackOverflow doesn't have a monopoly on that. Be ethical: credit people for their work.
    – Richard
    Feb 11 at 23:01
2

It's already being done

Science is better when you build a big tent and make friends. It is sadder when you engage in foolish gatekeeping.

Be like Anonymous 4chan poster, Robin Houston, Jay Pantone, and Vince Vatter, the authors of this paper who discovered an elegant, previously unpublished mathematical proof on a 4chan anime board, cleaned it up, and made "Anonymous 4chan Poster" the first author on their paper (story here).

Help folks feel the warm fuzzies this Tweeter got when he found out this table on Github was cited 10 times. One of those citations being from this arXiv paper which has been cited ~22,000 times.

This Academia.SE answer thinks SO citations are a good idea.

Gatekeeping is BS

The current top answer to this question says

Anyone can write anything, and the only guarantee you have as to the quality of the writing is the reputation of the author and the number of up-votes the answer received.

The folks who wrote this did so before the replication crisis called entire literatures into question and before arXiv (where anyone can write anything) became the de facto journal for machine learning (60% of ML papers are on it).

They wrote this before Brian Wansink's entire scientific output was called into question, before Matthew Walker's widely lauded book "Why We Sleep" was found to contain fraudulent data, before Retraction Watch taught us that journals show only minimal interest in keeping the scientific record correct, at a time when Beall's List was just beginning to show us all the ways you could pay to get your "research" "published" in a "journal".

Sanctimoniously rejecting Stack Overflow as a source of knowledge ignores many real problems in science. It also implicitly encourages folks to use knowledge without crediting the people that create it. That kind of unethical behaviour fits right in with the other problems I've listed above.

Don't gatekeep. Find ways to credit people for their good work.

Stack Overflow is an undervalued resource

If you find code on Stack Overflow or information that solves what would have been a significant problem for you or otherwise greatly accelerates your research, you should find a way to credit that.

Let me make an analogy here. NumPy is the bedrock of mathematical programming in Python. Development began in 2006. NumPy received funding for the first time ever in 2017. In fact, in 2011, Fernando Perez, founder of IPython/Jupyter, gave a keynote at EuroSciPy explaining that the entire scientific Python stack was essentially relying upon on the "free time" work of only about 30 people---and no one had funding!

Software and code is enormously undervalued by academics and funding agencies. Stack Overflow is enormously undervalued in a similar way.

Personally, I donate to software funds and Patreons and have begun trying to cite every software package I use to make a paper happen. This typically adds ~20 citations. Anything that might help volunteers to keep doing what they do, to place more value on their labour.

If you see a good SO answer try to credit it; make the world a better place.

0

Simply put: NO.

From an academic point of view, SO is nothing more than a forum where a bunch of uncredited people discuss and rate themselves.

0
0

It would be a good idea to use Stack Overflow as a reference in some academic or professional presentation if you are using it as an example of an excellent community driven website with the aim of providing technical direction for its users.

Stack Overflow is a fantastic example of a late 2008 bleeding edge fully functional website. It's also remarkable that it is written using ASP.NET MVC technology which is just barely in beta.

Any of the current various merits of Stack Overflow are certainly reference-able with the caveat that, since this is a Beta website living on the cutting edge anything you reference may not exist 6 months or further from now.

So, as fodder for a presentation on late 2008 state-of-the-art technical community websites, Stack Overflow is perfect.

-1

Neither Stack Overflow nor Wikipedia should be used as a reliable source.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .