71

Possible Duplicate:
Remove nofollow on links deemed reputable

I propose that after a certain amount of time has passed since the last edit on a question or post, that all links within that post should no longer be flagged with rel="nofollow". Here is my reasoning:

  1. The web is built on creating meaningful links. There are literally hundreds of thousands of really good answers on Stack Overflow pointing to lots of great information elsewhere on the web. By blocking crawlers from following these links, those pages (with useful info on them) are not getting the google juice they deserve, and the web as a whole suffers.

  2. The community is very active at self-policing. It would be unthinkable that a question or answer which has spam links in it would survive for 5 minutes, let alone whatever the time limit would be on nofollow.

I can however see a downside - users would have extra motive to link to their own websites or blogs in answers. I could see that this might lead to a bit of resentment from other users who might question your reasons - you already feel slightly guilty doing the occasional self-plug...

Perhaps instead of a time limit, it might be a certain number of votes, or the reputation of the user, or whatever combination of these things isn't too complicated.

Update: this proposal was tested and the data so far is pointing to it resulting in a fairly severe drop in organic web traffic for Server Fault. Therefore, we are declining this proposal indefinitely as too risky.

18
  • 4
    rel="nofollow" doen't tell crawlers not to follow the links, it only tells them not to count the fact that SO links to some page towards the page's rank. (That doesn't invalidate your "google juice" point, though)
    – balpha StaffMod
    May 25, 2010 at 15:47
  • 3
    @balpha - each search engine treat nofollow differently. Some, in fact, do not follow the link at all and do not index it unless it appears on some other page. Others follow it, but do not calculate it. Still others follow it, and still accrue rank to the linked page, though with a different calculation than regular links. Wikipedia has quite a bit of info on what people have discovered about it, as SEO experts have explored this area and use nofollow and other tools to sculpt their pagerank in elaborate ways.
    – Pollyanna
    May 25, 2010 at 18:14
  • 13
    @polly I find your beliefs fascinating, but since Google is 99.6% of all incoming search traffic, and 87% of our total traffic, I humbly submit that I don't give a damn what other search engines may or may not do with nofollow. Oct 31, 2010 at 5:46
  • @Jeff - Hey now, don't ruin a good argument with facts and figures!
    – Pollyanna
    Oct 31, 2010 at 6:13
  • 6
    @Jeff so by status-declined .. case closed? Perhaps there's an innovative solution waiting to be discussed, discovered, etc.? If one of SO's goals is "making the Internet a better place", finding a better mechanism to judiciously follow/no-follow supporting links is entirely consistent with that goal. Oct 31, 2010 at 14:49
  • 1
    @chris we are following Wikipedia's lead here -- if they change their policy, we will change ours Nov 3, 2010 at 3:55
  • 5
    I think it's amusing that people are engaging in free labour to help Google overcome the webspam problem that they've caused. Yet @Jeff I'm even more amused by the display of double standards here. It's one thing to hide between someone elses policy. But since the Stackoverflow content license (blog.stackoverflow.com/2009/06/attribution-required) forbids nofollow links back, this revokes your right to complain. (Also as contributor I didn't knowingly agree to the pseudo-CC license restrictions, so it's not just webethically void.)
    – mario
    Nov 23, 2010 at 0:54
  • 3
    @mario We do care about content authors credit, which is why we insist on proper creative commons attribution for authors blog.stackoverflow.com/2010/08/defending-attribution-required and also why we automatically follow (remove nofollow) the URL on your user profile page at 2k reputation. Spam is a serious, deep problem on the web and we allow anonymous users to post on our sites -- so it's a bit dangerous to open the barn door too wide, and risk getting de-listed by Google (90% of our traffic is through Google) because of a few malicious users posting nefarious links. Nov 23, 2010 at 1:20
  • 1
    @Jeff Obviously this topic is more a politicum than a technical issue. In my opinion, nofollow doesn't help an inch in the war on spam. It very much helps the business case though. (Clean content.) Not trying to smear Google; they caused this unintentionally, but we all understand it's really just about not upsetting them and losing ranking in the process. Socially the right thing to do would be to use proper hyperlinks. But I also believe this is technically and community-wise a non-issue.
    – mario
    Nov 23, 2010 at 1:38
  • 5
    @Jeff, it's also ignoring the basic premise of the original question. Stack Overflow has the ways and means to know who are trusted users via reputation. As said in the FAQ: Reputation is a rough measurement of how much the community trusts you. By putting "nofollow" on everyone's links, you're saying that you don't trust anyone. The "one link from your homepage at 2k rep" is also a weak response. It's not about rewarding top users with a link to their homepage, but about rewarding the people who are writing the articles which answer the questions on the site.
    – nickf
    Nov 23, 2010 at 10:42
  • @nickf "By putting "nofollow" on everyone's links, you're saying that you don't trust anyone" no, we're saying that when it comes to links, we treat everyone equally and fairly. You'd prefer some other kind of preferential treatment for certain users? What next, their upvotes carry more weight? Dec 20, 2010 at 8:26
  • 6
    @Jeff: so how is that different from removing nofollow from links in a user's profile page when they reach 2k? That's "treating everyone equally and fairly" because everyone has the chance to attain that level of reputation? By implementing this feature request, you'd still be treating everyone equally and fairly because everyone would be subject to the same criteria of having nofollow removed from their links.
    – Andy E
    Dec 20, 2010 at 10:56
  • @Jeff, well that's clearly not what I'm arguing for. Please revisit the original text of this post: I'm not saying that certain users should get all their links nofollowed, I'm saying that there are valid ways to tell whether something is spam or not - some amount of time plus the self-policing community I think would be good enough. The fact that a certain link has been added by someone who you trust highly (by the way of their reputation) could also be added into the equation.
    – nickf
    Dec 21, 2010 at 12:47
  • 2
    @Jeff: Andy Es is correct. SO already has a intricate system of treating those with higher reputations differently. I personally think basing whether or not nofollow is applied on the reputation of the user makes a lot of sense. To me, it makes more sense than the approach suggested by nickf. And putting nofollow on every single link seems to make the least sense of all. Jan 1, 2011 at 16:29
  • 3
    @Jeff: is it possible that the downturn of traffic was actually caused by the recent Google algorithm update? google.com/q=google+panda+update
    – nickf
    Apr 17, 2011 at 17:52

7 Answers 7

44

URGENT UPDATE

We were seeing a significant drop in Google (organic) traffic for Server Fault after instituting this "follow links if enough upvotes post-edit or post-create" policy.

We traced it back to what we currently think are a string of posts on Server Fault that got nofollow removed through "trust", but were being interpreted by Google as link farms or spammy pages. Examples:

So, for the record, we're not 100% sure, but this is the only thing we've changed -- and the evidence is fairly damning so far, though not quite conclusive. I had many reservations about risk in doing this at all, so the idea of following any links is going away for the forseeable future until we rule out other possibilities.

Back to the old safe, sane, policy that I advocated from the first day this came up: no external links are followed.*

Sorry.

-- Jeff Atwood

* Except for the URL field on your user profile, which at 2k rep follows. This isn't new but I wanted to be clear it isn't going away.

Stack Overflow is driven by incoming links—ranking SO results high in Google is crucial—and it is those incoming links that bring people to the site.

We really should take your comment about "not getting the Google juice they deserve" to heart. Stack Overflow is about making the Internet a better place and giving back.

rel="nofollow" serves a purpose but, perhaps, there is some criteria that can determine when a post surpasses the potential for wide-spread abuse and can be placed in the "giving back" column. Some combination of:

  • Time
  • Voting
  • Reputation (i.e. trust)
  • User History (i.e. tracking tendency for a user to link to the same domain)

I'm not pretending to know the criteria, or even if there is a criteria. But agree with the principle of your request, though.

6
  • Is there a timeline for implementing any of this, or even discussing it further @ SO? Oct 31, 2010 at 0:44
  • 12
    Robert now that you became "powerful" any chance you can convince Jeff that this is a good idea? =) Oct 31, 2010 at 6:16
  • 1
    @kop turns out, it was every bit the bad idea I originally thought it was. And now we can prove it. I gotta say, goo.gl/c0D6Y Apr 17, 2011 at 1:59
  • 4
    I didn't see anything spammy on thse links you posted. Have you contacted anybody at Google (Matt Cutts?) to see if they can give you an idea of what happened?
    – Gabe
    Apr 17, 2011 at 2:46
  • 3
    This would make for a nice blog post now
    – Pekka
    Apr 17, 2011 at 6:43
  • 6
    For anyone circling back on this, it turns out it was not a bad idea and was eventually implemented by StackOverflow. Thanks for doing so by the way! (meta.stackexchange.com/questions/111279/…) Dec 17, 2011 at 21:55
17

In addition to Robert's excellent points, I want to point out that StackOverflow's reputation system was built with Google's Page Rank in mind - your value is based on what others say about you.

Taking out nofollow will result in additional spam, but I believe that SO's userbase and community moderators are vigilant enough (and numerous enough!) that such spam will be dealt with automatically and easily.

Not only will Stackoverflow be giving to those sites that provide quality programming information, but Stackoverflow itself will benefit to a small degree as quality outbound links do affect one's own pagerank. Pages with no outbound links are leaves on the graph and are actually less connected to the internet and may not be ranked as highly as those that are tightly integrated with both good incoming and good outgoing links.

The nofollow attribute, correctly applied, is a useful tool, but I believe it's not being used correctly here, and the only reason Stackoverflow is abusing it this way is to become a smaller target for spam. The cost is much greater than the benefit, though, so let's either get rid of it, or apply it selectively.

If getting rid of nofollow entirely is out of the question, perhaps rules similar to the following might be implemented:

  • Any question or answer with a vote tally of 3 or more should certainly have it removed.
  • Question edits which involve link changes should probably reset that and accrue 3 more upvotes prior to being nofollowed again (ie, no one can wiki edit 100 answers that have met the other criteria and get a sudden benefit)
  • Users with X reputation have a lower bar (ie, only one or two upvotes required at 10k and 3k reputation to have answers followed)
3
  • These rules would be a great starting point!
    – webjunkie
    Aug 9, 2010 at 20:47
  • @shog9 #1 and #2 are kind of daunting from an implementation standpoint; the state tracking there is weird Nov 23, 2010 at 10:55
  • See my update to the accepted answer. This was a dangerous experiment to run. Apr 17, 2011 at 1:48
11

Here's a recent opinion I came across in a blog post, and I'm tending to mostly agree with it. While the author's point is specifically about askubuntu.com, it's a Stack Exchange platform issue:

And, case in point: If anybody here upvotes this answer of mine, I (being the "thief" "borrower") will benefit from the reputation awarded (assuming I eventually reach 2000 reputation at meta.SO and earn a do-follow home page link in my profile), while the supporting link above that my argument is based on receives no benefit, under the current model.

So, is there a way we can work in this feature request? A single "do-follow" link to the user profile really is insufficient, considering most answers leverage and link to outside resources, many of which deserve the benefit of a full link.

9
  • 2
    all your arguments apply to Wikipedia as well, which has a strict nofollow policy. Oct 31, 2010 at 5:42
  • 2
    We just going to follow what Wikipedia does, just cuz? Oct 31, 2010 at 14:39
  • 7
    @Jeff - StackExchange has the means to intelligently decide which links to follow, via the user's reputation and a flagging/moderation system which is actively used by the community.
    – nickf
    Nov 1, 2010 at 10:19
  • 11
    Indeed, perhaps StackExchange can innovate a method that would be good enough for Wikipedia to imitate. Think big! Nov 1, 2010 at 13:16
  • 2
    @chris and spend even more time battling with spammers? no thanks. convince wikipedia to change their policy and we will change ours. Nov 1, 2010 at 16:13
  • 4
    Not for the first time, I wish I could down-vote a blog post. If your goal in answering questions is to promote your own sites, you've got entirely the wrong attitude.
    – Shog9
    Nov 22, 2010 at 23:40
  • @shog9 we now allow this, though the rules are strict and intentionally vague. Mar 29, 2011 at 2:32
  • See my update to the accepted answer. This was a dangerous experiment to run. @shog9 Apr 17, 2011 at 1:49
  • @Jeff: ouch. But, hard data, hard-won. I would hope the effects are transient.
    – Shog9
    Apr 17, 2011 at 1:56
9

Jeff, I'm afraid your answer doesn't hold water. Wikipedia's use of the nofollow link is based on greed and selfishness. Mass use of no follow on all external links is unethical and a complete failure to combat spam. Jimmy Wales uses Wikipedia as a spring board to promote his commercial wiki site called Wikia. And, Wales admits the nofollow attribute does little or nothing to combat spam on Wikipedia:

http://andybeard.eu/296/wordpress-founder-admits-nofollow-didnt-work.html

More discussion about why using nofollow makes you a bad Internet citizen:

http://whatjapanthinks.com/wikipedia-nofollow/

Further details on why Wikipedia is unethical and why no one should do follow link to Wikipedia:

http://andybeard.eu/294/wikipedia-nofollow-plugin-wikidigg.html

Stack Exchange has the tools and resources to do better. More importantly Stack Exchange is supposed to care about being a good Internet citizen. Greed is rule on the Internet, and Stack Exchange is free to look out for itself at the expense of others, but it's against the spirit of the Ubuntu code of conduct and no Ubuntu website sponsored by Canonical should operate as an Internet Black hole.

0
8

Wikipedia itself nofollows.

http://meta.wikimedia.org/wiki/Nofollow

As of 2006, external links on almost all Wikimedia wikis are tagged with rel="nofollow". The sole exception to this was the English Wikipedia, where, as of May 22, 2006, rel="nofollow" was used only on non-article pages (i.e. pages outside the main namespace). See this proposal for the decision to enable nofollow outside articles on the English Wikipedia. In January 20, 2007, [nofollow] was activated in the main namespace at the request of Jimbo Wales.

http://en.wikipedia.org/wiki/Stack_Overflow

View source on the article to see

<a href="http://stackoverflow.com/" class="external free" rel="nofollow">
http://stackoverflow.com/
</a>

Our policy is the same as Wikipedia's. You get a followed link in the "website" field of your user profile at 2000 reputation. Beyond that, everything outside the network is nofollowed as a simple matter of standard policy. Exactly like, and for all the same reasons as, Wikipedia.

UPDATE

We tested this, and the data so far points to a fairly severe organic web search traffic drop on Server Fault as a result. Per my update to Robert's answer, we're abandoning this indefinitely for the sane, simple precedent that Wikipedia set: no followed external links, period.

12
  • 10
    While you're certainly entitled to set whatever policy you like, I was slightly disappointed that you gave a form-letter response describing the policy instead of explaining it
    – Brad Mace
    Oct 31, 2010 at 6:34
  • 2
    @bemace just look up the discussion of why wikipedia does this; we do it for all the same reasons. Oct 31, 2010 at 6:56
  • 3
    @bemace - It doesn't much matter anymore. A lot of SEO experts are indicating that pagerank is being accrued even from nofollow links on some trusted websites, such as wikipedia, twitter, etc. There's still some controversy about whether it's happening or not, but apparently people are started to see results from nofollow link spamming.
    – Pollyanna
    Oct 31, 2010 at 6:59
  • 2
    Ah, didn't realize there was any explanation there, that page is a mess. Beginning of it made me think it was just a link explaining what nofollow does.
    – Brad Mace
    Oct 31, 2010 at 7:04
  • 1
    @Polly you should ask that on webmasters.stackexchange.com Oct 31, 2010 at 7:14
  • 4
    The explanation page linked lists more reasons against nofollow than it does in favour, and the vote shows an overwhelming majority in favour of getting rid of it - it may well be that using nofollow on Stack Overflow is the correct choice, but the link does nothing to explain why.
    – Justin
    Nov 3, 2010 at 3:31
  • People have spammed on Wikipedia, and on other wikis. Have people successfully used SEO on SOFU? Nov 7, 2010 at 4:46
  • 7
    @andrew yes, they have. We fight it every day. It's really tiring. Nov 22, 2010 at 23:09
  • 4
    I can not believe the hate over this. If nofollow helps SE better able to provide the great service they do, I'm happy they do it. Dec 20, 2010 at 6:06
  • @Andrew'saUnitato, It's not about hate. It's simply about doing what's right.
    – Pacerier
    May 14, 2014 at 10:04
  • @Pacerier Wow. "Doing what's right"? Taking this a bit took seriously, I think. May 14, 2014 at 11:12
  • @Andrew'saUnitato, You are misinterpreting it. It's simply "be nice": en.wikipedia.org/wiki/Don't_be_evil
    – Pacerier
    May 14, 2014 at 12:45
7

The issue you bring up is very valid and I'm sad that it has been ignored by Stackoverflow so far. I wrote a long essay about the negative impacts of Stackoverflow abusing nofollow here.

What Stackoverflow is doing is nofollow abuse and they could easily avoid this abuse by using the question's age, number of up votes, and/or the reputation of users.

19
  • 8
    This blog post at least contains a reasonable argument: if answers on SO could contribute to, say, MDC overtaking W3schools, then that would be a great thing indeed... However, I dispute the idea that links from a high-rep user should be automatically trusted; we've already seen instances of users with several thousand points trying to work OT links to their own sites into otherwise-reasonable answers - no need to increase the payoff for that sort of rubbish. Pollyanna's suggestion to judge links on a per-answer basis, while perhaps difficult to implement, seems like a safer course of action.
    – Shog9
    Dec 20, 2010 at 6:07
  • 1
    you should really read shog9's comment closely. I'd rather have fair and equitable treatment of unknown links -- they all get nofollowed. Users get a followed link on their profile at 2k rep. If this is something you can't come to terms with, perhaps other internet sites will be more willing to follow your links? Dec 20, 2010 at 6:49
  • 9
    @Jeff, why are you always so quick to usher users that disagree with you away from your network? Brian is a top 50 user, I'm inclined to assume that he is probably someone that you would want to keep around.
    – jjnguy
    Dec 20, 2010 at 7:57
  • 1
    @jjnguy my point is that almost no other sites on the internet behave in the way he's describing, so it's really the internet he's fighting here. The norm is to nofollow all unknown links, and allowing it to be undone per-rep is a perverse (and I would also argue, unfair and inequitable) incentive that would aggravate a problem we already have with users who post excessively promotional content -- see shog9's comment, above. Dec 20, 2010 at 8:13
  • 1
    (all that, IN ADDITION TO opening us up to even more spam problems than the ones we deal with right now every day.) Dec 20, 2010 at 8:18
  • @Jeff, I don't agree with his stance. I see no problems with nofollow. I just don't think that ostracizing highly reputable users is the right thing to do.
    – jjnguy
    Dec 20, 2010 at 8:34
  • 1
    @jjnguy I don't think ostracizing our highly reputable website for something that is a standard, accepted convention on the internet -- something that virtually every other site on the internet, including Wikipedia, has done for years -- is the right thing to do, either. So I guess we have that in common. Dec 20, 2010 at 8:41
  • @Jeff, true. It was not the best idea to single out Stack Overflow the way he did.
    – jjnguy
    Dec 20, 2010 at 8:47
  • 1
    @Shog9: Don't we have a flag option for those dubious posts? Dec 20, 2010 at 13:08
  • 3
    If a link reaches a certain age, the question is not closed, the poster has a high reputation, and the answer has a high amount of upvotes. I think we can agree that the answer is beyond reasonable amount of doubt not spam. If you'd like to treat all unknown the same don't take reputation of the user into consideration and leave the other factors. I love Stackoverflow and have no personal problem with Stackoverflow. I just have a problem with this one aspect of Stackoverflow and hence why I wrote this constructive argument against the evilness which is being implemented as of now. Dec 20, 2010 at 13:26
  • 2
    @Marcel: Yes. It's also possible to edit out offending links, block users, etc. But I'm concerned about something more insidious: gray-area links in reasonable answers from dedicated users looking to piggyback on SO to promote their own blogs or businesses. This has already been a problem at times, and it's difficult to control - turning SO into a full-on SEO tool won't make it better.
    – Shog9
    Dec 20, 2010 at 19:53
  • 3
    @Shog9: People will do that whether there is nofollow or not. nofollow is not the deciding factor of plugging one's site. Simply don't remove nofollow until there are sufficient upvotes on an answer (let's say 3) then you know via manual review it's a real answer. Dec 20, 2010 at 20:12
  • 2
    @Brian: that sounds workable, and stays with the general SO philosophy of "rate the content, not the user".
    – Shog9
    Dec 20, 2010 at 21:10
  • 3
    Right, upvotes on answers rates the content, and the link is no longer "unknown". If it is upvoted it is deemed applicable and a good answer to the question. And not all other sites behave with nofollow abuse, slashdot is a good example. Wikipedia is much different from Stackoverflow in it can't easily determine via upvotes on answers like Stackoverflow can. Dec 21, 2010 at 23:18
  • 2
    For anyone circling back on this, it turns out it was not a bad idea and was eventually implemented by StackOverflow. Thanks for doing so by the way! (meta.stackexchange.com/questions/111279/…) Dec 17, 2011 at 21:53
3

I think the comparison with wikipedia goes astray here. The big difference being: wikipedia doesn't require attribution with a non-nofollow link.

Let's do a thought experiment. There exists Another Stack Overflow, let's call it AS. Now AS is doing everything exactly the same as Stack Overflow. So in user contributions all links have the nofollow attribute, and if somebody else wants to use the content, they require that there is a link back without the nofollow attribute.

Now a certain user called Joel Atwood has place some perfect text about huskin puppies on AS. It is such a good text, that nobody can even come close to such a good explanation how to treat the puppy's furr, so that it is shining like the sun.

Now on Stack Overflow, there is asked a similar question. Jeff Spolsky sees the question and thinks immediately, Joel's answer from AS would be a great fit here, so he dumps a link. Now somebody starts to complain in the comments; "please, just copy the content from that question and attribute it, the license says that it is possible!". Jeff off course agrees, but... he can't. He has no way to put a link without nofollow to attribute the text of Joel.

See the dilemma?

Now the owners of Stack Overflow and Another Stack Overflow should think. Do they really want to block information sharing between these to sites? Do they want to prevent, that their content is reused on wikipedia (which is now legally impossible)?

Of course they have a point in using the nofollow attribute, but if they want that, wouldn't it be reasonable that they remove that requirement from the attribution rules?

Not the answer you're looking for? Browse other questions tagged .