What is the reason behind marking forbidden pages as 404?

Question

This came to my mind after my previous question. Actually, most of the 404 pages I stumble into on SE are forbidden (403, eh?), not that they can't be found. Examples of such pages would be trying to access /admin despite not being a mod, or trying to access deleted chatroom, or somebody's inbox or profile editor, etc. Why do they throw 404s instead of 403s or something similar?

Answer 1

"What is the reason behind marking forbidden pages as 404?"

Security through obscurity.

See Obscurity is a Valid Security Layer for nice explanation (as well as the citation below).

---

Source When Should You Return 404 Instead of 403 HTTP Status Code? by Max McCarty.

When talking about web application security, one common denominator that repeatedly comes up is the act of disclosing sensitive application data.

...

In my last post on Security Misconfiguration, part of the discussion was on properly handling error messages to ensure we don’t expose sensitive data to our users. But it was obvious that in certain circumstances we can inadvertently disclose information that a malicious user could use to their advantage just by returning the real HTTP status code. One of those situations is when the resource is forbidden (403).

An authorized user has requested a forbidden resource in which they receive a HTTP 403 forbidden response to the request is a common scenario. However, by returning the applicable and valid 403, we have also made it clear that the resource does exist. The disclosure of the resource might only provide a piece of the profile puzzle a malicious user was assembling or it might actually directly provide the user opportunities. For example, a valid case could be that a 3rd party web resource (.axd) had a known security flaw and could be taken advantage of through other side channels when knowing the resource exists.

Though, security through obscurity is never in of itself reliable security, it can be and should be used as part of any security in-depth approach. We can leverage this approach when we determine that it would be better to not disclose the existence of a resource, but return 404 instead of 403 HTTP status code. Unfortunately, if you have ever attempting to do so, you will have found it less than possible and far from easy. Maybe you’ve even given up on the attempt.

...

Answer 2

Both existing answers say that this is a valid use of security through obscurity. Unfortunately, both miss the point of the original post: anyone attacking SE can easily determine with high confidence that all these 404s are in fact actually 403s in the same way the OP did … or just by reading this or similar posts, which are public. It would be silly indeed to suppose that someone attacking a highly visible site like SE would fail to dig up all the information (public and otherwise) that they can. And supposing that the designers of malware will stoutly cling to some standards-compliant, fully-generic attack code that has no hacky customizations for particular targets would be madness.

Therefore, this cannot be a valid security policy. It means nothing at all; it's just pointless obfuscation for the sake of it, or perhaps carelessness in setting up conformant response codes. And since it does make any legitimate generic clients that are standards-compliant a little bit less aware of the true situation, it's a bad idea, just like using tables for layout.

Breaking standards in a predictable, publicly-documented way to ensure security will only sacrifice standards-compliance to achieve no additional security at all.

Answer 3

This is an acceptable practice used by many major websites. For example, GitHub returns a 404 when trying to access a private repository that you don't have access to. Facebook will also return a 404 when attempting to access the profile of an individual that you aren't authorized to see. The intent is that you shouldn't be able to learn about the existence of a resource unless you have authorization to view it.

This doesn't break standards either. RFC 2616 states that for a 403 Forbidden status code, "If the server does not wish to make this information available to the client, the status code 404 (Not Found) can be used instead."

Answer 4

Why not have a scenario? If you are a guard guarding something secret, would you say "Hey, it exists, but you can't get it!" or "Nope, dunno what that is" when someone wants it? Obviously the second! Same here. If you say the first, the person now knows the existence of the thing. The second does not let you know. So if we try to access something confidential about a user, giving 403 will give the existence of it away, and hackers know where to find it. For most of us, 404 means not there! So we don't know if it really exists or its just a lie to protect user data.