HNQ RSS feed stopped updating

Question

https://stackexchange.com/feeds/questions hasn't been updated for 12 hours and https://stackexchange.com/feeds/questions for two days. Both protocols return different content, but neither are up to date.

However, when logged in, both versions give the same content, which looks like the correct one. The <updated> field in <feed> is just minutes ago.

Diff of response headers for https between logged out and logged in (http has similar values):

Header           Logged out                      Logged in
-------------------------------------------------------------------------------------------
Cache-Control    public, max-age=63626889542     public, no-cache="Set-Cookie", max-age=120
Content-Length   37765                           36807
Expires          Tue, 04 Apr 2017 07:59:02 GMT   Wed, 05 Apr 2017 19:15:55 GMT
Last-Modified    Tue, 04 Apr 2017 07:57:02 GMT   Wed, 05 Apr 2017 19:13:55 GMT
Age              126879                          (none)
X-Served-By      cache-yyz8335-YYZ               cache-yyz8332-YYZ
X-Cache          HIT                             MISS
X-Cache-Hits     4                               0
X-Timer          S1491419523.730773,VS0,VE1      S1491419635.717309,VS0,VE102

Edit: The plot thickens: I get the same thing on my phone on data and on two machines on my network (two different ISPs), but both a friend who lives in the same city and Cai in the comments below get an up-to-date version. I have no idea what's going on.

stackexchange.com resolves to 151.101.193.69 on both ISPs, but friend's gives 151.101.65.69. We both flushed the DNS cache and now we're both getting 151.101.1.69, but it's still giving me an outdated version.

Going through a VPN fixes all the issues, which is not that surprising, but turning my VPN off and doing a ctrl-f5 gives me back the outdated version. And like I said above, sending my session cookie always gives me the correct, up-to-date version:

$ curl -I -b "cookie string copied from browser"  https://stackexchange.com/feeds/questions
HTTP/1.1 200 OK
Cache-Control: public, no-cache="Set-Cookie", max-age=120
Content-Length: 36558
Content-Type: application/atom+xml; charset=utf-8
Expires: Thu, 06 Apr 2017 18:50:22 GMT
Last-Modified: Thu, 06 Apr 2017 18:48:22 GMT          <----- Note
X-Frame-Options: SAMEORIGIN
Set-Cookie: (redacted)
X-Request-Guid: c9482245-8d6b-4de5-a8de-1e25205e54c9
Accept-Ranges: bytes
Date: Thu, 06 Apr 2017 18:48:22 GMT
Via: 1.1 varnish
Connection: keep-alive
X-Served-By: cache-yyz8330-YYZ
X-Cache: MISS
X-Cache-Hits: 0
X-Timer: S1491504502.382361,VS0,VE51
Vary: *,Fastly-SSL
X-DNS-Prefetch-Control: off

$ curl -I https://stackexchange.com/feeds/questions
HTTP/1.1 200 OK
Cache-Control: public, max-age=63626889542
Content-Type: application/atom+xml; charset=utf-8
Expires: Tue, 04 Apr 2017 07:59:02 GMT
Last-Modified: Tue, 04 Apr 2017 07:57:02 GMT          <----- Note
X-Frame-Options: SAMEORIGIN
X-Request-Guid: 2087959f-edc8-4b74-a078-8bcb874b4337
Content-Length: 37765
Accept-Ranges: bytes
Date: Thu, 06 Apr 2017 18:48:37 GMT
Via: 1.1 varnish
Age: 211874
Connection: keep-alive
X-Served-By: cache-yyz8335-YYZ
X-Cache: HIT
X-Cache-Hits: 7
X-Timer: S1491504518.899367,VS0,VE0
Vary: *,Fastly-SSL
X-DNS-Prefetch-Control: off
Set-Cookie: (redacted)

Answer 1

First, for all website admins using sites with output caching:

If you use [OutputCache], DO NOT UPGRADE TO .NET 4.6.2

---

Details:

Unfortunately, this was far larger than your feed and we're still assessing the damage. There is a critical bug in the 4.6.2 release of the .NET Framework which causes cache poisoning in a severe fashion. Though the cause is a simple errant setter, the impact is tremendous. Here's the fix in .NET 4.7 (not available for our Windows Server edition yet): _{(You're looking at HttpCachePolicy.cs, here's the link for .NET 4.7)}

The manifestation of this bug is an errant setting _utcTimestampRequest to 0, and amounts to running the following to calculate the max-age:

0 - (DateTime.MinValue - DateTime.UtcNow).TotalSeconds

Which is 63,627,196,173 seconds, or about 2017 years (since it's the diff to DateTime.MinValue or 0001-01-01 00:00:00Z in .NET).

This means browsers and proxies are instructed to cache content not for the 15 to 180 seconds we instructed, but 2 millennia instead. This is a huge, huge bug that needs a hotfix for all customers ASAP. I am stressing that to Microsoft as strongly as we can.

While we are rather agile and will deploy a private .NET 4.7 build if we can arrange one, most people aren't so lucky and often have to wait many months before they can update their releases, whereas a minor 4.6.x release can happen quickly. Your only options (currently) are to roll forward, roll back, or ditch output caching. We've currently opted for #3.

We have disabled all output cache directives at Fastly and stripped the rest in code. The only exception to this is user flair (a bit heavier on bytes), which we're manually overriding in Fastly to the 180 seconds it should be.

I will continue to work with Microsoft on a resolution into the evening and update this as I am able.