If I decide that I hate hats and I don't want to see them good on me. But what happens if I later learn to stop worrying and love the hats? Do I lose any hats I earned before hating hats? And do I miss any hats I may have earned while hating hats? Or if I hate hats, are you still giving me hats I may not want, but just keeping them all to yourselves?
All that I hate hats does is hide the festivities. You won't see hats and won't be told about new hats you've earned. You won't be listed on the leaderboard and your profile won't show the hats link (even for people that do love hats).
Re-enabling the hats will return all your hats in full glory. Any hats you earned while in denial will also be awarded.