The StackExchange.com global profile still uses Gravatar with the email address from your primary account, even if that account isn't using Gravatar itself.
For example, my global profile is synced with my Stack Overflow acount, yet it displays my Gravatar:
Even though my Stack Overflow profile is using an avatar that was directly uploaded:
The correct avatar is used in the global flair.
The global flair also displays it correctly, which though the my site-specific flair does not: