This issue has come up a few times in the past (here and on other sites, mostly progse) when Jeff unilaterally used his moderator powers against the community despite not being an employee or an elected moderator. Here's an example where Jeff just nuked a tag without allowing the community to cleanup the questions in the tag. From my comment discussion with him re: his role at SE and the reasons for his having a diamond, it became clear that Jeff will keep the diamond forever:
I thought about this and here is my answer. As co-founder and primary architect/designer of the Stack Exchange engine design, I will be an honorary all-site moderator essentially forever. As a moderator I may occasionally take moderator actions just like any other moderator, such as removing a tag that was already on the Tag Cleanup list that I feel is especially egregiously meta. [...]
So, there are a couple of ways a non-employee may have access to PII: They can be someone we've asked to do something for us who has signed a contractor's agreement that includes protection of PII, or they can be a moderator, (which, for clarity, is defined as anyone who's signed the mod agreement & has a diamond next to their user name). Currently, Jeff is a contractor.
I'm satisfied with that. I'd rather they be upfront about it and show us visibly (with a diamond) that he has access to PII, than hide it by moving him to a class of users that can access your PII without a diamond.
An update from Jaydles:
Jeff is not currently a contractor, but is a moderator. From a privacy perspective, the key defining aspect of being a (non-employee) moderator is that you've signed the mod agreement, which binds you to protect PII, etc. Jeff's signed it.