I've just noticed, if somebody deletes one of his Stack Exchange accounts, his username is replaced. That's pretty nice, but comments aren't updated and still reveal the original username "@username message goes here". I think that should be updated, too.
It's important to note that comments are anonymized if the author deletes their account; there's no inconsistency there.
What you're referring to are other authors' comments which may happen to mention the former name of the now-deleted user. There's also no inconsistency in that: the same applies to answers which might happen to mention the name of another answerer, questions which might name another asker, blog posts elsewhere on the 'Net that might mention the name of a person who formerly had an account here, etc.
It shouldn't really be surprising that we don't mass-edit other people's work in response to one user's actions. Comment replies are an instance where, in very limited circumstances, we might have enough context to do this... Albeit not very reliably. In most scenarios, this would be impossible to automate; there are numerous users with identical names, and there's a non-zero chance such automated edits would inadvertently mangle some texts if attempted (for example, there are no restrictions on picking a programming language keyword as your username).
Unless comments are changed to reference other users by ID, this isn't going to happen. For other contexts, it won't ever happen period.
I think there's a misconception here. There is no right to having the history of your data changed on this site.
If you contribute, you make that content available to Stack Exchange. This means that you should choose well what you post and what you don't post, just as if you'd post it publicly.
In selected cases, you might get the network to remove your content, but that's them just being nice, they are not required to do that.
And modifying the content of other people that communicated with you seems wrong.