Currently, on any chat message, there is the "reply" button that automatically inserts the required markup to reply to a certain message. This includes messages from deleted users.
However, if you use the button to insert the reply code on a message that was posted by a user with a deleted chat profile, and then post your message, the message isn't actually a reply: it just shows the code number at the beginning. It also doesn't highlight the message it was trying to reply to on mouseover, as most normal uses of the reply button would.
Since you can't reply to messages from deleted users, one way to fix this bug is to stop showing the reply button on messages from deleted users. Another way is to allow replying, showing the message as a reply, but not actually attempt to ping the deleted user.
Update: I ran into another instance of the same thing just today. I attempted to reply to a message from a user who had just deleted their account, but my chat page hadn't refreshed to show their account deleted. I attempted to reply to it, and it showed the old username in the message initially but with no reply markup. I refreshed the page, and it showed the code number instead of the old username.