When an answer has been deleted the information on unanswered question tabs still says that users modification was the last to the thread. For instance on Stack Overflow right now the top two Unanswered questions are listed as having been modified by zerkms.
While waiting for a moderator Zerkms, modified several answers in an effort to remove spam. However, once these answers were removed Zerkms still appears as the last person to modify the thread. While I imagine the hidden/deleted answers still exist somewhere causing this to occur, it can be confusing if you don't know why Zerkms is listed, as his modification is found nowhere within the linked question.
I would propose that when an answer / modification is deleted / rolled back, that the associated user is removed as the last modifier.