On most sites, deleted comments are primarily just junk and expired content, most of which becomes too worthless to even bother tracking. There's also the common practice of using comment deletion to make a post-5-minute edit, which means a lot of mostly duplicate comments are just hiding in the seams of the site. I'm aware that Skeptics has a bit of a unique issue with how comments tend to run.
The other thing is that deleted comments would produce a lot of noise were they visible. The ability to see them, currently, could only be used for one thing by moderators - review. We cannot revise, reverse, or otherwise interact with these things in any meaningful way like we can do with deleted posts. They thus stand in our face as if actionable, when there is nothing that can be done. And in the vast majority of scenarios, there isn't anything that should be done. The fact it's deleted is the signal that it already has been done.
Contrast this with our visibility to deleted posts. We don't see them in the questions list, but we can view any profile page to see all deleted posts of a specific user. Yet even though this is automatic, it doesn't produce that much noise (aside from different "total questions and answers"). Part of it is because their context is more like a itemized data catalogue - you're searching through a list of items and they just comprise additional elements in that list. The other part is that there is no hint of importance or need to take action - the highlight just tells you the status without meaning to draw special attention.
Deleted comments primarily sit in the flow of a question. There is a lot of data that is parsed, and it is not in the same fashion as a catalogue. They interrupt the flow of that data, especially in the case of noise and junk (the majority of them, and the reason why we flag and delete most in the first place). Even if it were an "opt in" method as your closing paragraph suggests, that instead places a "highlight of importance". Like the number of spam flags, it implies that the presence of deleted comments means one should investigate or even monitor these, especially when the number changes. But there isn't anything to investigate or even do to them.
I'd be more of a fan if we had a reference utility, like with migrated posts. Something that can be looked at when necessary, and is otherwise outside our normal daily routine. Due to the nature of how comments work, I'm not sure how it should be constructed in a way that would be effective. Review of specific incidents is the main reason we would ever need to see deleted comments. Outside of such a utility, I feel that visibility of deleted comments would simply produce far more noise than signal to moderators. There are some incidents in which it'd be a super big help, but the vast majority of deleted comments serve no purpose to ever be reviewed. It feels like it would interfere with our job.
This has now been implemented
Moderators can now view any deleted comments on a post, via an option on the post's menu that aptly says "show deleted comments". It is only usable if there exist deleted comments. I'm happy enough with this implementation, as it falls in line with the points I made about their visibility.
