I am a regular reader of tools/flags. A common category in there is a non-answer with, not surprisingly, a 'not an answer' flag.
Now, I have two choices. I can use one of my very small collection of delete votes, or I can just add another 'not an answer' flag. 'small-collection' -- five delete votes barely scratches the surface of a typical big day on the review page.
I usually do the later, and conserve delete votes for bad questions. There's no flag for questions that means 'hey moderator, I think that someone should dispose of this, just not me.'
Would diamonds rather I did my tiny bit of voting to delete a few bad answers per 24-hours, or are they happen enough to get my additional ring on the bat-phone calling for them to delete?
I know that some havoc was wreaked early on with too many deletes, but perhaps 20K users or some other threshold should be given more latitude on answers with one or more not-an-answer flags?
To expand based on some comments: Many non-answers should be comments. However, many of them are (a) ancient and (b) contain no content worth preserving. So, while one path here would be to allow sufficient-rep users access to 'convert to comment', I think that deletion is still a question of interest.