Today I joined the long list of contributors who came here wondering why their answers had been inexplicably posted as comments. I'm not going to bother arguing that this feature belongs on the scrap heap‡, but I'd like to at least suggest a few changes.
Tell people about it: I would imagine the first reaction of most people caught out by this feature is to assume that it's the result of a bug. There is no mention of this behaviour in the "about" page or even on the help page, where you would at least expect a search for "answer posted as comment" to produce a screen full of helpful information (it doesn't). Don't wait for people to get caught out before offering explanations in /meta. That's just inconsiderate.
If possible, I'd like to see a description of the heuristics that you use to decide when an answer needs to be zapped in this way. Then at least I'd know how to avoid this situation in future.
Provide adequate feedback: Would it be possible to provide people with some sort of warning or notification while they are still editing their answers? If the system is about to do something strange and unexpected, then there needs to be a prominent warning presented to the user. Perhaps a warning icon in the submit button, or a modal dialog that appears when the form is submitted.
(You already have a length indicator in the comment section, with a helpful message to the effect that answers containing fewer than 15 characters cannot be accepted. Well done there.)
Don't mess up my links: In an answer, hyperlinks are included using this format
Hyperlinks in comments, on the other hand, are formatted like this:
As I understand it, one of the criteria for selecting answers to transfer to the comments section is the presence of a single link plus a short amount of additional text. So why, after all this time, has nobody yet bothered to find a way of fixing the link formatting so that they are still rendered correctly in the comments section?
Give people a chance to fix their submissions: If someone inadvertently submits an answer before completing it, surely it would be nicer to give them the opportunity to put things right before you mess up what they were doing? I would much rather see a message saying "Sorry, you can't submit that" instead of having to go fixing the mess you made of my answer in the comments section.
‡ It really does, though