@Oded: Concerning "There is not universally accepted form of markup to caption an image in HTML" and "HTML doesn't support it natively": What about the
The markdown code
could be rendered to
<img src="image.png" alt="Caption text">
and result in relatively clean and straight-forward HTML code IMHO.
Perhaps the problem of missing standardization remains. As long as the W3C does not standardize it, there will be several different definitions of Markdown that exist and develop in parallel. One MD could implement it, the other not. But is it really a problem?
As Jeremy Banks pointed out in a comment, Stack Exchange for sure could implement this. But I would see more advantages for image captions in writing (scientific) articles than on Stack Exchange. For example in pandoc.