The link to an answer doesn't have your user ID in it for tracking like the one from the question does, so it can't be tracked. Should it be possible? maybe - but the links would be even longer than they are now.
For example (for me - being user 135201
on meta), this question link is:
http://meta.stackexchange.com/q/71474/135201
And the link to this answer:
http://meta.stackexchange.com/questions/71474/should-it-be-possible-to-earn-the-publicist-badge-by-linking-to-an-answer-of-a-qu/71479#71479
Currently you see they have very different formats...so really it's also a discussion about changing the overall link format, which would be required to support awarding the badge for answer links.
Doing it yourself, it would look like this:
http://meta.stackexchange.com/questions/71474/135201#71479
Format: http://site/questions/QuestionID/UserID#AnswerID
This is just a question link, but it scrolls down to the question with the hash just the same...so it's not tracked as an answer hit (if this matters), but will have the desired results: a tracked link to the question while still scrolling to the answer.