Now that following posts (see also the prior announcement) is possible, I feel like this needs a canonical answer:
Question: What's the difference between "bookmarking" (formerly "favoriting") and "following"?
Thus far, I noticed that following applies to answers (as well as questions), prompts inbox notifications, and is not displayed to other users. There's probably several other aspects I'm unaware of, e.g., badges.