The activity overview currently links to specific comments, for comments on answers. The scrolling relies on
#comment-xx URL fragments, and fails if the comment is initially hidden, bringing one to the top of the page instead.
So, I could either file a bug report for the confusing links, or give this feature request some more attention. Bump! :-) (This has meanwhile been reported as a bug.)
Implementing auto-expanding will still make