In my opinion as a moderately new user (relinquished lurker status and started contributing in November 2013), there's no problem here. I started on Cross Validated (CV) and didn't really branch out from there for about a month. I was never under the impression that CV was a "general-purpose QA", nor would I have been under that impression on Stack Overflow. IMO, it's plain as day what these sites are meant to focus on primarily.
Edge cases of barely-on-topic questions are harder to navigate, but it doesn't take much searching to find help pages and meta-posts that address the basic issue of what is on-topic and what isn't. For the less clear cases, one can ask a meta-question, or simply take a trial-and-error approach; it's not like you're going to get permabanned just for pushing the boundaries with a few off-topic questions.
In my experience, branching out from CV to other sites in the Stack Exchange (SE) network has been enriching, so in contrast to your suggestion, I recommend it. I wouldn't say you "have to" follow your questions if you spread them around various sites...it's all voluntary here. I don't find it confusing at all to switch sites, personally...I would expect most users to prevent their own problems by not spreading their activity more widely than they can manage, or by effectively disowning their posts via neglect. There's not much point in checking on old questions or answers on sites of tangential interest anyway, until you receive a notification of some kind about activity on those posts.
Your inbox and recent achievements on the top bar are global with respect to the SE network: they'll notify you of comments, answers, edits, upvotes, and badges on any SE site no matter which one you're browsing at a given moment. When something happens on an old post I've mostly forgotten about on some site I don't use regularly, be it an upvote, comment, or answer (usually – most badges and edits tend not to come as late in the life of a post), I take that as a cue to take a glance at the post. Again, this is a voluntary expression of personal curiosity; there's really no maintenance required.
I also glance infrequently at the list of my accounts on my profile and ponder how I've spread my attention and whether I should redistribute it at all, but I mostly stick to sites of most direct professional interest. I tend to see this behavior in other users as well: most seem to have one or two sites of primary activity, a few others of secondary/tertiary interest, and maybe several other accounts on sites of passing interest. Just for the sake of window-shopping, I've created accounts on most sites I could see myself having some interest in, and posted a few upvotes on cool questions, but otherwise left them alone.
Those extra accounts aren't dragging my experience down in any way – I'm glad I have them. Spreading my primary activity across CV and Cognitive Sciences, while trying to get a grip on a few other sites is enriching for me – makes me feel like some kind of polymath Renaissance man. I see no harm in having extra accounts and spreading my activity across sites. If I ever feel different, I'll delete an account or two and be done with it. In the meantime, I try to think of special questions for specific sites I don't visit regularly. I don't get around to posting them usually, but maybe someday...