I hooked up StackOverflow to Chime and it gave me the badge, presumably because Chime visited the site invisibly for me. Can this badge be removed from my profile somehow? I don't want it unless I've earned it.
Nope; you have it now. Removing the badge wouldn't do much good, since you still have 30 consecutive access days, so the badge would be automatically reawarded.
You're right though, this shouldn't really count, and it won't in the future (so you won't get the Fanatic badge this way). But the Enthusiast badge you get to keep. After all you were enthusiastic enough to make us aware of this :)