Originally the idea was for each Stack Exchange site to have its own associated chat site. There was chat.stackoverflow.com, chat.superuser.com, chat.askubuntu.com, chat.webapps.stackexchange.com (there was also chat.nothingtoinstall.com, but that's a whole other story).
But we soon realized that for most sites, the chat sites ended up being pretty sad and lonely places, with just one or two chat rooms each. And so in January 2011 we decided to create a single chat.stackexchange.com instead, with chat rooms for all the different sites. The only exception (other than then-MSO, now-MSE, which is always a special case) was the Stack Overflow chat, which had (and has) enough rooms and activity to warrant having its own site.
Rooms on chat.stackoverflow.com tend to be organized around tags (or related technology groupings), whereas for most other Stack Exchange sites the discussion concentrates in a single general chat rooms for the whole site (imagine such a room for Stack Overflow!).
Chat.SO and chat.SE have similar amounts of activity (on the order of 600k messages per month these days), so if chat.SE were to assimilate chat.SO, half of the mega chat site's activity would be from Stack Overflow, drowning everything else.
So, like so often when the question is "Why is X different about Stack Overflow?", the answer is "Because of volume."