First, define "game".
Before answering your question, I think its helpful to have a definition of "game". Wolfgang Kramer, designer of many Spiel des Jahre games, writes1:
Games are objects which consist of components and rules and have certain criteria: rules, a goal, always changing course; chance; competition; common experience; equality; freedom; activity; diving into the world of the game; and no impact on reality.
The whole article is well worth reading and as you do, I expect you'll be drawing your own connections between Stack Exchange and a good game. As I see it, the rules include the various behaviors enforced by the system (you can't vote on your own posts, for instance) and the social conventions discussed here on Meta (don't add taglines to posts). Our primary components are questions and their answers, but we also play with lesser components such as comments and chatrooms.
When people criticize Stack Overflow, they regularly find fault with the game aspect of the site. Either the game fails some of Kramer's criteria or gets in the way of the important work of answering questions. My own critique fell under the later tent. A recent review of Stack Overflow notes:
The way [Stack Overflow] is structured rewards people who put as little work as possible across as many simple questions as possible within only the most popular segments. Spending thought (and thus time) on answers interferes with points- and badge-mongering. Answering questions outside of the top ten languages similarly interferes.
In other words, the game fails Kramer's "equality" criteria2 and, thus, isn't a very good game.
"...and no impact on reality."
When people defend Stack Overflow, they invariably point to quality of answers to long-tail questions. If John Carmack is to believed, this "game" has had impact on the outside world to the tune of billions of dollars. If you let Google be your filter and if you don't look too closely at how the sausage is made, Stack Exchange is serious business.
So the remarkable thing about Stack Exchange isn't that it's an addictive and enjoyable game (though under the right circumstances, it can be), but that it's a game that creates a valuable by-product. It would be as if the game Torres was used to plan city development. Therefore, if a rule change increases the value of answers on the site, it will be strongly considered even if it potentially damages the enjoyment of the game for some people3.
Fundamentally, the purpose of the system is not the game itself:
We build libraries of high-quality questions and answers, focused on each community's area of expertise.
The game-like elements of the system are geared to make that process more fun for people who would likely be interested in building that library already.
1. Kramer's article is translated from German, which uses the word [Spiel](http://translate.google.com/#auto/en/spiel) to describe both what in English would be called a game and also less-structured "play". The article makes clear that he's defining "games with rules".
2. Also, likely, "common experience" and "freedom". If you want to "win the game", you probably ought not answer many lua questions.
3. In particular, the game is probably least fun for those who ask poor questions. We get many emails a day from people who have hit the [quality ban](https://meta.stackexchange.com/q/86997/1438), whose questions are closed, or who have been [prevented from asking](https://meta.stackexchange.com/q/56817/1438) in the first place. That said, I think there are plenty of things we can still do to make the game aspect more enjoyable for more people without harming the end product. Our [recent changes to question closing](https://blog.stackoverflow.com/2013/06/the-war-of-the-closes/) were intended, in part, to make the enforcement of rules a little less painful for new players, for instance. It works a little bit like a tutorial level.