The scoring system can be found here.
Is there any mathematical grounding to this scoring scheme or are these numbers just based on intuition mostly?
There is not a real mathematical grounding. As a mathematician, I don't even know what that would exactly mean. However, there are some reasons why some values are the values they are compared to the amount of reputation one needs to get to receive privileges.
All the reputation numbers, as you can see by examining the various blog posts and meta discussions about them, were derived by the rigorous and precise process of picking numbers that seemed vaguely plausible and iterating until no blatant systemic problems remained evident. It is difficult indeed to imagine a more mathematical grounding than this, surely!
For example, initially upvotes on questions gave +10 rep, but later this was halved, because it was encouraging the mere asking of questions beyond their real worth. (The real worth of a good question, as a moment's thought will immediately reveal, is of course precisely half that of a good answer, once normalized for the highly uniform voting patterns all sites have been shown to share.)
In similar fashion, closing questions is very important early in a beta, so the ability to vote to close is gained at 500 rep, while freely editing is not so crucial, so it is gained at 1000 rep. In a graduated site, though, closing is less crucial (an observation well backed up by the backed-up close vote queues on most mature sites), so it requires 3000 rep, while editing requires only 2000. So we see that closing questions on mature sites requires 50% more trustworthiness than editing, rather than half as much, as on a beta site.
Finally, you will note that most privilege thresholds are situated on nice round numbers like 5000, 2500, and so forth. This is not because those were convenient to think of, but because the peculiar dynamics of Stack Exchange work in a highly specific but remarkably stable way to make those exact numbers optimal within very narrow tolerances.