Should I use the Math SE beta or Stack Overflow? Or if Stack Exchange doesn't have an appropriate site, where should I go? I like Stack Exchange, though.
There is now a dedicated Mathematica Q&A site. Any questions about Mathematica should be directed there.
And there's also https://scicomp.stackexchange.com/, for Computational Science. Depending on the nature of your question, that might be the right place. ("For scientists doing science by heavy computations", the FAQ currently says.)
If it's about programming it then go to Stack Overflow, if it's about using it then go to either Super User or the Math Stackexchange.
Assuming your aim is to get a useful answer I would ask about Mathematica on Math SE, as that is the site that is most likely to have users of Mathematica on it. I would then try Math Overflow, as there must be some users of Mathematica on Math Overflow.
I tend to think of each site as a community defined by a common interest, then I think of the community that is most likely to have members that can answer my questions.
Even a problem installing Mathematica, provided the problem is with Mathematica rather than the OS you are much more likely to find someone that has solved the problem before on Math SE then super User