The problem/Motivation
Asking
I will only do so if I think nobody except the person facing my exact challenge will ever see my answer (i.e. questions with hardly any up-voting or even one with a bit of down-voting).
That's interesting.
Why? Well, maybe "Why" is not the right question to ask here, as you answer it shortly afterwards.
But here's my motivation:
Firstly, I participate more on science-y sites. I don't like SO too much, mainly because the questions seem to be of the "fix this for me" type; problem-oriented instead of concept-oriented. I know that having a large quantity of such questions is part of what makes SO successful, and that these questions are widely accepted by the community. I just personally don't like the majority of questions on SO, and as I try not to impost that personal opinion on others (though initially I naively used to use that opinion in meta discussion), IMO that's OK.
I'll address why I answer questions on science sites first (as you did mention SE in general in your question), and then I'll get to SO.
On sites like Physics, Chemistry, Space Exploration, and Astronomy, I answer to learn. When I answer a conceptual question, I have to codify my abstract thoughts on the subject into understandable English. This isn't always easy; concepts can be quite nebulous in my mind without me ever knowing that. Doing this exercise automatically ends up with my own thoughts getting clarified — especially in situations where I didn't know they were murky in the first place! Besides this, I get to learn from comments (especially when I deliberately answer something I'm not sure about and add a disclaimer), as well as "competing" answers. Many a time I find that other people have interesting ways of explaining something, and one can properly appreciate their explanation only after having attempted to explain the same thing oneself.
What about non science sites? Well sites like Information Security (and Robotics, and a couple of others), have at least a partial focus on problem solving, but are quite conceptual too. On Sec.SE I've learned a lot about the topic while answering, partially from comments, partially from other answers, and partially from having to confirm my knowledge (in a topic that I'm nowhere near being an expert at) and learning more things in the process.
What about Stack Overflow? I'll first note that I have 2k rep on SO, so that probably puts me in the "pleb" category. Oh well.
I'm an amateur programmer. I don't program for a job, nor do I intend to. I do participate in open source and write fun projects of my own. But I don't have industry-level expertise with any language.
On SO I usually answer stuff when I'm bored. Do I get anything out of it? Well, many times I make small mistakes (more times than I'm proud of, for sure), and commenters tell me. Sometimes I may write code inefficiently or not use some built in function that makes the code simpler. I've gotten pretty helpful comments in these cases, and I learn.
Of course, you're probably an expert, or at least not prone to leaving answers that may attract such comments as you won't make such mistakes. In that case, I recommend you read this post (the OP, at the time, was not an employee of SE), and the answer on it by the OP. It's a very good read, and it focuses on this same issue — why should an expert answer stuff on this site?
solved the quandary of time travel
FYI: It's already been solved. Travelling to the future with a shorter proper time is possible, and non contradictory. Travelling to the past, using any mechanism (be it a wormhole with a spacelike separation, or a spinning cosmic string, or whatever) is not possible as each of these mechanisms destroys itself by a feedback loop of vacuum fluctuations the moment they become time machines.
(Hey, it's Meta. I can include some physics in my answer, right?)
Answering
I will trawl Google's nether regions before posting a question on SO. I will scour forums that still think blinking text is cool, before asking a question on SO. I have never asked a question on SO. This is only my second on ALL the Exchange sites.
This is good. That's exactly what we would like everyone to do, more or less.
I like to think I do the same (usually). On SO most of my questions have cropped up because Google couldn't help me. On concept-oriented sites, usually this isn't an issue at all because Google isn't as good at fetching conceptual explanations as it is at fetching solutions to specific problems.
But if you don't have any questions on SO because of this, that's great! It reflects on the progress of the Internet, and also on your own expertise :)
Of course, if you find a particularly tough problem that took a lot of Googling and piecing together to solve, then perhaps you may wish to share your eventual solution by self-answering. There's a "Ask and answer your question" checkbox on the ask question page, where you can ask and answer the same question simultaneously. It's a good way to share your solution to a problem you think is relatively hard to find via Google/etc, where the problem is of the type you feel would be at home on SO.
The reasons
Moderation-asking
Exchange moderators are a bit, um, what's the word? Well, the nicest word I can come up with is "complicated", possibly "unpredictable". I have seen other users get flamed for seemingly minor "offenses". You sit on the sidelines going --> o.O <-- and thinking to yourself "Even though I wanted to know the answer to that question, I am so glad I didn't ask it. Thanks for taking the heat there, fellow user". I myself had an answer deleted after 1 down-vote. (Between you and me I think it was because, turns out, the mod was a Java dev and I made a Java joke... Knock, knock... teehee ^_^). I don't mind the deletion, but after only 1 down-vote? And no feedback as to why I'm a complete nincompoop?
If you're talking about SO, I assume you're including high rep users with partial moderation privileges because the moderators form a very small part of the overall moderation and aren't encountered that much.
Firstly, I'm not sure if I'd call it flaming. But you're right, users do feel antagonized when other users try to teach them the rules. Well, the main issue is that SO looks similar to a forum at its outset, but the forum mentality doesn't gel well with the policies here as it was more or less explicitly designed to not be a forum. And this design seems to be working well for the primary goal, "Making the Internet better" (creating a repository of easily found Q&As which are useful to many others)
SO tries to be focused towards the goal. Really focused. So, for example, answers that don't answer the question at hand may get deleted. Questions which are of a type that won't lead to constructive answers or won't really be helpful get closed in a similar vein.
I myself have seen many examples of users getting "flamed" for not following the rules, an I myself may have participated in some of this. I don't know. Ideally, I feel that users should be handheld through this. I've done this on many smaller sites, and it really leads to a more constructive experience for all involved. But I can't think of any way to make the majority community of SO to listen to this. Each person has their own reason for being here, and many are too exasperated to be very nice. It's the Internet, it's hard enough keeping things civil.
Still, as an experienced user, I don't think you should have a problem with this. You've read the rules (yes?) and probably can abide by them. Many times it's the way a question has been written and not the core question itself that others take objection against.
Moderation-answering
So... if I sit down and try and write a good answer (taking hours off my pathetic little existence), 1 of 3 things might happen: a) I get the answer right before everyone else and get some SO points. Hooray; b) I get the answer right but am too late and get no SO points, but maybe some up-votes. Hooray; And of course a) and b) contribute to the greater learnings of mankind;
Meh, I personally don't care much for rep. But I think I've addressed the "motivation" bit above in the "Answering" subsection of the "The problem" section.
c) I get it all befuddled and get nothing, not even feedback to help me develop. (Don't nobody refer me to the FAQ on how to answer well. I've read it!)
I'm a busy old girl and I don't know about anyone else, but there seems to me to be limited to no benefit to contributing to SO.
As for "anyone else", again I refer you to this post :)
Combativeness
SO users, by and large, seem somewhat... "combative". If I ask a question or give an answer I would like to come back to positive and constructive responses. I want to grow not groan inwardly when I check back in. And I know some of you are saying SO isn't responsible for the behaviour of random users, except that from what I've observed it tends to be the more entrenched members (the +10ks or whatever) who are the most inhospitable.
[...]
So... if I sit down and try and write a good answer (taking hours off my pathetic little existence), there is every likelihood that a +10ker is going to come along and say something mean... and make me cry... sob, sob...
Huh, well, I think I addressed this in part in my moderation-asking section (because both of these points you made are pretty similar).
Anyway, the thing is that these folks are exasperated with seeing the same type of issues again and again. And they really shouldn't get combative because of this, they should preferably just not comment, but most people feel like saying something about it.
I've somehow not received such comments myself, though.
There seems to be a very defined and accepted methodology around here and it seems to me divergent thinking is not encouraged.
There is, in a sense. The SE model is one that seems to work, and the policies are made t go with it and reshaped by the community at times.
Divergent thinking is encouraged in a fashion. On the main site, if you post something that doesn't follow the rules, that's definitely not encouraged. That's not divergent thinking, that's divergent behavior. But bringing up these issues on meta and seeking for a way to get them fixed — that's encouraged.
Your summary
Most of this has already been addressed above, though not this bit:
If I am looking for tried and tested and practical, I come to SO. If I am looking for exciting, bleeding-edge, fraught-with-difficulty, new strategies - not SO. A mod probably deleted that answer already. To me, SO is not the cutting edge of development. The Google leads me here. If I'm really looking for creative, innovative solutions I exclude SO from my search to skip all the false positives. Sorry SO :(
That's because this isn't really part of the goal of SO. SO doesn't mind innovation in answers, but its goal is to help create a repository of easy to find solutions to practical problems that people actually face. "What's an innovative way of doing X" doesn't really work towards this goal, I'm afraid.
Though Programming Puzzles & Code Golf Stack Exchange comes close. Not exactly what you wanted, but close.
The actual question
(Reader: If you have reached this point without skipping anything, I present you with an imaginary meta-waffle)
What can the SO members (and "elite" in particular as the "leaders" and content decision-makers) proactively do to create a more friendly SO
Handhold. See a user making a mistake? Politely explain him and help him correct it. Sure, close the question if you wish, but leave a nice comment explaining how to fix the question if possible. And not a boilerplate, I'm talking about one tailored to that specific problem.
And be nice. If you want to say something derisive to an answerer? Don't say it at all.
that encourages the development of developers and ideas (horribly scary, fabulously subversive ideas that don't even have docblocks) and not just the development of SO points. From what I understand that is the point of the site (the developing developers bit, not the points bit).
As mentioned before, that's not exactly the point. True, I'd love a more conceptual Stack Overflow (Programmers is more conceptual, but not in the same way), but that's not the reason SO was made.
Note that not much can be done to solve the problem at hand given your constraints. It's virtually impossible to convince the majority of a large body of people to change their habits, which you seem to have touched on. New features and policies seem to help, though. The recent changes to the close system have lead to some implicit handholding from the system, which partially solves things.
Really, it boils down to "How does one convince a large body of users to behave differently without modifying the environment", for which the answer is "you don't". The behavior of such large bodies is controlled by the environment, and IMO modifying that (eg new features, etc) is the best way to go. Unless you can get a smaller (but at the same scale) body of people to behave in the new way. In that case, people may start changing their ways. But the SO community is so large that I'm not sure how this could be done. Other sites with similar problems may find it easier, though.
Why is Stack Overflow - and Stack Exchange in general - so scary (and filled with crazy people)?
Because the technical sites are spawned from the culture that gave us THIS