This is a follow-up to my comment on meta SO.
I'm going to start on a chemical plant analogy. To achieve mass production with a new process, one goes through several scaling steps. At each step, the amount of the product increases (simplifying things a little) tenfold: lab, bench, pilot plant, full-scale.
Stack Exchange developers have internal review processes. Number of critics (N) =1. Let's be optimistic and say N=10. Even then, all the critics are biased (that's the consequence of working inside the system).
After passing internal review, SE devs post on meta SO/meta SE and get N=1000 or N=10000 at once:
- Introducing the Developer Story (with later clarifications in The Developer Story Part 2: We didn't explain that very well )
- The MIT License – Clarity on Using Code on Stack Overflow and Stack Exchange
- The Power of Teams: A Proposed Expansion of Stack Overflow
...and, to crown it all...
This amounts to full-scale responses, and let's be clear - those responses are frank and not really flattering. This frankness provokes a defensive reaction from the devs/CMs, who did not see it coming. What's more, Stack Exchange undermines its reputation by untested projects.
What is missing: an intermediate N=100 feedback phase. An opinion survey that doesn't go full Meta. The exact implementation is unclear at the moment. Could be an e-mail survey for moderators (since they have already signed an NDA).