I've been quite vocal about the fact that we're overhauling post blocks, and dropped some hints as to how we're contemplating improving them. The last few weeks of my professional existence has gone into analyzing what we don't like about them, re-visiting what we hoped to accomplish using them, and coming up with a system that better supports those goals.
What we have was good, at the scale that it was introduced.
Our post blocks need to catch up with the times. Let's reiterate briefly, here's what we don't like:
- They don't slow you down fast enough at the point where you really need them to. You need to fall on your face a few times.
- Once tripped, due to the scale of voting that we see today, they can be practically impossible to escape. Some questions simply can't be improved and subsequently up-voted because you fell on your face in wildly off-topic ways a few times.
- It's easier to delete your account than follow the advice that we give you.
The third item in particular led to a big problem with recidivism, or folks that just entered this maddeningly myopic and dysfunctional cycle of throwing themselves at a wall until we blocked them, then rinsed and repeated again. We put a stop to it mostly, but that's as much of a stop-gap as it is a test to see how effective increased rate limiting would be. The jury is still out on how effective it has been, we're optimistic, needs more time.
Now, onto what we really wanted to accomplish - that's not difficult to state. We want to ensure that Stack Overflow maintains the level of quality that made it the phenomenal success that it is. Tough love, as it were, seemed the easiest and most logical way to make those that can ask better questions do so, and those that simply can't to go elsewhere.
We over-simplified the problem.
Yet, something about the way we block has always bothered me, and I couldn't quite put my finger on it until recently. One of our very public secret agendas has been to trick programmers into becoming better communicators through better writing. By blocking as we do, we're stopping folks that have this potential from pursuing it and prospering. Stack Overflow tends to make people better at more things than programming, if you tough it out.
We need a system that repels help vampires, while helping inexperienced programmers - the folks that can be helped - ask better questions without discouraging them to the point that they see account deletion as the path of least resistance.
Let's bolt some airbags on the front of the question block mechanisms to slow people's acceleration and the force with which they hit them. To do this, we need to look at how well you ask questions overall, and how well you've asked questions recently. The first question doesn't count, because we're programmers and enjoy testing gravity with our faces and some pavement from time to time.
This is where it gets ... tricky, and I'd love some input.
In a single sentence, a question block is the system preventing you from asking questions based on your history of asking questions, until you manage to improve your questions sufficiently to earn some up-votes, or provide quite a few up-voted answers. Basically "You seem to suck, show us that you don't and we'll let you ask again".
Imagine seeing this after asking two questions that didn't go over so well:
The experience you had with your last two questions wasn't what we hoped it would be. Why not take some time and browse other questions tagged (tags) that have been well received? You can then come back tomorrow and try again.
Grr, that stupid site! you grumble as you search around some more, and come back the next day to try again. This time, you do slightly better and manage to not get down-voted, and possibly up-voted. If you do that, we stay out of your way as consistently as you ask questions that don't make people scream.
If you don't - then you get one question every few days, one question per week, three per month and then ultimately:
Sorry, we're no longer accepting questions from this account
To be clear, this doesn't allow more low quality questions in, keep in mind, folks are heavily rate-limited before that particular side of it actually kicks in.
Basically, the system trains on how we'll you've done overall (either your entire account history, or the last 45 days for newer accounts, dropping the most negatively scored), and how well you've done recently (the last 15 days). That gets us the following:
- Number of questions asked in the last 45 days
- Average score of your questions in that time
- Number of questions asked in the last 15 days
- Average score of your questions in that time
- Average time elapsed between questions in the last 45 and 15 days
People that ask good questions also tend to ask fewer questions, so what we're looking at is pretty simple:
- Does user ask good questions? Nothing more to do
Does user have a history of bad questions?
- Do they seem to be improving based on what we can see? Let them ask more questions, with limits commensurate with recent quality contributed
- Are they not improving? Limit them, in a manner commensurate with the way they're being received.
Sounds easy, huh? Not quite.
Now, I've got numbers and such in mind, and much like the existing quality blocks, we won't be revealing the exact mechanics behind how rate limiting works or it just turns into a case of carrots on sticks.
However, a chief goal is to not put off someone that could actually be a good contributor to the point that they just get frustrated and quit, while souring the metaphorical milk enough to make the help vamps go somewhere else.
Those that stay, and prosper, have done so because they've treated questions as a resource that is not infinite, and made them count when asking.
How would you slide these scales in a manner not likely to put off potentially good contributors, but annoyingly enough to drive the vamps to the hills? What pitfalls do you see with such a system? What else should we be looking at?
The last thing to keep in mind is other work going on to raise the perceived quality of most new questions, so this is one of many moving pieces. Unlike the current block, this doesn't essentially 'make or break' the quality of incoming questions. Oh, and - this applies only to sites that have 'big city' problems that come with larger scale.
Stack Exchange Quality Improvement Project
- Allow users to optionally filter out low-quality questions
- Feedback requested: New “recommended” homepage, phase 1
- What should the system be deleting automatically that it already isn't?
- Let's have an explicit triage system for questions from new users
- Breaking down question blocks - let's talk about rate limits