New three-tiered badge idea: Explainer →Refiner → Illuminator

Question

The difference between a poor or meh question and a stellar question can often simply be someone understanding it and providing it a great answer. I can't begin to count the number of times I've justified re-opening a question as a moderator by saying:

Look at the answer it got, though. This isn't something we want deleted, this is something we want fixed, because it's obviously valuable. Someone with domain knowledge can easily edit that question based on the answer it received.

This doesn't only apply to borderline-poor or poor questions – sometimes a good edit means the difference between 100 and 10,000 people finding something through typing things into Google. Good titles are hard.

We think we've come up with a way to provide incentive for good, opportunistic edits to happen more often, ideally without questions needing to go through closure or collecting a centimeter of dust before they do. What if we simply rewarded folks that answered and edited questions with a special, relatively difficult to earn badge?

Let's say that you answer something, and:

• You edit the question 12 hours before or after answering it. This allows you to edit now, answer later — or answer now and edit later, when you have the time.

• Your edit isn't rolled back, or outright rejected if it was a suggested edit

• The question is not closed for any reason, even simply being a duplicate

• Your answer has a score of 1 or higher

… then you've done something that we probably want to recognize. You understood something, you provided the knowledge that you have, and then you provided an edit to make sure that more people in need of this knowledge can find it, while raising the overall quality of the site.

Here's the proposed tiers for the badge (within the above stated criteria):

• Explainer (Bronze): Answered & edited 10 questions
• Refiner (Silver): Answered & edited 50 questions
• Illuminator (Gold): Answered & edited 500 questions

We think this is going to be pretty hard to game. You can't go leaving streaks of terse snark all over the place on questions you know are going to be closed anyway, and junk edits aren't going to fly.

We're talking about rewarding folks that take a tiny bit more time out of their day just to write a more descriptive title so that folks can find the awesome answer that they wrote. Just doing that alone can make a big difference. Sure, it's a tad hard to get at the gold level, but it's supposed to be. If you routinely edit questions that you answer and have done so for some time, you'd probably get it the day we roll it out.

While this is technically something that we came up with on the quality initiative (more on MSE | more on MSO) — this badge could be earned on all of our sites.

What do you think? What did we miss? What could be better about the idea?

Answer 1

It's an interesting suggestion, but as per the comments, what this would do is incentivise answerers to always edit every question they answered, with some edit, however minor. And that would encourage more bad behaviour than good behaviour.

So it's a nice idea, but doesn't quite achieve what it sets out to. Perhaps delving a bit more into what is trying to be achieved, will help.

I think the observed phenomenon that has prompted the question is this:

A badly-phrased question can be salvaged by a well-phrased answer.

AIUI, the suggestion is to reward the rephrasing of the question, so that it does justice to the answer.

Symptoms of positive behaviour:

• a question with an answer is closed, edited, then reopened.
• that question goes from being a bad question to a good question; to put that in a way that can be tested for automatically, it would go from having a net negative score to a net positive score.

Should it make any difference whether the edit was done by the answerer or not? I don't think so. Isn't it the quality of the edit that counts?

So, there are a couple of criteria there, that don't appear in the suggestion in the question - the edit that leads to a reopening, and/or leads to the question's net score going from negative to positive.

But what if there are several editors active between closure and reopening, and/or between negative and positive net scores? Who among the editors gets the credit towards the badges? If I see a really good edit that turns a bad question into a good one, am I being incentivised to do an irrelevant minor edit on that question in order to share the credit for turning the question around? (if you're tied to the concept of these badges only applying to people who'd answered the question, then imagine this happening on a question where more than one person had answered it, and the question-editors, substantive and trivial, are among the answerers).

Can the relative substantiveness of edits be established automatically (not easily, I'd wager - it just wouldn't be worth the programming effort). Could there be community voting on the usefulness of individual edits in such cases (again this sounds like more hassle than it's worth)?

Answer 2

I started writing the SQL to determine what posts would be eligible for this badge; it's horrible:

select p.id, p.owneruserid
  from posts p
  join posts q
    on p.parentid = q.id
   and p.owneruserid <> q.owneruserid
  join posthistory ph
    on p.parentid = ph.postid
   and p.owneruserid = ph.userid
   and p.creationdate between dateadd(hour, -12, ph.creationdate)
                          and dateadd(hour,  12, ph.creationdate)
  join votes v
    on p.id = v.postid
   and p.creationdate between dateadd(hour, -12, v.creationdate)
                          and dateadd(hour,  12, v.creationdate)
 where p.posttypeid = 2
   and ph.posthistorytypeid in (4, 5)
   and q.posttypeid = 1
   and q.closeddate is null
   and v.votetypeid in (2, 3)
   and not exists ( select 1 
                      from votes 
                     where postid = p.id
                       and v.votetypeid = 6
                       and v.creationdate <= dateadd(hour, 12, p.creationdate)
                           )
 group by p.id, p.owneruserid
having sum(case when v.votetypeid = 2 then 1 else -1 end) >= 1

This probably means that the conditions are overly complex... as currently described the following badges would be granted:

badges count
------ ---- 
bronze 1764 
silver 280  
gold   6    

This is pretty low, but I guess the badge is to incentivize behaviour which isn't might not be happening currently - low isn't necessarily a bad thing.

However, I also think this disencentivizes positive behaviour. If the question was closed when the badge script was run but was subsequently reopened, for instance, then why should the editor/answerer who may have got the post into this state be penalised? Surely, this is an excellent outcome for both the questioner and future viewers?

In short, I think that the time restrictions actively work against this proposal; as others have said. Removing these time restrictions changes the number of awarded badges to the following:

badges count
------ ---- 
bronze 3500 
silver 830  
gold   24   

I'd also argue that changing the score to 2 or greater would be more beneficial than having time restrictions on the order of hours. SE's currently trying to improve the visibility of potentially high quality questions, which is going to mean that the lower quality questions are theoretically going to disappear even faster. If one gets a good answer and becomes higher quality due to the efforts of the answerer then it may be some time before the answer gets the upvotes necessary. It makes little sense to disincentivize people from improving less visible questions.

Answer 3

A few quick notes, since there's some good discussion buried in comments here:

• The primary goal here is to reward folks for doing something that benefits themselves as well as others, but which isn't immediately obvious. Think of any question you answer as being effectively the introduction to an instructional blog post you're writing: would spend time writing a great explanation only to hide it behind a broken English introduction and a nondescript title? Even in cases where the question wouldn't be closed, by failing to edit you're potentially making it harder for your work to be found - and that's a shame.

• I disagree that tag-only edits should count toward this. Not that tag edits aren't useful - but if you're only touching the tags, you're missing a huge opportunity. Also, our existing editing badges (Editor, Strunk & White, Copy Editor, Excavator and Archaeologist) don't count tag edits, so we should be consistent there.

• You do actually have to write a useful answer. Hence the score requirement on the answer - this is probably the biggest potential for abuse here, and it's entirely possible for the community to mitigate this by downvoting useless answers.

• I don't think additional quality controls for the edits are necessary here. In fact, I don't think we even need to take rollbacks into account - if your edit was approved / applied, and the question it was applied to stays open (and visible - deleted questions shouldn't count either!), then it counts. Anyone abusing this on a regular basis will be exposing both the question and their answers to a fair bit of extra scrutiny; that's probably not something you want to do frivolously. At the end of the day, it's your answer, your work that's on the line here: if you're not making the most of your edits, you're selling yourself short. Checking for rollbacks is expensive, so if we must do so we should limit the check to rollbacks within a few hours of the edit - but if at all possible, that criteria should be discarded entirely.

• The 12-hour window exists to both discourage abuse and encourage good behavior - fixing a question a week later is never bad if you actually fix it, but fixing it while the topic is fresh in your mind is what we're hoping to encourage here. Strictly-speaking, we could count any edit up to 12 hours after the answer is posted and still accomplish this, but "within 12 hours" seems easier to explain.

• By the same token, we do not want to encourage folks to answer and then wait around to see if their answer gets traction before editing - at the point you've posted an answer, you've already invested a fair bit of work into the post; you're not gonna know if it was worthwhile unless people see it... And the best way to make sure the right folks see your answer is to make the question look good.

• I tend to agree that 10 is too high for a bronze badge - it should be 1, to encourage "just in time learning" - the first time you edit a question you've answered, you'll be informed that this is behavior we explicitly encourage! Silver and gold badge levels should be a lot harder.

And to address some of the comments on this answer now... The purpose of these badges is not to encourage higher-quality edits; rather, it is to encourage some of the people best able to make high-quality edits - those who actually purport to understand the questions they're answering - to at least consider doing so on a regular basis. If you want a badge that rewards heroic editing, that's an awesome idea - but the criteria are gonna look very different, so post a separate proposal.

Answer 4

Final requirements:

Edited n questions within 12 hours of posting an answer (that's 12 hours before or after answering), where:

• The question was asked by someone other than the answerer
• Neither the questions nor the answers are deleted
• The questions are not closed
• The answers score > 0
• The question edits changed either body, titles, or both

If n >= 1, an Explainer badge (bronze) is awarded.

If n >= 50, a Refiner badge (silver) is awarded.

If n >= 500, an Illuminator badge (gold) is awarded.

Each badge can be awarded only once per person, per site.

If you're interested in the implementation, this SEDE query is roughly what's being run.

Answer 5

This is quite similar to the Editor (1 edit) / Strunk & White (80 edit) /Copy Editor badges in the sense that all curators will have the copy editor badge, all archivists will have the Strunk & White badge and all keepers will have the Editor badge.

In any case, I would avoid mixing "answerer" and "editor" rewards. Why would an "editor" receive more reward if he/she is also one of the "answerers" of the question? This is not fair for those who did not answer.

As for the "good answer salvage bad question" aspect, there is the gold "reversal" badge for this (provided answer of +20 score to a question of -5 score). Adding silver and bronze badges to this group could help reward answerers to spend time even if the question is not well rated, but this counteracts the motivation of the answerer to edit the question.

Answer 6

I think the best thing about this idea is its simplicity. You answer a question, you make some kind of change (presumably positive) to the question, you get rewarded. This is a behavioural pattern we should be encouraging - answer and edit.

It's inevitable some of these edits will be trivial, but even trivial edits are normally improvements. It will be interesting to see what happens to the edit queue if this is implemented - it may explode. If so, consider offering the badge only to those already with the edit privilege.

Some other users have suggested we need a net-negative to net-positive score swing in order to attain the badge. I don't think this is appropriate, since a question with a negative score rarely obtains a positive score, even after an astoundingly good edit.

Answer 7

The thing is, while junk edits won't fly, really trivial ones will. You can fix a one-character typo in a question and no one will ever roll it back. And you can even make a change to suit your personal style that doesn't really improve the question; as long as it doesn't make it worse, it won't get rolled back. And encouraging people to go out and edit a bunch of questions while simply not making them worse... meh.

I'm not sure how best to address this, but at the very least, what if the edits only counted toward the badge if they exceeded some number of changed characters, similar to suggested edits requiring 6 characters, except maybe a bit higher of a limit?

Yes, it would ignore some actually helpful tiny edits (little spell corrections and so on). But while those are good, they're not exactly the behavior the badge is trying to encourage. Surely any edit that really substantially helps the question is going to exceed whatever character limit you choose. And if someone has to work a little harder to get the badges, great.

See also Shokhet's answer with similar sentiments.

Answer 8

I can certainly see trivial edits being a concern with this. One way to combat that would be the requirement:

• The question gets an upvote after your edit (from somebody other than yourself.)

This rule would:

• Validate that the question is a good question (and presumably better now that it has been edited)
• Eliminate the need for the time period that was being used to prevent everybody from going back and editing every question they had ever answered. Now editing old questions won't help much because old questions rarely get more up votes.

Answer 9

If the end goal is to encourage edits that improve poor questions with good answers, to make those answers easier to find, why not explicitly make that the measure?

E.g., The badge would be awarded if you

• edit a negative-scored question,
• which has or later receives an answer that scores X or better, and
• after your edit the question receives Y number of upvotes, or receives a popular/notable/famous question badge (since you want to encourage edits for SEO purposes).

X, Y, and the level of question popularity badge would change according to whether this is a bronze/silver/gold badge. No time limits, so you don't penalize the less visible tags (or questions which are less visible because they were so badly downvoted).

For rewarding good answers which have a similar impact, @radouxju's proposal for creating bronze and silver levels of the Reversal tag is solid.

Mixing the answer and edit incentives into one tag somewhat confounds the goals—as others have said—encouraging minor edits, or worse, editing the question to match the answer regardless of the OP's intent. There is nothing in the current badge definitions that actually measures an improvement to the question.

Although I have on occasion edited a question I'm answering to use more standard terminology or more clearly reflect the real problem discovered in their code, that is rare. Generally for substantive improvements to questions, I rely on comments to ask for more information and also suggest that the OP edit for clarity. More often, if I am editing a question it is for minor things like code formatting, typos, and embedding external-linked images or code. And while I wouldn't mind getting a badge for the above, I don't think it matches the goals described in this post.

Answer 10

Sounds like a good idea but, as EnergyNumbers pointed out in his answer, these badges create the temptation to make a lot of "junk edits," just for the badges, which would "encourage more bad behaviour than good behaviour."

So, as a suggestion for his point that we need to figure out how a "question goes from being a bad question to a good question; to put that in a way that can be tested for automatically" -- what if these badges will only be awarded for questions that were flagged (by other people, to prevent abuse) as "Very Low Quality"?

Answer 11

I have to say that, after reading the first three paragraphs of the suggestion, I expected something completely different (no, not a Monty Pythonesque pun) than what it turned out to be after finishing reading it. I agree with most comments and answers suggesting it might end up with loads of badge-chasing trivial edits. It also seems like a great idea was lost to the simplicity of writing SQL queries for it. But I'm not writing this answer to reiterate those points, or to bash (or is is shellshock now?) the proposal with vague arguments.

What I want to describe is what I believe would be a better way of rewarding substantial improvements of the questions themselves;

1. First of all, lose the requirement that these new badges can only be awarded to those editors that also answered the questions they edited. Such badges should be awarded regardless if editors can answer the questions they edited to substantially improve them, or indeed felt they require additional answers to those they might have already received.
2. Award these badges for improved question's approval or visibility, i.e. that the question was indeed improved and it enjoys either substantial increase in site-wide visibility, or it clearly started gaining more up-votes / less down-votes per number of views by registered members.
3. Also award these badges for edits that saved the question, so those questions that were already closed but were reopened due to / after the edit.
4. Badges should be awarded solely by the quality of edits, not their quantity. So lose awarding them by number of edits and instead establish qualitative metric. That means you could gain them multiple times, but each time your edit should improve the question to some measurable effect. Bronze badges for helpful edits of questions that aren't yours, silver for great edits that made a substantial difference, and gold badges for those exemplary edits that moved the question from meh to fantastic on your qualitative scale.
5. Whatever you decide qualitative metric should be, award these badges also retroactively. And yes, if the editor merely copy/pastes OP's comment to the question's body itself and that helped it along, that should do, too. It might even communicate to our members better what the purpose of comments under contributions is.

The increased visibility point might be a bit moot, since we already award badges (Announcer, Booster and Publicist as bronze, silver and gold, respectively) for sharing a link to a question that was visited by certain amount of unique IPs, but if we limited that to only track new views (even revisits) by registered members, it should create an altogether different incentive.

Now, writing SQL queries to support awarding of such badges might be a bit less trivial, and I'm not sure it's even possible, but that's for the dev team to answer. I just wanted to suggest one way (or a few of them actually, depending on your interpretation) of rewarding our editors for qualitatively and measurably better non-trivial edits substantially improving our questions. Thoughts?

Answer 12

Why set the rates so high? 10 seems pretty high for bronze.

What about 5, 40, and 100?

Answer 13

After reading this good answer by ben, I think the specifics definitely should be changed.

Here is what I think:

• Answers that are eligible for this badge must have received at least 3 upvotes and 0 downvotes within the first 24 hours of being answered.

• The edit to the question must happen between the 12 hour time period before the answer was posted and the 12 hour time period after the answer was posted.

Also, the badge structure/levels should change. With the currently proposed rules, it will have people editing each and every question that they answer. And a lot of questions receive more than 1 answer, so you'll have so many edits on the question that are probably not really needed, but only edited for the sake of trying to get the badge. To make it less likely for people to be abusing this, and better user experience for everyone, I suggest the badge structure should be as follows :

• bronze badge: answered & edited 1 question
• silver badge: answered & edited 10 questions
• gold badge: answered & edited 50 questions.

The only issue I see with my suggestions, is the 3 upvotes, since some less popular tags may be at a greater disadvantage. But isn't that the case with many other badges, anyways? So instead of 3 upvotes, I'd be fine with 2, but I'd prefer 3.

Answer 14

I've gone in and edited a closed question so it's more understandable and was re-opened a number of times. I think that having the requirement that the question was closed and re-opened after your edit (and perhaps flag) should be part of the badge. Answering should be optional.

So basically: have a closed question re-opened because of your edit. To be fair, all editors to the question between being closed and re-opened (which I doubt would be many in the majority of cases) would be eligible.

Answer 15

I have answered quite a few bad questions in the past, producing useful answers to questions that other users couldn't even understand the requirements to (I know this because of the comments). Yet, I cannot remember an instance where I have re-written the question. Personally I don't think it's right to change a user's question. Instead, we should encourage them to improve it. I have no problem with fixing formatting, spelling, tags, etc... I just don't agree with re-wording.

Anyway, that's just my opinion. The more important issue I foresee is as follows:

In this proposal, a 'good' answer is defined by at least one upvote, but on what grounds do users decide that an answer is worthy of an upvote when they don't even understand the question?

I believe in instances where a question is unclear, but a 'good' answer has been upvoted. It is down to a lot of voters thinking "oh... so that's what the question means".

Just because someone posts a seemingly good answer, who is to say that they actually did understand the question (as per the OP's requirements)? I think only the OP can truly be the one to say "yeah, that's what I meant". Until the OP confirms your presumptions, what gives you the right to reword a question?

So on that basis surely only accepted answers should be included? This would make it much harder to earn the badges (the limits could be lowered to account for this), but at the same time would make it much harder to cheat your way to the badges, and thus prevent a lot of non-useful question edits.

Answer 16

1. Answer a question with less than 2 upvotes.
   Answer receives at least 2 upvotes.

2. Edit question within 12 hours after answering.
   For the next 12 hours, further edits to the question are only minor.

3. Question upvotes surpass 2.

(with "upvotes" the total score is meant)

---

Bronze: 1
Silver: 25
Gold: 250

Although the exact values for silver and gold should be ascertained statistically (so you don't have a thousand people getting a gold badge at once or - respectively - no one). I like the badge names very much btw.