Return to Answer

As you noted in Part II:

Most users learn how to be good reviewers by actually reviewing posts, but sometimes find themselves making wrong decisions.

Given this, I’d love to see this formalized into an explicit review training queue, similar to @PM-2Ring’s proposal, “Practice queues for review training”.

Review Training Queue

When a user first gains privileges for a queue, they will be presented with a set of curated audits which are hand-picked by moderators to be a representative sample of that queue’s review criteria.

When a user fails a training audit, there is no penalty. But they receive an explanation that educates them about the review criteria; e.g.,

Careful! While that answer may be incorrect, it is nevertheless an answer. Reviews should remove posts that are not answers; voting should be used to rank answers by usefulness.

Once the user has successfully completed each of the training audits, they will then receive a message:

Congratulations! You’ve finished the training period, and are now ready to review some actual posts. But be careful! After this point, unhelpful reviews may result in a review suspension. Still feeling unprepared? Be sure to read the FAQ.

Variations

• Require the reviewer to correctly pass a percentage of training audits before being let loose on real audits. This would require having a larger pool of curated audits.
• Require suspended reviewers to go back through the training audiences before regaining their review privileges.
• Prompt reviewers to retake the training audits after a failed audit—or even if their reviews are suspicious (e.g., based on frequency of Looks OK posts or minority reports).

Moderator Responsibility

This ties in neatly with something @Cody has raised on Stack Overflow:

This is why I (and several of the other moderators) are staunch advocates of revamping the audit system to allow us to nominate posts as audit candidates (both false-positives and false-negatives). We would select posts that are obvious and unmistakable, yet still represent corner-cases that often seem to trip up reviewers. This would be far more effective for pedagogical purposes, and I believe it would still scale adequately, as we could serve similar audits to all users without defeating the purpose.

I’m not advocating for anything so expansive here. But this would offer a simpler version of that, while prioritizing the moderator effort toward onboarding and training new reviewers.

The goal is to provide an interactive simulation of reviews as an easier way of training new reviewers—and a friendlier way of addressing initial mistakes.

As you noted in Part II:

Most users learn how to be good reviewers by actually reviewing posts, but sometimes find themselves making wrong decisions.

Given this, I’d love to see this formalized into an explicit review training queue, similar to @PM-2Ring’s proposal, “Practice queues for review training”.

Review Training Queue

When a user first gains privileges for a queue, they will be presented with a set of curated audits which are hand-picked by moderators to be a representative sample of that queue’s review criteria.

When a user fails a training audit, there is no penalty. But they receive an explanation that educates them about the review criteria; e.g.,

Careful! While that answer may be incorrect, it is nevertheless an answer. Reviews should remove posts that are not answers; voting should be used to rank answers by usefulness.

Once the user has successfully completed each of the training audits, they will then receive a message:

Congratulations! You’ve finished the training period, and are now ready to review some actual posts. But be careful! After this point, unhelpful reviews may result in a review suspension. Still feeling unprepared? Be sure to read the FAQ.

Variations

• Require the reviewer to correctly pass a percentage of training audits before being let loose on real audits. This would require having a larger pool of curated audits.
• Require suspended reviewers to go back through the training audits before regaining their review privileges.
• Prompt reviewers to retake the training audits after a failed audit—or even if their reviews are suspicious (e.g., based on frequency of Looks OK posts or minority reports).

Moderator Responsibility

This ties in neatly with something @Cody has raised on Stack Overflow:

This is why I (and several of the other moderators) are staunch advocates of revamping the audit system to allow us to nominate posts as audit candidates (both false-positives and false-negatives). We would select posts that are obvious and unmistakable, yet still represent corner-cases that often seem to trip up reviewers. This would be far more effective for pedagogical purposes, and I believe it would still scale adequately, as we could serve similar audits to all users without defeating the purpose.

I’m not advocating for anything so expansive here. But this would offer a simpler version of that, while prioritizing the moderator effort toward onboarding and training new reviewers.

The goal is to provide an interactive simulation of reviews as an easier way of training new reviewers—and a friendlier way of addressing initial mistakes.

As you noted in Part II:

Most users learn how to be good reviewers by actually reviewing posts, but sometimes find themselves making wrong decisions.

Given this, I suggest we formalize this into an explicit review training queue, similar to @PM-2Ring’s proposal, “Practice queues for review training”.

Review Training Queue

When a user first gains privileges for a queue, they will be presented with a set of curated audits which are hand-picked by moderators to be a representative sample of that queue’s review criteria.

When a user fails a training audit, there is no penalty. But they receive an explanation that educates them about the review criteria; e.g.,

Careful! While that answer may be incorrect, it is nevertheless an answer. Reviews should remove posts that are not answers; voting should be used to rank answers by usefulness.

Once the user has successfully completed each of the training audits, they will then receive a message:

Congratulations! You’ve finished the training period, and are now ready to review some actual posts. But be careful! After this point, unhelpful reviews may result in a review suspension. Still feeling unprepared? Be sure to read the FAQ.

Variations

• Require the reviewer to correctly pass a percentage of training audits before being let loose on real audits. This would require having a larger pool of curated audits.
• Require suspended reviewers to go back through the training audiences before regaining their review privileges.
• Prompt reviewers to retake the training audits after a failed audit—or even if their reviews are suspicious (e.g., based on frequency of Looks OK posts or minority reports).

Moderator Responsibility

This ties in neatly with something @Cody has raised on Stack Overflow:

This is why I (and several of the other moderators) are staunch advocates of revamping the audit system to allow us to nominate posts as audit candidates (both false-positives and false-negatives). We would select posts that are obvious and unmistakable, yet still represent corner-cases that often seem to trip up reviewers. This would be far more effective for pedagogical purposes, and I believe it would still scale adequately, as we could serve similar audits to all users without defeating the purpose.

The goal is to provide an interactive simulation of reviews as an easier way of training new reviewers—and a friendlier way of addressing initial mistakes.

As you noted in Part II:

Most users learn how to be good reviewers by actually reviewing posts, but sometimes find themselves making wrong decisions.

Given this, I’d love to see this formalized into an explicit review training queue, similar to @PM-2Ring’s proposal, “Practice queues for review training”.

Review Training Queue

When a user first gains privileges for a queue, they will be presented with a set of curated audits which are hand-picked by moderators to be a representative sample of that queue’s review criteria.

When a user fails a training audit, there is no penalty. But they receive an explanation that educates them about the review criteria; e.g.,

Careful! While that answer may be incorrect, it is nevertheless an answer. Reviews should remove posts that are not answers; voting should be used to rank answers by usefulness.

Once the user has successfully completed each of the training audits, they will then receive a message:

Congratulations! You’ve finished the training period, and are now ready to review some actual posts. But be careful! After this point, unhelpful reviews may result in a review suspension. Still feeling unprepared? Be sure to read the FAQ.

Variations

• Require the reviewer to correctly pass a percentage of training audits before being let loose on real audits. This would require having a larger pool of curated audits.
• Require suspended reviewers to go back through the training audiences before regaining their review privileges.
• Prompt reviewers to retake the training audits after a failed audit—or even if their reviews are suspicious (e.g., based on frequency of Looks OK posts or minority reports).

Moderator Responsibility

This ties in neatly with something @Cody has raised on Stack Overflow:

This is why I (and several of the other moderators) are staunch advocates of revamping the audit system to allow us to nominate posts as audit candidates (both false-positives and false-negatives). We would select posts that are obvious and unmistakable, yet still represent corner-cases that often seem to trip up reviewers. This would be far more effective for pedagogical purposes, and I believe it would still scale adequately, as we could serve similar audits to all users without defeating the purpose.

I’m not advocating for anything so expansive here. But this would offer a simpler version of that, while prioritizing the moderator effort toward onboarding and training new reviewers.

The goal is to provide an interactive simulation of reviews as an easier way of training new reviewers—and a friendlier way of addressing initial mistakes.

As you noted in Part II:

Most users learn how to be good reviewers by actually reviewing posts, but sometimes find themselves making wrong decisions.

Given this, I suggest we formalize this into an explicit review training queue, similar to @PM-2Ring’s proposal, “Practice queues for review training”.

Review Training Queue

When a user first gains privileges for a queue, they will be presented with a set of curated audits which are hand-picked by moderators to be a representative sample of that queue’s review criteria.

When a user fails a training audit, there is no penalty. But they receive an explanation that educates them about the review criteria; e.g.,

Careful! While that answer may be incorrect, it is nevertheless an answer. Reviews should remove posts that are not answers; voting should be used to rank answers by usefulness.

Once the user has successfully completed each of the training audits, they will then receive a message:

Congratulations! You’ve finished the training period, and are now ready to review some actual posts. But be careful! After this point, unhelpful reviews may result in a review suspension. Still feeling unprepared? Be sure to read the FAQ.

Variations

• Require the reviewer to correctly pass a percentage of training audits before being let loose on real audits. This would require having a larger pool of curated audits.
• Require suspended reviewers to go back through the training audiences before regaining their review privileges.
• Prompt reviewers to retake the training audits after a failed audit—or even if their reviews are suspicious (e.g., based on frequency of Looks OK posts or minority reports).

The goal is to provide an interactive simulation of reviews as an easier way of training new reviewers—and a friendlier way of addressing initial mistakes.

As you noted in Part II:

Most users learn how to be good reviewers by actually reviewing posts, but sometimes find themselves making wrong decisions.

Given this, I suggest we formalize this into an explicit review training queue, similar to @PM-2Ring’s proposal, “Practice queues for review training”.

Review Training Queue

When a user first gains privileges for a queue, they will be presented with a set of curated audits which are hand-picked by moderators to be a representative sample of that queue’s review criteria.

When a user fails a training audit, there is no penalty. But they receive an explanation that educates them about the review criteria; e.g.,

Careful! While that answer may be incorrect, it is nevertheless an answer. Reviews should remove posts that are not answers; voting should be used to rank answers by usefulness.

Once the user has successfully completed each of the training audits, they will then receive a message:

Congratulations! You’ve finished the training period, and are now ready to review some actual posts. But be careful! After this point, unhelpful reviews may result in a review suspension. Still feeling unprepared? Be sure to read the FAQ.

Variations

• Require the reviewer to correctly pass a percentage of training audits before being let loose on real audits. This would require having a larger pool of curated audits.
• Require suspended reviewers to go back through the training audiences before regaining their review privileges.
• Prompt reviewers to retake the training audits after a failed audit—or even if their reviews are suspicious (e.g., based on frequency of Looks OK posts or minority reports).

Moderator Responsibility

This ties in neatly with something @Cody has raised on Stack Overflow:

This is why I (and several of the other moderators) are staunch advocates of revamping the audit system to allow us to nominate posts as audit candidates (both false-positives and false-negatives). We would select posts that are obvious and unmistakable, yet still represent corner-cases that often seem to trip up reviewers. This would be far more effective for pedagogical purposes, and I believe it would still scale adequately, as we could serve similar audits to all users without defeating the purpose.

The goal is to provide an interactive simulation of reviews as an easier way of training new reviewers—and a friendlier way of addressing initial mistakes.