Here is me poking at the idea.
I see where it is coming from. It would make reviewing even more efficient, i.e. result in more reviews done with the same investment of human resources, albeit with maybe a higher error rate. Even domain experts make mistakes from time to time (and sometimes even are biased). The idea of the 3 close votes requirements and the 5 close votes requirement before that was that we wanted consensus to reduce the overall error rate.
And indeed we not only need assurances about the decision to close or not to close but also about a suitable close reason. The close reason is the most important information for the content creator, so it better be right. How do we know that we present the right close reason(s)?
I agree with the general idea (to weigh review votes somehow by trust/experience), but I see the same problem I saw when we reduced from 5 to 3 votes and will repeat it here:
We must control the error rate of reviews
Why having a vote weight of 2 for experienced users, why not 1.64 or 2.47 and a threshold of Pi or something else completely arbitrary? Obviously there should be arguments why 2 is the best and nothing else is better. The argument should be that at this weight the error rate is still reasonably low while the reviewing efficiency is already quite high.
Ideally you would measure the error rate (by letting multiple people do the same review independent from each other) and then compute a review efficiency vs. error rate relationship in dependence of experience and select an experience dependent weight that controls the error rate uniformly over all experience levels from there.
Measuring the error rate (if we assume that reviewers get it right on average and aren't biased on average) should not be that difficult (and also add a few more false positives in the data set). This idea here only makes sense if the error rate indeed goes down with increasing experience. However, if this is the case, we might not need experience as proxy for the error rate, simply estimate an error rate for every single reviewer and take this as inverse weight. A new user could be a better reviewer than any gold badge holder. Why not? Just measure directly what you want to know.
I could imagine a system where everyone starts with weight 1 (or even zero for really new users) and depending on his review actions can either increase or decrease his weight.
And finally one more thing that needs attention:
Reduction of close-worthy content could reduce the need to review that much
Sure, there will always be content that needs to be closed and everybody makes mistakes, but a lot of duplicates might be avoided if only people searched more and a lot of unclear, unfocused questions or questions without debugging details (on SO) might not need to be closed if only the content creator would have been more careful and more knowledgeable when creating the content.
While this is somewhat complementary to this idea here, it affects the urgency of it and therefore should be kept in mind and resources should be spent on it equally.
This is strongly related to:
While we can't really hope to make reviewing fun, we can hope to make it feel less futile.
Reviewing can be fun (it's nothing else than giving feedback on existing content in order to improve it), if only the reviewers get the impression that the content creators invest the same amount of time and concern in the creation and improvement of existing content than the reviewers themselves invest into reviewing. If only that would be the case.
