99

Today, I started exploring the reviews section on Stack Overflow and specially the Triage queue.

In the history section I can see all my reviews, and also the triage category I chose (for example, 'Looks OK', 'Should Be Improved', 'Skip', etc.), but I can't see the final consensus of each review unless I click on one of them, and I have to select each review manually to see its consensus.

Why doesn't the review's history include the consensus of our each review next to it, so every one will know that he is doing reviews correctly? And how can I suggest that to the community?

Update:

Maybe something like this:

Enter image description here

Second Update:

Following Saposhiente's remarks within comments, I want to clarify that the checkmarks in the picture above means that the reviewer's choice matched the consensus.

However, the picture is only a simplified example regarding the emplacement where we can add the consensus column. Thus, I join the opinion of Saposhiente, suggesting to add a:

Consensus column that lists what exactly the consensus was, and bolds the differing items

6
  • 3
    Five other people agreed with me when I commented that this would be helpful: meta.stackoverflow.com/questions/285638/… Feb 11, 2015 at 22:27
  • 3
    I think that a checkmark or X would imply that you are supposed to always agree with the majority, and to choose based on what you think the majority would think. I think this is a bad groupthink attitude to encourage. I would prefer to see a "Consensus" column that lists what exactly the consensus was, and bolds the differing items.
    – Vitruvie
    Feb 28, 2015 at 2:30
  • @Saposhiente I can't understand the difference between "a checkmark or X would imply that you are supposed to always agree with the majority," and "Consensus column that lists what exactly the consensus was"? If someone have a checkmark that will mean he already choosed the exact consensus! Anyway I used the picture just to simplify and atract more attention, while the core of the question is meant for the exact consensus, So I do agree with you :)
    – Tarik
    Mar 2, 2015 at 21:48
  • 2
    @Tarik The difference is that a checkmark means "correct", but I'm making the point that the consensus is not always correct. I know it's just an example, but want to point this out anyway.
    – Vitruvie
    Mar 3, 2015 at 2:31
  • @Saposhiente no the checkmark doesn't always mean correct :) in my case that mean your review mutched the consensus
    – Tarik
    Mar 3, 2015 at 2:47
  • 8
    @Tarik The symbol checkmark means "correct." Since what you want to say is "matched" rather than "correct", checkmark is not the ideal symbol.
    – Vitruvie
    Mar 3, 2015 at 3:31

1 Answer 1

74
+100

We'll have to see if this gets implemented anywhere between 6 and 8 units but until then this userscript might give you the same information. Here is a screenshot with it in action:

review with mis/match with peers

and here is the script. If you run it you might notice that it takes a little time to show all stuff. The script uses an interval to fetch the data for you and tries to do that as quickly as possible within the boundaries of the SE ratelimits.

// ==UserScript==
// @name         Match against peers in review
// @namespace    http://stackoverflow.com/users/578411/rene
// @version      0.7
// @description  how you reviewed against your peers
// @author       rene
// @match        *://stackoverflow.com/review/*/history*
// @grant        none
// ==/UserScript==

/*global $:false */
(function ($, window) {
    "use strict";
    var tasks = [],
        intervalTime = 500, // 200ms (make this larger when throttled often)
        per = 90000, // milliseconds
        penalty = 60000, // msec to wait after 503
        rate = 150, // per 90000 milliseconds  (make this smaller when throttled often, but on 80 you're safe )
        interval;

    // for each review put a task in the queue
    function addTasks(tasks) {
        $('#content > div > table > tbody > tr > td:nth-child(3) > a').each(function () {
            var a = $(this);
            tasks.push(a);
            a.parent().append($('<span class="match" style="float:right; padding-right:20px; "></span>').html('&hellip;'));
            // if (tasks.length > 30) {return false; }
        });
    }

    // find the key, return the index
    function find(arr, val) {
        var index = -1,
            j;
        for (j = 0; j < arr.length; j = j + 1) {
            if (arr[j].key === val) {
                index = j;
                break;
            }
        }
        return index;
    }

    // update the outcomes array based on the html
    // in instruction
    function handleInstruction(instruction, outcomes) {
        var index,
            outcome;

        if (instruction.find('b').length > 0) { //  review-results
            outcome = instruction.find('b').text();
            index = find(outcomes, outcome);
            if (index === -1) {
                outcomes.push({ key: outcome, value: 1});
            } else {
                outcomes[index].value = outcomes[index].value + 1;
            }
        }
    }

    // parse the review outcome html and return an array
    // with outcomes and it's number of occurences
    function handleInstructions(instructions) {
        var i,
            ul,
            handled = false,
            outcomes = [];

        ul = $(instructions).find('ul > li');
        ul.each(function(i) {
            var t = this;
            handleInstruction($(t), outcomes);
            handled =true;
        });

        if (!handled) {
            // stangely enough instructions doesn't behave fully as a
            // jquery object, hence the juggling here
            for (i = 0; i < instructions.length; i = i + 1) {
                handleInstruction($(instructions[i]), outcomes);
            }
        }
        //sort on value
        return outcomes.sort(function (a, b) {
            return a.value > b.value ? -1 : a.value === b.value ? 0 : 1;
        });
    }

    function getInstructions(val) {
        if (val !== undefined && typeof val === 'string' && val.indexOf('Approve,') !== 0) {
            return val;
        }
        return '<div></div>';
    }

    // if the postback results are in process and show the (mis)match
    function postresultHandler(data, url) {
        var stats = $(data),
            status = $(getInstructions(data.instructions)),
            peers = handleInstructions($(getInstructions(stats[0].instructions))), // rank hold the reviews of your peers
            rank = find(peers, url.text().trim()), // url is your own review
            match,
            comp = false,
            more = false,
            audit = false;

        if (typeof status[0].textContent === 'string') {
            comp = status[0].textContent.indexOf('Review completed') > 0 ||
                   status[0].textContent.indexOf('Rejected') > 0 || // this is how we know the suggested edit is complete
                   status[0].textContent.indexOf('Approved') > 0;
            more = status[0].textContent.indexOf('needs more reviews from other users') > 0;
            audit = status[0].textContent.indexOf('Review audit') > 0;
        }
        if (comp) {
            if (rank === 0) { // if on top, match
                match = { html: '=', color: 'green'};
            } else if (rank > 0 && peers[rank].value === peers[0].value) { // not on top, but equal
                match = { html: '~', color: 'orange'};
            } else { // no match
                match = { html: '!', color: 'red'};
            }
        } else {
            if (more) {
                match = { html: '.', color: 'black'};
            } else if (audit) {
                match = { html: 'A', color: 'blue'};
            } else if (stats.isUnavailable === true || (stats[0] && stats[0].isUnavailable === true )) {
                match = { html: ' ', color: 'black'};
            } else {
                //console.log(status[0].textContent);
                match = { html: '?', color: 'black'};
            }
        }
        url.parent().find('span.match')
            .html(match.html)
            .css('color', match.color);
    }

    // get array with timestamps from localstorage
    function getThrottle() {
        var calls = window.localStorage.getItem('se-throttle');
        if (calls === null) {
            calls = [ Date.now() ];
        } else {
            calls = JSON.parse(calls);
            if (!Array.isArray(calls)) {
                calls = [ Date.now() ];
            }
        }
        return calls;
    }
    
    // update timestamp array for throttle
    function setThrottle(time) {
        var calls = getThrottle(),
            i;
        
        if (time === undefined) {
            time = Date.now();
        }
        for(i = 0; 
            ((i < calls.length - 1) && (calls[0] < Date.now() - per)); 
            i = i + 1) {
            calls.shift();
        }
        if (calls.length > rate) {
            calls.shift();
        }
        calls.push(time); 
        window.localStorage.setItem('se-throttle', JSON.stringify(calls));
    }
    
    // gets called by the setInterval
    function taskWorker() {
        var url = tasks.shift(),
            partReviewId = 3,
            partQueue = 2,
            // http://meta.stackexchange.com/a/214527/158100
            reviewTypeMap = {
                "suggested-edits": 1,
                close: 2,
                "low-quality-posts": 3,
                "first-posts": 4,
                "late-answers": 5,
                reopen : 6,
                triage : 10,
                helper: 11
            },
            parts;

        if (url !== undefined) {
            parts = url.attr('href').split('/');
            $.post('/review/next-task/' + parts[partReviewId],
                {
                    taskTypeId: reviewTypeMap[parts[partQueue]],  /* triage -> 10 */
                    fkey: window.StackExchange.options.user.fkey
                },
                function (data) { 
                    setThrottle();
                    postresultHandler(data, url); 
                })
                .fail(function (xhr, stat, error) {
                    // Service Unavailable means we're throttled, panic
                    console.log(xhr);
                    if (xhr.status === 503) {
                        // wait a full minute to get free
                        setThrottle(Date.now() + 60000); 
                    }
                });
        } else {
            setThrottle(); 
            window.clearInterval(interval);
        }
    }
    
    // check if we are within the throttle boundaries
    function isAllowed() {
        var calls = getThrottle(),
            timepassed;
           
        timepassed = Date.now() - calls[0];
       // console.log(timepassed);
        return (((calls.length < rate) || 
            (timepassed > per)) && 
            (calls[calls.length-1] < Date.now()));
    }
    
    // handle a task
    function task() {
        if (isAllowed()) {
            taskWorker();
        } else {
           // console.log('<< throttle >>');
        }
    }
    
    function init(tasks, time) {
        addTasks(tasks);
        return window.setInterval(task, time);
    }
    
    interval = init(tasks, intervalTime);

}($ || unsafeWindow.$, window || unsafeWindow));

The script has seven possible outcomes:

  • a match (=),
  • a mismatch (!) or
  • a tie/draw (~)
  • an audit (A)
  • a review you voted on but is not yet complete (.)
  • a pending review ( ) (space)
  • an unknown state (?)

It uses the ajax call to /review/next-task/1 to retrieve the review results of your peers (where 1 is replaced with the actual review-id).

Notice: The script has two variables that can be changed if you hit the SE rate-limit too often. The first is how many milliseconds between calls (currently 200 msec, higher than 1125 msec should be the max on non-shared IP-addresses). The second is how many calls within 90 seconds. Current setting is 150, lower than 80 is not needed on non-shared IP-addresses

Version 0.2 has dynamic ratelimiting and refactored code to clean-up and jslint-ed
Version 0.3 added three new states, audit, pending (others/own).
Version 0.4 replaced the use of innerText to textContent to support FF

Works and tested in:

  • Google Chrome with Tamper Monkey
  • FireFox with Grease Monkey 3 (if broken with version 4 it will stay broken)

Install from Greasy Fork

Install from GitHub (source)

14
  • 1
    Thanks @rene for you effort :) IMOP the answer worth the bounty, but I can't mark it as the accepted answer as my hope is to have that feature officially or to get an official reply that it will never be ... I am sorry that I can't test it these days as I have a lot of wok and travels these weeks, but I trust this will wok and I will give a feedback ASAP
    – Tarik
    Mar 22, 2015 at 23:30
  • What's the rationale behind choosing each of those signs? For example, I'm not sure what an equals sign is supposed to represent.
    – TylerH
    Oct 29, 2015 at 18:25
  • 1
    @TylerH I'm not a front-end guy. I needed three different characters that were distinguishable for the color blind. I explained their meaning in the post. If you rather have picture of unicorns I'm happy to add those if you provide me some.
    – rene
    Oct 29, 2015 at 18:31
  • 1
    @rene I think unicorns would be good, but what's key is that we get the Haskell icon for the ones that were failures.
    – TylerH
    Oct 29, 2015 at 19:16
  • At present the script seems to treat "(Improve) Edit" as being different from "Approve", "Reject and Edit" as being different from "Reject", "Edit" different from Leave Open/Looks OK, and so forth. That's a little misleading at times if all you really want is a quick visual skim to find all the red bangs. Oct 30, 2015 at 2:50
  • @NathanTuggy That is in the suggested edit queue, right? I didn't test in that queue but I'll have a look
    – rene
    Oct 30, 2015 at 9:32
  • @rene: Any time an action can unilaterally remove a review item with the same outcome as another review action that doesn't, you get the same effect. Close, Reopen, SE, LQP. Oct 30, 2015 at 17:47
  • @rene my browser tells me that jQuery's $ is undefined on running the script. Do you have any ideas how to solve it?
    – Nico Haase
    Jul 5, 2018 at 20:08
  • @NicoHaase are you on FireFox with Grease Monkey by any chance?
    – rene
    Jul 5, 2018 at 21:05
  • Thats true - is the script not compatible with GM?
    – Nico Haase
    Jul 6, 2018 at 7:05
  • 1
    @NicoHaase it was but after the release of GM4 things changed, specially for the unsafewindow behavior. I'll dig into this in the weekend and will post an update if it is solved. If you're not bound to Grease Monkey I hear both Tamper Monkey and Violent Monkey offer good alternatives as userscript manager in Firefox. I don't use either of those in the FF context so this is info I got second hand.
    – rene
    Jul 6, 2018 at 7:18
  • 2
    \o/ what an excellent way to waste 5 minutes of my time everyday. Thanks for this Dec 4, 2018 at 18:53
  • Where do I go to use this?
    – user474678
    Sep 28, 2019 at 16:03
  • 1
    @JL2210 any history tab in a review queue, for example: stackoverflow.com/review/low-quality-posts/history
    – rene
    Sep 28, 2019 at 16:46

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .