17

Using the data dump I came up with a list of "answers" containing no useful text. Those answers are all just noise and should be deleted by a moderator.

And instead of spending the next days with flagging them for moderator attention one by one, here they are:

Answers posted in duplicate:

Improve this question
1
  • (for info, I think we've got most of them...)
    – Marc Gravell
    Commented Jan 2, 2010 at 11:19

2 Answers 2

Reset to default
16

To get the list I searched for posts containing only a small amount of different characters, based on the assumption that random typing like "hgfghhgffhgfgh" usually contains the same characters over and over again. To measure this redundancy I basically calculated the entropy of all posts:

double calc_entropy(const std::string &txt) {
   static const double log_2 = log(2.0);
   const int sz = txt.size();
   int chars[256] = {0};

   for (int i = 0; i < sz; i++) {
      if (txt[i] == '>' || txt[i] == '<' || txt[i] == ' ')
         continue;
      chars[(unsigned int)(unsigned char)txt[i]]++;
   }

   double H = 0.0;
   for (int i = 0; i < 256; i++) {
      if (chars[i]) {
         double p = (double)chars[i] / sz;
         H += p * log(p) / log_2;
      }
   }
   return -H;
}

When sorting the posts by the score calculated by this function (or minor variants of it), the problematic answers came to the top. But of course there are also false positives, like this or many variations of "Yes" and "No".

Improve this answer
2
  • length seems to be a reasonably good indicator as well.
    – Jeff Atwood
    Commented Jan 2, 2010 at 11:51
  • 5
    +1 for doin' it in c++
    – Shog9
    Commented Jan 2, 2010 at 16:58
2

Hmm... in some cases there is no edit history, but in some cases it looks like the answer has been removed (usually by the OP, perhaps lacking a "delete" button). I wonder which is the greater evil - rollback (against poster) or delete (of perhaps useful content).

Improve this answer
1
  • 4
    there's no restriction on deleting answers, though -- only some restrictions on deleting questions because of the implied auto-delete of all answers.
    – Jeff Atwood
    Commented Jan 2, 2010 at 10:14

Not the answer you're looking for? Browse other questions tagged .