What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 127 Stack Exchange communities.

Using the data dump I came up with a list of "answers" containing no useful text. Those answers are all just noise and should be deleted by a moderator.

And instead of spending the next days with flagging them for moderator attention one by one, here they are:

Answers posted in duplicate:

share|improve this question
    
(for info, I think we've got most of them...) –  Marc Gravell Jan 2 '10 at 11:19
add comment

2 Answers

To get the list I searched for posts containing only a small amount of different characters, based on the assumption that random typing like "hgfghhgffhgfgh" usually contains the same characters over and over again. To measure this redundancy I basically calculated the entropy of all posts:

double calc_entropy(const std::string &txt) {
   static const double log_2 = log(2.0);
   const int sz = txt.size();
   int chars[256] = {0};

   for (int i = 0; i < sz; i++) {
      if (txt[i] == '>' || txt[i] == '<' || txt[i] == ' ')
         continue;
      chars[(unsigned int)(unsigned char)txt[i]]++;
   }

   double H = 0.0;
   for (int i = 0; i < 256; i++) {
      if (chars[i]) {
         double p = (double)chars[i] / sz;
         H += p * log(p) / log_2;
      }
   }
   return -H;
}

When sorting the posts by the score calculated by this function (or minor variants of it), the problematic answers came to the top. But of course there are also false positives, like this or many variations of "Yes" and "No".

share|improve this answer
    
length seems to be a reasonably good indicator as well. –  Jeff Atwood Jan 2 '10 at 11:51
5  
+1 for doin' it in c++ –  Shog9 Jan 2 '10 at 16:58
add comment

Hmm... in some cases there is no edit history, but in some cases it looks like the answer has been removed (usually by the OP, perhaps lacking a "delete" button). I wonder which is the greater evil - rollback (against poster) or delete (of perhaps useful content).

share|improve this answer
4  
there's no restriction on deleting answers, though -- only some restrictions on deleting questions because of the implied auto-delete of all answers. –  Jeff Atwood Jan 2 '10 at 10:14
add comment

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .