I think Stack Overflow could be a very interesting AI test bed for answer bots.

Obviously it would be too much work for a single person, but maybe a university or corporate research group...

I'm thinking specifically about the IBM DeepQA bot.

It seems like it might be able to at least ID and answer the most easy "what is the definition of:"-style questions on Stack Overflow, which is why I was wondering whether it had been done before.

Do you think such a bot would be capable of maintaining a positive reputation on Stack Overflow? Or would wrong answers pummel it into the group?

  • 43
    I'm not sure you realize it but Jon Skeet is a bot ;-)
    – mjv
    Dec 6 '10 at 18:32
  • do you have IBM's resources? Dec 6 '10 at 18:33
  • 7
    It only needs to handle regex and pointer questions, and reply "is this homework" to anything worded as a assignment and it will deal with most questions :) (Ha, ha, only serious) Dec 6 '10 at 18:33
  • @Mauricio Let's pretend that I do. Do you think it is possible? Dec 6 '10 at 18:34
  • BTW, with its broad and "meta" nature, this question has good chances of being closed. On this occasion I'd like to plug the upcoming stackexchange site Artificial Intelligence (area51.stackexchange.com/proposals/6607/artificial-intelligence) where this type of question [well... maybe expressed in more specific ways] would have more chances of surviving. If you haven't yet committed to this site, I encourage you to do so.
    – mjv
    Dec 6 '10 at 18:37
  • 2
    The beta version of such a bot could be just the standard search for duplicates...
    – MPelletier
    Dec 6 '10 at 20:39
  • I've considered trying to bot some of the more mundane moderator activities, but that would be really hard to explain in the event of a catastrophe. An answer bot seems much safer. Dec 6 '10 at 20:53
  • 1
    @Bill but what about a janitor bot that has the same rights as, say, a 2k user? That can e.g. vote to close but not close unilaterally? To top it off, written in some standardized language that people can suggest edits and patches in. Oh, and flag bad decisions. Would be really interesting to see that in action - could be a project for a AI-related University course or a lab....
    – Pekka
    Dec 6 '10 at 21:33
  • @Rejoice: That would definitely be interesting to play with. I can't decide if it would be more useful on newer sites with fewer 2K users, or older sites with more established standards. Dec 6 '10 at 21:38
  • @Bill SO would be the most interesting field for a bot I think, because the topic is so very technical. I can see a bot failing much more often for, say, gardening related questions than questions on jQuery
    – Pekka
    Dec 6 '10 at 21:41
  • @Rejoice: That's true, an advanced enough bot could test out its own programming answers. Not true for gardening or cooking. :) Dec 6 '10 at 21:44
  • @Bill awww. The real contest will then be "who can make the bot crash by feeding it an undigestible code sample?" :)
    – Pekka
    Dec 6 '10 at 21:45
  • In order to be real AI and not just a bot, it should handle natural language. There is a whole computer field about this issue, where last works on declartive paradigm makes it a must to have tool.
    – user150068
    Dec 6 '10 at 22:29
  • Did you mean to say "pummel it into the ground"?
    – Glen_b
    Aug 8 '16 at 2:52
  • This question has been up for 7 years and no one's linked xkcd 810, yet? Time to fix that.
    – Michael
    Feb 8 '18 at 17:49

I doubt whether a bot could give consistently decent answers even in this day and age except for the very simplest of questions.

What might work, however, is AI serving as "janitor-bots".

  • Vote to close as duplicate - e.g. judging from a network of related questions of which many were closed as dupes

  • Vote to close as spam

  • Ask for clarification when code is missing (but understand when it is not missing)

  • Suggest missing tags

  • Suggest re-formatting when user forgets code blocks

I would find it fascinating to see a very small number of (officially licensed and displayed) bots doing janitorial work - always within the limitations a normal user has, too - and seeing how well they work out.

  • Well, this answer bot had consistency, and was quite good. It gave answer way better that "decent". Aug 6 '16 at 16:31

Yes. There was an answer bot created by the University of Antwerp which ran on Stack Overflow. It mainly focused on Git questions.

This is the paper which gives us more information of the bot. The technical report is found here.

The bot is currently closed.

It had answered lots of questions, with many answers actually valid.

  • 7
    seems like the bot should have been voting to close as dupe since its methodology was to use the answers to duplicates of the asked question. The rep it earned seems very low. Why was it banned? Aug 6 '16 at 16:35
  • 2
    @KateGregory: Because it was a bot answering questions it should have been flagging/voting to close? Aug 6 '16 at 18:33
  • 1
    I'm curious to know if a mod made a decision, or of all the downvotes it got trigged a "we are no longer accepting" ban. Aug 6 '16 at 18:54
  • 1
    @KateGregory Regarding your rep question: Well the second account which disclosed that it was a bot got very less rep because of the ban by mods. The first account which acted like a human got 300 rep, which I think is great for a bot. Regarding your ban question: I'm not sure. We have to wait for a mod Aug 7 '16 at 6:19
  • what account was the bot run under?
    – user340193
    Nov 29 '16 at 17:08
  • @MarkYisri Tay, Mike, Ricardo, Jake, Joey (they are in a specific order). Go through the technical report I just linked in my answer to see why one account was better than the other. Nov 30 '16 at 2:52

It probably would be possible, given enough effort.

However, SO has effective anti-bot defenses.
You would need a special interface from the SO team.

  • 2
    Yeah, to be clear, I was more interested in the theory of it than the practice. Dec 6 '10 at 18:33

I don't think (given today's technology) it would be able to maintain a positive reputation unless it's extremely selective in what it tries to answer - in which case it may only answer 1-2 queries/week. We already have something similar...it's called Google. If people are posting questions where the first Google response ISN'T the answer, then I doubt someone's going to be able to do better (unless they had more resources than Google). If people are posting questions where the first Google response IS the correct answer, then they are nimwits & their questions should just be voted down.

  • 5
    I'd disagree somewhat - I think a bot could get a positive reputation, at least until a recalc, by playing cheap. It just has to use the "fastest gun in the west" strategy, which an AI is uniquely suited for, on common repeat questions. With this strategy it doesn't even need to use Google - it just looks for questions by new users, with no responses, for which its regex finds a similar (closed) question from its copy of the data dump. It then pastes the accepted answer from that question.
    – David
    Dec 6 '10 at 19:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .