Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 157 Stack Exchange communities.

What is meta?
Here's how it works:
  1. Any Stack Exchange user can ask a question
  2. The community provides support, votes on ideas, and reports bugs
  3. Your voice helps shape the way Stack Exchange operates

I think Stack Overflow could be a very interesting AI test bed for answer bots.

Obviously it would be too much work for a single person, but maybe a university or corporate research group...

I'm thinking specifically about the IBM DeepQA bot.

It seems like it might be able to at least ID and answer the most easy "what is the definition of:"-style questions on Stack Overflow, which is why I was wondering whether it had been done before.

Do you think such a bot would be capable of maintaining a positive reputation on Stack Overflow? Or would wrong answers pummel it into the group?

share|improve this question

migrated from Dec 6 '10 at 18:50

This question came from our site for professional and enthusiast programmers.

I'm not sure you realize it but Jon Skeet is a bot ;-) – mjv Dec 6 '10 at 18:32
I like this :) +1 – basarat Dec 6 '10 at 18:32
do you have IBM's resources? – Mauricio Scheffer Dec 6 '10 at 18:33
It only needs to handle regex and pointer questions, and reply "is this homework" to anything worded as a assignment and it will deal with most questions :) (Ha, ha, only serious) – The Archetypal Paul Dec 6 '10 at 18:33
i am not an answerbot. but this might be better asked on – stillstanding Dec 6 '10 at 18:33
@Mauricio Let's pretend that I do. Do you think it is possible? – John Shedletsky Dec 6 '10 at 18:34
BTW, with its broad and "meta" nature, this question has good chances of being closed. On this occasion I'd like to plug the upcoming stackexchange site Artificial Intelligence ( where this type of question [well... maybe expressed in more specific ways] would have more chances of surviving. If you haven't yet committed to this site, I encourage you to do so. – mjv Dec 6 '10 at 18:37
The beta version of such a bot could be just the standard search for duplicates... – MPelletier Dec 6 '10 at 20:39
I've considered trying to bot some of the more mundane moderator activities, but that would be really hard to explain in the event of a catastrophe. An answer bot seems much safer. – Bill the Lizard Dec 6 '10 at 20:53
@Bill but what about a janitor bot that has the same rights as, say, a 2k user? That can e.g. vote to close but not close unilaterally? To top it off, written in some standardized language that people can suggest edits and patches in. Oh, and flag bad decisions. Would be really interesting to see that in action - could be a project for a AI-related University course or a lab.... – Pëkka Dec 6 '10 at 21:33
@Rejoice: That would definitely be interesting to play with. I can't decide if it would be more useful on newer sites with fewer 2K users, or older sites with more established standards. – Bill the Lizard Dec 6 '10 at 21:38
@Bill SO would be the most interesting field for a bot I think, because the topic is so very technical. I can see a bot failing much more often for, say, gardening related questions than questions on jQuery – Pëkka Dec 6 '10 at 21:41
@Rejoice: That's true, an advanced enough bot could test out its own programming answers. Not true for gardening or cooking. :) – Bill the Lizard Dec 6 '10 at 21:44
@Bill awww. The real contest will then be "who can make the bot crash by feeding it an undigestible code sample?" :) – Pëkka Dec 6 '10 at 21:45
In order to be real AI and not just a bot, it should handle natural language. There is a whole computer field about this issue, where last works on declartive paradigm makes it a must to have tool. – user150068 Dec 6 '10 at 22:29
up vote 12 down vote accepted

I doubt whether a bot could give consistently decent answers even in this day and age except for the very simplest of questions.

What might work, however, is AI serving as "janitor-bots".

  • Vote to close as duplicate - e.g. judging from a network of related questions of which many were closed as dupes

  • Vote to close as spam

  • Ask for clarification when code is missing (but understand when it is not missing)

  • Suggest missing tags

  • Suggest re-formatting when user forgets code blocks

I would find it fascinating to see a very small number of (officially licensed and displayed) bots doing janitorial work - always within the limitations a normal user has, too - and seeing how well they work out.

share|improve this answer

It probably would be possible, given enough effort.

However, SO has effective anti-bot defenses.
You would need a special interface from the SO team.

share|improve this answer
Yeah, to be clear, I was more interested in the theory of it than the practice. – John Shedletsky Dec 6 '10 at 18:33

I don't think (given today's technology) it would be able to maintain a positive reputation unless it's extremely selective in what it tries to answer - in which case it may only answer 1-2 queries/week. We already have something's called Google. If people are posting questions where the first Google response ISN'T the answer, then I doubt someone's going to be able to do better (unless they had more resources than Google). If people are posting questions where the first Google response IS the correct answer, then they are nimwits & their questions should just be voted down.

share|improve this answer
I'd disagree somewhat - I think a bot could get a positive reputation, at least until a recalc, by playing cheap. It just has to use the "fastest gun in the west" strategy, which an AI is uniquely suited for, on common repeat questions. With this strategy it doesn't even need to use Google - it just looks for questions by new users, with no responses, for which its regex finds a similar (closed) question from its copy of the data dump. It then pastes the accepted answer from that question. – David Dec 6 '10 at 19:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .