32

I think Stack Overflow could be a very interesting AI test bed for answer bots.

Obviously it would be too much work for a single person, but maybe a university or corporate research group...

I'm thinking specifically about the IBM DeepQA bot.

It seems like it might be able to at least ID and answer the most easy "what is the definition of:"-style questions on Stack Overflow, which is why I was wondering whether it had been done before.

Do you think such a bot would be capable of maintaining a positive reputation on Stack Overflow? Or would wrong answers pummel it into the group?

Improve this question
15
  • 45
    I'm not sure you realize it but Jon Skeet is a bot ;-)
    – mjv
    Dec 6, 2010 at 18:32
  • 1
    do you have IBM's resources?
    – Mauricio Scheffer
    Dec 6, 2010 at 18:33
  • 7
    It only needs to handle regex and pointer questions, and reply "is this homework" to anything worded as a assignment and it will deal with most questions :) (Ha, ha, only serious)
    – The Archetypal Paul
    Dec 6, 2010 at 18:33
  • @Mauricio Let's pretend that I do. Do you think it is possible?
    – John Shedletsky
    Dec 6, 2010 at 18:34
  • BTW, with its broad and "meta" nature, this question has good chances of being closed. On this occasion I'd like to plug the upcoming stackexchange site Artificial Intelligence (area51.stackexchange.com/proposals/6607/artificial-intelligence) where this type of question [well... maybe expressed in more specific ways] would have more chances of surviving. If you haven't yet committed to this site, I encourage you to do so.
    – mjv
    Dec 6, 2010 at 18:37
  • 2
    The beta version of such a bot could be just the standard search for duplicates...
    – MPelletier
    Dec 6, 2010 at 20:39
  • I've considered trying to bot some of the more mundane moderator activities, but that would be really hard to explain in the event of a catastrophe. An answer bot seems much safer.
    – Bill the Lizard
    Dec 6, 2010 at 20:53
  • 1
    @Bill but what about a janitor bot that has the same rights as, say, a 2k user? That can e.g. vote to close but not close unilaterally? To top it off, written in some standardized language that people can suggest edits and patches in. Oh, and flag bad decisions. Would be really interesting to see that in action - could be a project for a AI-related University course or a lab....
    – Pekka
    Dec 6, 2010 at 21:33
  • @Rejoice: That would definitely be interesting to play with. I can't decide if it would be more useful on newer sites with fewer 2K users, or older sites with more established standards.
    – Bill the Lizard
    Dec 6, 2010 at 21:38
  • @Bill SO would be the most interesting field for a bot I think, because the topic is so very technical. I can see a bot failing much more often for, say, gardening related questions than questions on jQuery
    – Pekka
    Dec 6, 2010 at 21:41
  • @Rejoice: That's true, an advanced enough bot could test out its own programming answers. Not true for gardening or cooking. :)
    – Bill the Lizard
    Dec 6, 2010 at 21:44
  • @Bill awww. The real contest will then be "who can make the bot crash by feeding it an undigestible code sample?" :)
    – Pekka
    Dec 6, 2010 at 21:45
  • In order to be real AI and not just a bot, it should handle natural language. There is a whole computer field about this issue, where last works on declartive paradigm makes it a must to have tool.
    – user150068
    Dec 6, 2010 at 22:29
  • Did you mean to say "pummel it into the ground"?
    – Glen_b
    Aug 8, 2016 at 2:52
  • This question has been up for 7 years and no one's linked xkcd 810, yet? Time to fix that.
    – Michael
    Feb 8, 2018 at 17:49

4 Answers 4

Reset to default
24

I doubt whether a bot could give consistently decent answers even in this day and age except for the very simplest of questions.

What might work, however, is AI serving as "janitor-bots".

  • Vote to close as duplicate - e.g. judging from a network of related questions of which many were closed as dupes

  • Flag as spam

  • Ask for clarification when code is missing (but understand when it is not missing)

  • Suggest missing tags

  • Suggest re-formatting when user forgets code blocks

I would find it fascinating to see a very small number of (officially licensed and displayed) bots doing janitorial work - always within the limitations a normal user has, too - and seeing how well they work out.

Improve this answer
3
  • Well, this answer bot had consistency, and was quite good. It gave answer way better that "decent".
    – Box Box Box Box
    Aug 6, 2016 at 16:31
  • Janitor bots were (is) tried on Quora (even the low-paid (and low-skilled) moderators became too expensive for them), adding to the downward spiral of the site. With disastrous results. Stack Overflow embarrassed themselves by having some of the creators on the podcast. What is worse than having unsupervised moderation bots with unspecified IQ roam a site?
    – This_is_NOT_a_forum
    Jun 7, 2022 at 13:28
  • @This_is_NOT_a_forum those are serious accusations, care to please post a link to something more official explaining what is horrible about all of this?
    – Shadow Wizard Is Sad And Angry
    Jun 7, 2022 at 13:42
17

Yes. There was an answer bot created by the University of Antwerp which ran on Stack Overflow. It mainly focused on Git questions.

This is the paper which gives us more information of the bot. The technical report is found here.

The bot is currently closed.

It had answered lots of questions, with many answers actually valid.

Improve this answer
6
  • 8
    seems like the bot should have been voting to close as dupe since its methodology was to use the answers to duplicates of the asked question. The rep it earned seems very low. Why was it banned?
    – Kate Gregory
    Aug 6, 2016 at 16:35
  • 3
    @KateGregory: Because it was a bot answering questions it should have been flagging/voting to close?
    – Nathan Tuggy
    Aug 6, 2016 at 18:33
  • 1
    I'm curious to know if a mod made a decision, or of all the downvotes it got trigged a "we are no longer accepting" ban.
    – Kate Gregory
    Aug 6, 2016 at 18:54
  • 1
    @KateGregory Regarding your rep question: Well the second account which disclosed that it was a bot got very less rep because of the ban by mods. The first account which acted like a human got 300 rep, which I think is great for a bot. Regarding your ban question: I'm not sure. We have to wait for a mod
    – Box Box Box Box
    Aug 7, 2016 at 6:19
  • what account was the bot run under?
    – user340193
    Nov 29, 2016 at 17:08
  • @MarkYisri Tay, Mike, Ricardo, Jake, Joey (they are in a specific order). Go through the technical report I just linked in my answer to see why one account was better than the other.
    – Box Box Box Box
    Nov 30, 2016 at 2:52
2

I don't think (given today's technology) it would be able to maintain a positive reputation unless it's extremely selective in what it tries to answer - in which case it may only answer 1-2 queries/week. We already have something similar...it's called Google. If people are posting questions where the first Google response ISN'T the answer, then I doubt someone's going to be able to do better (unless they had more resources than Google). If people are posting questions where the first Google response IS the correct answer, then they are nimwits & their questions should just be voted down.

Improve this answer
1
  • 5
    I'd disagree somewhat - I think a bot could get a positive reputation, at least until a recalc, by playing cheap. It just has to use the "fastest gun in the west" strategy, which an AI is uniquely suited for, on common repeat questions. With this strategy it doesn't even need to use Google - it just looks for questions by new users, with no responses, for which its regex finds a similar (closed) question from its copy of the data dump. It then pastes the accepted answer from that question.
    – David
    Dec 6, 2010 at 19:00
1

It probably would be possible, given enough effort.

However, SO has effective anti-bot defenses.
You would need a special interface from the SO team.

Improve this answer
1
  • 2
    Yeah, to be clear, I was more interested in the theory of it than the practice.
    – John Shedletsky
    Dec 6, 2010 at 18:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .