-24

One of the common problems on practically any site of Stack Exchange are questions that are off-topic. Many users do not read the FAQ or take the tour, but even to an experienced member (like myself) it can often times be frustrating to figure out which of the subsites is appropriate for a given question.

On the other hand we have a lot of data and also ML loves data. So I would suggest adding a single form to ask questions and have ML decide which subsite this question belongs to. An LLM or even a simpler approach should do a decent job. Doing that I hope we will both reduce the frustration of asking a question and reduce the off-topic questions that we see.

15
  • 4
    What happens if someone asks a question that happens to contain the word "Arduino", but isn't about the Arduino (per se), and they want to ask it on SE.Electrical Engineering, but the ML decides to post it on the SE.Arduino site instead? Commented Feb 22 at 19:09
  • @Greenonline similarly if the user is confused where to post it, there is a good chance, they decide to post it (wrongly) on SE.Arduino too. Confidence score can be used in some situations so that ML can say "I don't know", but inevitably there will be errors. So ultimately that would be a tradeoff Commented Feb 22 at 19:35
  • 2
    What happens if the question could work on multiple sites? What if it would get different answers on those sites? What if the question doesn't work on any site? There's nothing stopping you from prototying this and showing all of us skeptics how wrong we are. You wouldn't even have to build something that submitted a question, merely something that recommended a site. Commented Feb 22 at 19:36
  • 1
    While I agree we have a problem with users not reading the FAQ, help center and/or tour, your proposed solution seems very handwavy to me. I don't see this as a viable request.
    – Mast
    Commented Feb 22 at 19:37
  • 1
    @Mast It's always step 2 that is the problem: researchgate.net/figure/… Commented Feb 22 at 19:40
  • @RobertLongson the situations you describe are problematic even today without any solution. What should a user do in that case (honest question)? Should they post on all sites? Commented Feb 22 at 19:40
  • 4
    No, cross-posting is generally frowned upon.
    – Mast
    Commented Feb 22 at 19:40
  • 1
    They could try asking which site in a question here and we'll try to suggest options (just don't try to ask the question without making it clear you want a site recommendation for it). Or they could pick the most likely site and ask on its meta whether it would be OK and if not whether there is anywhere else. Commented Feb 22 at 19:41
  • I believe from ML perspective the problem is quite clearly defined - you have a classifier that takes a text sequence as input (the question) and returns a class - the subsite the question best fits or "unsure". This could be evaluated using the existing questions and so one would know what is the expected performance. Assuming performance is good enough then it is a matter of handling special cases like the ones shared above Commented Feb 22 at 19:48
  • 1
    Knowing what site to post on is a massive problem for new users in particular (who have no knowledge of the network and no privileges to get help in places like meta/chat if they even knew about it). However, I don't think that AI is a good solution here because I don't think it can understand the nuances of site scope (for example, as a result of policy changes on meta).
    – Laurel
    Commented Feb 22 at 19:56
  • 3
    I agree that this problem seems relatively well defined, from an NLP perspective. But you've set up a 173 class multi-class classification problem with wild class imbalance. I think this is certainly a better idea than their "edit this StackOverflow question automatically" plan. But I also don't think it's the question author's job to implement this for them. Commented Feb 22 at 20:22
  • So you're thinking of enhancing the existing "is this a duplicate?" lookup
    – Criggie
    Commented Feb 22 at 20:29
  • 2
    ML = machine learning, presumably? Commented Feb 23 at 5:31
  • @This_is_NOT_a_forum yes. To me that is a well established acronym but I work in that field so maybe that is not as obvious to others Commented Feb 23 at 19:15
  • Loosely related: meta.stackexchange.com/questions/396884/…
    – Kevin B
    Commented Feb 23 at 19:34

3 Answers 3

6

I'm not sure why this is being downvoted to oblivion. The idea could use some work, but there are some good thoughts here.

I agree that it's hard for people to understand which sites on the network may be appropriate for their questions. However, you don't need an LLM to do this. Some techniques in natural language processing have been around for decades and could be applied. The root of this type of problem involves terminology extraction, entity linking, topic segmentation, discourse analysis, and sentiment analysis.

Although I disagree with automatically posting to a site, I believe that it's entirely possible to, during the posting process, perform actions such as:

  • Determine if a question appears on-topic for a given site. Training on downvoted, closed, and deleted questions can provide hints about whether the question, as written, will likely be downvoted, closed, and deleted on the target site.
  • Find one or more other sites where the user could find related questions, link to their Tours and Help Center pages about topicality, and perhaps even search for possibly related questions on these specific sites to minimize cross-site duplication.
  • Allow the user to redirect their question to an appropriate site. I don't think this should be automated, but lowering the barrier of entry for getting the question to the right audience can be helpful.
  • Inform users of questions they can answer or curate (comment on, vote on). In other words, put questions in front of experts even if they don't necessarily visit or even have an account on another site based on past activity.

Since we care about quality, automatically making decisions on behalf of human users is risky. However, there are ways to use the vast amounts of data the network has coupled with proven techniques, and perhaps some newer and more modern and less proven techniques, to surface information that can help users - askers, answerers, and curators - make better decisions.

3
  • I agree with your reasoning and also agree you don't need to do an LLM to do this(in fact I also point this out) but I somewhat disagree about going down that deep in the techniques. As stack exchange has a lot of data (admittedly some noice in it too) I think using a pretrained model that has internal concepts akin to topic modeling will require less work(note: the reason I focus on that is because it leads to faster idea validation) Commented Feb 23 at 5:08
  • 2
    @IvayloStrandjev Given the stringent quality requirements of the network and the unique requirements and expectations of each site on the network, I don't agree that using a pre-trained model is appropriate. You would need to train a model on what each site believes to be good questions and bad questions, along with building relationships between topics. Commented Feb 23 at 10:51
  • Shoot my bad, what I meant was fine tune a pretrained model (still internal concepts will mostly be useful is my guess). I was thinking of a single encoder and then a classifier on top of that that. Most likely start encoder from checkpoint but also backprop to it during training Commented Feb 23 at 19:10
2

I appreciate the idea, and it would be nice to have some tools to help users. At the same time, I suspect this is not an easy problem to solve. I anticipate some difficult challenges here.

First, I want to warn you to beware the (faulty) premise that every question fits somewhere, on some Stack Exchange site. In practice, many questions -- perhaps questions -- aren't a good fit anywhere on any Stack Exchange site (at least not in the form that someone new to the site might naively write them down). It is not a goal of Stack Exchange that every question anyone might ever have should be able to be posted somewhere.

Most sites have their own quality standards and expectations. I think it would be problematic to have a user type in a question, for an AI to suggest a site, and for the user to post their question to that site without learning about that site's norms and expectations. Unfortunately, such an approach could easily lead to a bad experience, where the user posts their question, gets negative feedback because it doesn't follow the site's quality standards, the user then complains "but the website said to post it here!", community members respond negatively because their feedback is being ignored, etc.

See also the discussion at What is the expected way for a NEW user to identify the correct SE for a question?.

Still, it could be an interesting direction to explore. I'm not convinced it is the first place I would try to use AI. I suspect there might be other applications of AI that might be worth trying first. For instance, another possible starting point might be: given a site X and a question Q, will Q be well-received on site X? You could try to train the AI to learn the site-specific nuances of each site. You could also try to train an AI to learn common types of feedback from community members and to recognize questions that often lead to that feedback, to give posters this feedback before they post.

But despite my caveats, your idea does seem like it could be something interesting to experiment with and see how well it works.

2
  • Re "every question fits somewhere": An outcome could also be also be "Does not fit anywhere on the Stack Exchange Network". Commented Feb 23 at 5:39
  • @This_is_NOT_a_forum, absolutely -- but it's much trickier to determine whether a question that might, on the surface, appear relevant to site X meets site X's quality standards, norms, and expectations. I'm thinking it might be better to start with developing that ability, assuming the site X is already known. If that can be demonstrated to work well, then it makes sense to apply it to the harder question suggested in the proposal.
    – D.W.
    Commented Feb 23 at 7:25
2

The biggest problem I see is that the workflow should be:

  1. Identify where a question should be posted.
  2. Post a question written for that site.

Questions are not boxes one can freely move from one place to another. Reversing these two would mean that a user can write a question that does not fit the site they need to post on.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .