6

It might be useful to add a drop down field "Difficulty" for questions. With a very limited set of options, say:

  • Beginner
  • Advanced
  • Expert

This way, users could easily judge if they can answer a specific question (depending on their experience in the topic) and how much time it will probably take.

Beginners usually know they are new so they can easily select Beginner. Those with some experience just select Advanced and those working on a topic for years may chose Expert. I think it's not very difficult to judge halfway correctly.

And the difficulty rating does not have to be very precise anyway, it's more a rough estimate by the asking person.

As an addition, rep gained on this topic can be multiplied with a difficulty factor. This would solve the problem of lots of rep for often-viewed and easy to answer questions and few rep for less-viewed, difficult questions.

Update

Thank you so far for the answers. I clearly underestimated the fact that some users are going to always chose Expert difficulty.

Maybe readers voting on the difficulty would be a workaround. Of course, we cannot use the rep factor then, for the same reason.

This kind of voting would obviously introduce a whole new set of issues, since we could vote on both content and difficulty (I'm almost positive this kind of proposal existed before?).

Improve this question
3
  • 1
    It would be interesting to have this dimension on a question. Yes, it's subjective, but so are "votes". Amazon and netflix have user ratings, a community assigned "experience" level or "difficulty" level could easily be implemented in a similar way. And really there's no need for the asker to assign a value, they can't vote up their own questions either.
    – monkut
    Jul 27, 2012 at 1:05
  • Could we add "homework" to the list, right before "beginner"? :)
    – Chris Gerken
    Sep 30, 2012 at 3:16
  • I'd certainly like to see this, with readers (and questioners) voting. It would keep the question pool tidier.
    – PJTraill
    May 5, 2015 at 23:24

13 Answers 13

Reset to default
31

The fact that someone is asking the question implies "expert" automatically almost. They don't know: so they ask...

Another thought:

There are long, hard to read, badly posed questions with short answers, and very short questions with difficult answers because OP does not realise how damn difficult it could be.

I disagree with the proposal...

Improve this answer
1
  • 1
    What if there were just two levels instead of three? Just "beginner" and "advanced"?
    – lala
    Aug 29, 2010 at 14:36
44

I think it's not very difficult to judge halfway correctly.

You're asking people to rate themselves less than "expert"? Or rate their work and effort objectively in general?

This seems unlikely, based on my observations of how people work in the real world.

Improve this answer
7
  • 3
    I wonder if complexity could be determined based upon average length of response, average reputation of response-giver, and number of responses... :)
    – Sampson
    Jul 9, 2009 at 9:59
  • 6
    It's asking people to rate their questions as less than expert. For instance, when I asked an ASP.NET question a while ago, I'd probably have classed it as "beginner". My recent COM questions would have been "advanced" at a guess.
    – Jon Skeet
    Jul 9, 2009 at 10:31
  • 2
    My first comment was speculation about how we could infer, at a later time, the complexity of a question. But to follow up on Skeet's response, I too wouldn't be ashamed to ask "intermediate" questions...sometimes I simply don't know the answer.
    – Sampson
    Jul 9, 2009 at 11:14
  • 1
    Really? There are plenty of times I would rate myself as Beginner or Advanced. Not too many where I would call myself Expert. The way I look at it, if I was an expert, I either wouldn't be asking, because I'd know the answer, or I'm awesome and it's still a really hard and advanced question.
    – beska
    Oct 29, 2009 at 13:16
  • That's 'cause Jeff only hangs out with experts. ;)
    – Bernhard Hofmann
    Oct 29, 2009 at 17:44
  • 3
    question could be rated be people viewing the question. many questions are so basic that its painful to even read the title. Person that asked them didnt even read the simple tutorial about the language hes using.
    – IAdapter
    Jul 30, 2010 at 12:11
  • @ Jonathan Sampson: Average length - perhaps. Reputation - perhaps. Number of responses - not sure because the most difficult and most easy questions will sometimes receive few responses. Average time to answer (provided the answer is upvoted / accepted ) can probably given an indication. Easy questions are almost always answered within seconds on SO :)
    – JP19
    Jan 20, 2011 at 17:38
35

Within limits, aren't the people asking the questions the ones least likely to know the difficulty of the answer?

Improve this answer
1
  • 2
    The wording I was looking for...
    – gbn
    Jul 9, 2009 at 15:16
10

I have to agree with the OP because the amount of (obviously) beginner questions (like this one: Java generics) are decreasing the signal-to-noise ratio significantly for people who are looking to discuss topics that are reasonably advanced.

I mean absolutely no offense to easy questions or beginners who ask them, everyone has to start somewhere. However, classifying questions properly (and I understand it's easier said than implemented) is going to be helpful to beginners and advanced users alike.

Improve this answer
8

Maybe it would be better if the people answering the question rated the question. That way the average answer could be given. Of course it would have to be after a set number of responses.

Improve this answer
3
  • Yea, that idea was included in the update to the OP just a few minutes earlier. I wonder how people think about it.
    – mafu
    Jul 9, 2009 at 14:11
  • 1
    I suppose I could include a measure of how much thought I put into an answer, but I don't see why it would be worth the bother.
    – David Thornley
    Oct 29, 2009 at 14:22
  • I would pick Expert 100% of the time when I was answering a question, in order to maximise rep gain. Even if I wasn't being deliberately malicious, Jeff's point about people being able to objectively evaluate the level of work and thought they put in applies here.
    – user200500
    Jun 19, 2013 at 1:50
7

Part of the problem a system like this would address is that, as far as reputation, there is less incentive to answer a difficult question that requires some research than there is to answer an easy (for the answerer) question. It can be frustrating that a more advanced question just gets lost, even if it gets upvoted, when you are the poster or someone who would really like the question answered. I know the "bounty" system is designed to address this somewhat, but because of the delay before a bounty can be offered, it is difficult to get a question answered in a shorter amount of time. (While I would be in favor of reducing or eliminating the bounty delay, that proposal has been rejected.)

The difficulty rating, I think, would need to be provided by the answerers/moderators, not the asker. I think it would be possible over time to "start" a question at a given difficulty rating based on the history of the asker's questions. Perhaps the difficulty could be expressed as a reputation "multiplier"; that is, a question rated "1.3" in difficulty would earn 1.3 times as much reputation for an accepted answer as a question with difficulty "1".

I'd really like to see an idea like this taken seriously so that more difficult questions can actually get answers in a timely manner.

Improve this answer
5
  • "there is less incentive to answer a difficult question" - The bounty system is the mechanism that is supposed to balance this out.
    – Pollyanna
    Jan 20, 2011 at 17:24
  • @Pollyanna True, and I've explained why I find it deficient.
    – Andrew
    Jan 20, 2011 at 17:35
  • if it's truly a difficult problem then one shouldn't expect an immediate answer. The bounty is delayed so the community can answer it quickly if it's easy. It acts as a difficulty filter.
    – Pollyanna
    Jan 20, 2011 at 17:39
  • 2
    @Pollyanna I agree that immediate answers would not be forthcoming on difficult problems, both due to the research required and due to the fewer people able to answer them. I don't think the bounty system, even without the delays, exactly solves the problem. The bounty system does not actually indicate difficulty, but rather it indicates that someone is willing to "pay" for an answer with reputation. A difficulty-based system for awarding reputation for answers would award more on the community's perceived difficulty of the question (provided that answerers' votes don't count).
    – Andrew
    Jan 20, 2011 at 18:14
  • Quite likely. And well some just writes a code as an answer and says "Try this" and I want to ask "why?" Because it works for you and should work for me too? Where's the reasoning....even when OP requests for reasoning, some doesn't even bother a nod...
    – bonCodigo
    Feb 6, 2014 at 15:31
5

I could see an advantage to the system proposed by scheibk.

People answering the question could rate the question and the associated answers with {beginner|intermediate|advanced} level. The question and answer would have an level associated corresponding to the average of all of the rates.

This way, when I'm looking for a question, I could choose the appropriate level. This is more useful as a tool to consult the Q/A than to select the questions I could or I want to answer, as the level is associated to the average of the response.

Improve this answer
4

I disagree with this for much the same reason as a_m0d. ChrisF commented on his post and said that if someone did that (posted everything under the wrong difficulty rating) then people would downvote it, but I do not agree with this. You are introducing an opportunity for people to get downvoted based on the difficulty level, not the question itself.

Improve this answer
1
  • 1
    Having read this I now see my original comment is misleading. I still think (some) people would down-vote questions posted under the wrong difficulty rating, but I agree that doing that would be down-voting for the wrong reason.
    – ChrisF Mod
    Jul 9, 2009 at 13:26
3

I attempted tagging questions early on with "Beginner", "Intermediate" and "Advanced" but it never took off, for a really good reason:

There's no value in judging questions based on who they would likely apply to.

There are many questions tagged beginner, and that may be a little useful over time for those just starting out who want to limit their searches to question geared more to their level.

Otherwise, just search for or ask the question - the system doesn't need this categorization because everything is instantly searchable.

Improve this answer
1

I don't think that that rep idea would work - why wouldn't everyone just post "expert" questions so that they can get more rep for the question (or for answers that they post) and to help others get more rep quickly?

Improve this answer
1
  • 4
    If someone rated their question as expert but it clearly wasn't then I'd expect some people would down-vote it precisely for that reason.
    – ChrisF Mod
    Jul 9, 2009 at 11:03
1

I think it would be difficult to automate this:

Consider a user who has earned a lot of rep on tag <T> and then asks a question on tag <T'>. If <T> and <T'> are not related, the OP's rep earned from <T> has no effect in automating the classification of this question as easy/difficult.

So then we would consider using only the rep of the OP earned from questions tagged with <T'>. Well, questions can have multiple tags, so we might need to take the average (or some other function) of the difficulty rating generated for each of those tags. Then, we would need to recalculate this difficulty every time a re-tag is performed.
Note however, that this approach does not guard against a new or inactive expert. For example, I could be Guido van Rossum (the creator of the python programming language) and just be not very active on SO (therefore, I have very low rep on all tags). Does this mean that if I ask a python question, that it should be tagged as a beginner? Absolutely not. Also, what should we do when the OP's rep increases over time? Should we go back and re-classify all their questions?

So the next thing that I can think of is that the difficulty of questions increases with the the number of views the questions get, while remaining unanswered. That is, if more people have seen the question and it is still unanswered, then all those people who have seen the question are unable to answer it. Therefore, this must be a difficult question. Note that there are assumptions built into this method: no other activity (flag, vote to close, etc) is performed on this question. But then, what happens if it's an overly simple question that nobody wants to answer? What if it's a bad question and we're waiting for clarification from the OP?

Ultimately, I think that with some (or a lot of) heuristics and some AI classification methods, this might be doable, but not without a high computational cost, which is why I don't think it will be implemented. Still, I think this is an interesting idea

Improve this answer
1
  • 1
    I wasn't actually thinking of automating this - it's difficult enough to come up with a good value yourself.
    – mafu
    Dec 3, 2012 at 9:36
1

This is kind of done already. If a question is unanswered, has upvotes, and has a high viewed count, it's probably difficult. If a question has a bounty, it is probably difficult.

I think that rules out any solution based on feedback from viewers (who have already voted by viewing and not answering, in aggregate) or answerers (by which point the difficult is irrelevant.)

Furthermore this has a big risk of condoning poor questions. If the asker thinks a question is easy, shouldn't they research it more first? (Yes.) Conversely if they think it is hard and they won't get good answers, then... they should put a bounty on it.

In a nutshell, the difficulty of a question is a broken concept. We only want questions that are researched, and for which research has not turned up an obvious answer. The concept of intrinsic difficulty doesn't really exist when you're in a large community of specialists, e.g., some database programmers may be intuitively "smarter" than some Javascript programmers but won't have a darn clue how closures work. Real-world problems aren't "difficult" in any sense useful to SO; puzzles are.

Improve this answer
-1

I think we can give it a try and I do believe many people will or can learn to choose the correct difficulty of their own question. Or maybe add a "Not sure" option and a requirement of description to unlock expert difficulty.

Improve this answer
3
  • 1
    Every question is hard until you know the answer. Many a schedule has been destroyed by folks assuming tasks are easy before trying to do them for the first time...
    – Shog9
    Jan 12, 2014 at 23:55
  • The site is extremely hostile towards inexperienced coders. I bet it would get a heck lot more users if such a system is implemented. Lets say a division system gets implemented where the more points you have the more difficult you can rate your own question. Rating a question doesn't require that much knowledge, only common sense. Seriously I asked 3 questions which didn't have any duplicates and I got so downvoted I can't ask anymore, and that all happened in an hour of me registering in the site. Heck, I didn't offend anyone or something (I hope), just makes me look for a different one.
    – Ripedox
    Jan 13, 2014 at 0:04
  • 1
    The error you're making is in assuming that the site is hostile because folks assume knowledge when none exists; while this can be true, I see far more conflict caused by folks who either aren't forthcoming as to what they don't know, or are biting off far more than they can chew and can't accept "walk before you run" as an answer. This isn't going to help in either case.
    – Shog9
    Jan 13, 2014 at 0:13

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .