How about a difficulty rating for questions?

Question

It might be useful to add a drop down field "Difficulty" for questions. With a very limited set of options, say:

• Beginner
• Advanced
• Expert

This way, users could easily judge if they can answer a specific question (depending on their experience in the topic) and how much time it will probably take.

Beginners usually know they are new so they can easily select Beginner. Those with some experience just select Advanced and those working on a topic for years may chose Expert. I think it's not very difficult to judge halfway correctly.

And the difficulty rating does not have to be very precise anyway, it's more a rough estimate by the asking person.

As an addition, rep gained on this topic can be multiplied with a difficulty factor. This would solve the problem of lots of rep for often-viewed and easy to answer questions and few rep for less-viewed, difficult questions.

Update

Thank you so far for the answers. I clearly underestimated the fact that some users are going to always chose Expert difficulty.

Maybe readers voting on the difficulty would be a workaround. Of course, we cannot use the rep factor then, for the same reason.

This kind of voting would obviously introduce a whole new set of issues, since we could vote on both content and difficulty (I'm almost positive this kind of proposal existed before?).

Answer 1

The fact that someone is asking the question implies "expert" automatically almost. They don't know: so they ask...

Another thought:

There are long, hard to read, badly posed questions with short answers, and very short questions with difficult answers because OP does not realise how damn difficult it could be.

I disagree with the proposal...

Answer 2

I think it's not very difficult to judge halfway correctly.

You're asking people to rate themselves less than "expert"? Or rate their work and effort objectively in general?

This seems unlikely, based on my observations of how people work in the real world.

Answer 3

Within limits, aren't the people asking the questions the ones least likely to know the difficulty of the answer?

Answer 4

I have to agree with the OP because the amount of (obviously) beginner questions (like this one: Java generics) are decreasing the signal-to-noise ratio significantly for people who are looking to discuss topics that are reasonably advanced.

I mean absolutely no offense to easy questions or beginners who ask them, everyone has to start somewhere. However, classifying questions properly (and I understand it's easier said than implemented) is going to be helpful to beginners and advanced users alike.

Answer 5

Maybe it would be better if the people answering the question rated the question. That way the average answer could be given. Of course it would have to be after a set number of responses.

Answer 6

Part of the problem a system like this would address is that, as far as reputation, there is less incentive to answer a difficult question that requires some research than there is to answer an easy (for the answerer) question. It can be frustrating that a more advanced question just gets lost, even if it gets upvoted, when you are the poster or someone who would really like the question answered. I know the "bounty" system is designed to address this somewhat, but because of the delay before a bounty can be offered, it is difficult to get a question answered in a shorter amount of time. (While I would be in favor of reducing or eliminating the bounty delay, that proposal has been rejected.)

The difficulty rating, I think, would need to be provided by the answerers/moderators, not the asker. I think it would be possible over time to "start" a question at a given difficulty rating based on the history of the asker's questions. Perhaps the difficulty could be expressed as a reputation "multiplier"; that is, a question rated "1.3" in difficulty would earn 1.3 times as much reputation for an accepted answer as a question with difficulty "1".

I'd really like to see an idea like this taken seriously so that more difficult questions can actually get answers in a timely manner.

Answer 7

I could see an advantage to the system proposed by scheibk.

People answering the question could rate the question and the associated answers with {beginner|intermediate|advanced} level. The question and answer would have an level associated corresponding to the average of all of the rates.

This way, when I'm looking for a question, I could choose the appropriate level. This is more useful as a tool to consult the Q/A than to select the questions I could or I want to answer, as the level is associated to the average of the response.

Answer 8

I disagree with this for much the same reason as a_m0d. ChrisF commented on his post and said that if someone did that (posted everything under the wrong difficulty rating) then people would downvote it, but I do not agree with this. You are introducing an opportunity for people to get downvoted based on the difficulty level, not the question itself.

Answer 9

I attempted tagging questions early on with "Beginner", "Intermediate" and "Advanced" but it never took off, for a really good reason:

There's no value in judging questions based on who they would likely apply to.

There are many questions tagged beginner, and that may be a little useful over time for those just starting out who want to limit their searches to question geared more to their level.

Otherwise, just search for or ask the question - the system doesn't need this categorization because everything is instantly searchable.

Answer 10

I don't think that that rep idea would work - why wouldn't everyone just post "expert" questions so that they can get more rep for the question (or for answers that they post) and to help others get more rep quickly?

Answer 11

I think it would be difficult to automate this:

Consider a user who has earned a lot of rep on tag <T> and then asks a question on tag <T'>. If <T> and <T'> are not related, the OP's rep earned from <T> has no effect in automating the classification of this question as easy/difficult.

So then we would consider using only the rep of the OP earned from questions tagged with <T'>. Well, questions can have multiple tags, so we might need to take the average (or some other function) of the difficulty rating generated for each of those tags. Then, we would need to recalculate this difficulty every time a re-tag is performed.
Note however, that this approach does not guard against a new or inactive expert. For example, I could be Guido van Rossum (the creator of the python programming language) and just be not very active on SO (therefore, I have very low rep on all tags). Does this mean that if I ask a python question, that it should be tagged as a beginner? Absolutely not. Also, what should we do when the OP's rep increases over time? Should we go back and re-classify all their questions?

So the next thing that I can think of is that the difficulty of questions increases with the the number of views the questions get, while remaining unanswered. That is, if more people have seen the question and it is still unanswered, then all those people who have seen the question are unable to answer it. Therefore, this must be a difficult question. Note that there are assumptions built into this method: no other activity (flag, vote to close, etc) is performed on this question. But then, what happens if it's an overly simple question that nobody wants to answer? What if it's a bad question and we're waiting for clarification from the OP?

Ultimately, I think that with some (or a lot of) heuristics and some AI classification methods, this might be doable, but not without a high computational cost, which is why I don't think it will be implemented. Still, I think this is an interesting idea

Answer 12

This is kind of done already. If a question is unanswered, has upvotes, and has a high viewed count, it's probably difficult. If a question has a bounty, it is probably difficult.

I think that rules out any solution based on feedback from viewers (who have already voted by viewing and not answering, in aggregate) or answerers (by which point the difficult is irrelevant.)

Furthermore this has a big risk of condoning poor questions. If the asker thinks a question is easy, shouldn't they research it more first? (Yes.) Conversely if they think it is hard and they won't get good answers, then... they should put a bounty on it.

In a nutshell, the difficulty of a question is a broken concept. We only want questions that are researched, and for which research has not turned up an obvious answer. The concept of intrinsic difficulty doesn't really exist when you're in a large community of specialists, e.g., some database programmers may be intuitively "smarter" than some Javascript programmers but won't have a darn clue how closures work. Real-world problems aren't "difficult" in any sense useful to SO; puzzles are.

Answer 13

I think we can give it a try and I do believe many people will or can learn to choose the correct difficulty of their own question. Or maybe add a "Not sure" option and a requirement of description to unlock expert difficulty.