What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 130 Stack Exchange communities.

Possible Duplicate:
How about a difficulty rating for questions?

Right now, we have a "title", which provides a summary of the problem or the key question being asked, a "question" which provides the bulk of the content, and "tags" which (in my opinion) should be used to provide a way to index the content based on what is being asked. However, there's no way to express the difficulty level or experience of the asker.

Some people advocate the use of tags for this, but I disagree simply because the tags should describe what the question is asking (relevant technologies, tools, programs, topic areas) and not describe the depth of knowledge the user has in a given field or area.

The benefits of this are two-fold:

  1. Answers can be written at the appropriate level. An advanced user who knows something about the problem at hand can be given a more direct answer, while people who identify themselves as beginners would probably need to be walked through the process of solving the problem more.
  2. Questions can be sorted based on difficulty. This benefits both asker and answerers. If I'm an expert on the topic of a given Stack Exchange, then I would want to view all questions, since I can handle the more advanced topics. However, if I'm just a beginner, I might not want to wade through topics that are potentially beyond my abilities at this time. And if I'm an intermediate, I know that I probably can't answer the advanced questions but can learn from them, and I can probably answer a lot of the basic questions.

By depth of knowledge of the user, I mean in the area where the question is being asked, not overall. Let's take Jon Skeet as an ever-present example. He would probably be an "expert" in C# and .NET. However, he might consider himself a "beginner" in Haskell - if he was searching through the Haskell questions, he would want to filter out the intermediate and advanced questions for now.

There are a few possible implementations, but the best would probably be just displaying the depth of knowledge on the question lists and in the question page and then adding methods for searching based on difficulty when using the search box.

share|improve this question

marked as duplicate by Thomas Owens, Pops, devinb, mmyers Jul 26 '10 at 18:27

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

4  
Who is going to do this difficulty rating? The asker of the question himself usually can't really know how difficult his problem is going to be to solve. –  sth Jul 26 '10 at 1:54
    
That's not the point of the difficulty rating. The point is to let the asker let other people know the depth of his or her knowledge so that the best answers can be provided. If I've used X for 10 years and have a problem, I'd mark a question on X as intermediate or advanced because (1) I've tried the simple, obvious things and (2) I know my way around X, so you don't need to explain every little thing to me unless I ask for clarification using a comment. –  Thomas Owens Jul 26 '10 at 1:58
1  
Dupe: meta.stackexchange.com/questions/3811/… –  Margaret Jul 26 '10 at 7:40
    
Thank you for taking the time to ask this question. I had the same idea. –  Phillip Mar 21 '11 at 1:29

4 Answers 4

up vote 8 down vote accepted

A difficulty rating is so subjective and so fleeting, it borders on the metaphysical.

Take my question, for example: byte + byte = int… why?. The question is so simple, it would likely be covered in the first week of Intro to Computer Science 101.

Yet, the answer is so involved that even the most prolific and most qualified people on the system — I mean these people quite literally wrote the book on the subject — cannot agree on the answer.

So is it "difficulty level: 1" or "difficulty level: 10"?

Bit twiddling, pointer manipulation, and real-time programming (which was considered child's play back in my C-coding days) would probably confound most programmers today. So how would your rate those?

share|improve this answer
    
Yeah, after discussion, a few of these edge cases where difficulty != involvement came out. But I think a good solution would be just the encouragement of revisions to the question and answers based on comments. A "be bold" type thing - jump in and improve the clarity of questions and answers to make it more apparent what the asker wants (based on comments) and provide a number of good answers (just because one is accepted doesn't mean it's "the answer" - other answers might just be as valid). –  Thomas Owens Jul 26 '10 at 2:56
    
Subjective and argumentative? ;-P (interesting, a .NET question where neither Skeet nor Lippert have particularly good answers... Gonna have to bookmark that one!) –  Shog9 Jul 26 '10 at 2:58
    
@Shog9 - I feel like the "answer" lies in a conglomeration of the two perspectives. Whatever the case, Eric Lippert's discussions in that thread (and similar subjects) gave me a whole new perspective on people who flippantly say things like "The designers of X did blah-blah-blah because they're clueless or lazy." –  Robert Cartaino Jul 26 '10 at 3:04
    
Even if had an answer as good as this one, I don't think I would have been able to write it nearly this concisely. Incredible answer. –  devinb Jul 26 '10 at 11:32
    
It would be interesting to have this dimension on a question. Yes, it's subjective, but so are "votes". Amazon and netflix have user ratings, a community assigned "experience" level or "difficulty" level could easily be implemented in a similar way. –  monkut Jul 27 '12 at 1:02

Let's ignore all the arguments on whether or not difficulty has any objective meaning for as wide an audience of programmers as is found on SO.

Let's ignore the discussion of edge cases involving simple questions that experts struggle to answer, or difficult questions solved by students.

Let's boil this suggestion down to its essence...


Should we add one more piece of meta-data for askers to fill in before posting a question?

No.

There are a huge number of questions posted where the question itself isn't even in the title, and where the tags consist of whatever language the user works in... or just a word, picked seemingly at random, from the confused body of the post.

It's a pretty safe bet at this point that whatever new required meta field you add, most of the data entered into it will be wrong until some kind editor takes pity on it and tries to correct it. At which point it'll only be slightly wrong.

And then there's the matter of the 800K+ questions already asked, all of which will lack valid data in this field.

So, you'll have a new dimension to the data, but most of it will be empty, and most of what is there will be invalid. Garbage in, garbage out... Who would use such a thing?

share|improve this answer
  • Idea - Great Idea. In the perfect world it would make things alot easier.
  • Implementation - Horrible. Why? Because everything is subjective. Nobody could agree on weather 25% of the questions are hard, easy, or in the middle. Going down the latter is easy, since you can judge when during your learning time you should of known this. But going up the latter, how would you know? Is string manipulation easy, medium, or hard? Is regex easy or medium? Comments would turn into flame wars of essentially "I learned that during the first week!" "Your crazy, I never did it till 2 years ago" "Well thats because you are slow" "I'm not slow" and so on.
share|improve this answer

This is mostly my opinion here of course, but I'm not seeing this as very useful, let's go back to that C# example, what does the question difficulty have to do with the experience of the asker? I would think it has more to do with the answerer audience.

For example I don't do encryption daily, and a question around it I'd be more of a beginner than an expert, but a person who deals with it daily...that's an easy question. It's completely relative to who's answering, and unless you put in a "beginner to expert" rating somewhere for each question type (by tag I guess?) the system wouldn't know what to do. I look at probably 200-500 different tags on the questions I view daily...so this would be no small task of just assigning those values. Also, tags are too generic for this, at least in the larger categories.

I think the brick wall on this one is who or what determines that difficulty level, automating it means a lot of data entry for each user, and someone assigning it basically means a wrong vale, since it's completely relative to the person viewing the question.

share|improve this answer
    
Let's run with your example for a minute. Let's say you had a question on AES encryption in Java. You have been a Java developer for 15 years and know the language inside and out, but have never worked with AES encryption before. You would tag your question as intermediate. You don't need Java explained to you, but you do need help with a particular library or tool to solve your problem, perhaps even a single line of code. A beginner would need a much more in-depth explanation of the solution and any code would have to be explained so they could learn from it. –  Thomas Owens Jul 26 '10 at 2:07
    
@Thomas Owens: What's intermediate to me may be the simplest question to an answerer though, or impossible...how do I choose the difficulty level? Doesn't the fact I'm asking in the first place mean I have a higher perceived level of difficulty? This seems more like an inverse rating of your ability to google and find the information yourself than an actual difficulty rating to the question. For example if someone does AES daily, this may be a question they've answered a variant of 20 times...isn't my difficulty rating entirely irrelevant to other people? –  Nick Craver Jul 26 '10 at 2:11
    
Perhaps "difficulty" isn't the right word, then. What I'm trying to describe is a method of indicating my experience with something and the type of instructions I need to get from problem to solution. Some people need to be shown every step of the way because they don't know the steps, others just need a few keywords and a kick in the right direction. This would be a way to indicate that. –  Thomas Owens Jul 26 '10 at 2:14
1  
@Thomas Owens: Perhaps "Need Level" would be a better description, e.g. "A Little help", "An example", "Full run-down", maybe something in-between? In either case I still don't agree this would be a good idea, in the rare cases the asker cares they could just stick it in the question body. –  Nick Craver Jul 26 '10 at 2:18
    
I actually like the "need level". Perhaps if there was a community effort to actually improve questions, the need for such a thing could be alleviated. Based on this discussion, I'm thinking that ensuring that questions are asked in a better way and updated as the question changes and becomes more clear (based on comments to the question as well as answers and comments to those), it might be good enough. –  Thomas Owens Jul 26 '10 at 2:27

Not the answer you're looking for? Browse other questions tagged .