What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 128 Stack Exchange communities.

Question source: Which questions are the least frequently answered?

Note: the results here are only for Stack Overflow, as that is the most difficult to analyze due to data size. All the queries for this one are public, so if you want to run against the other sites, you can do so at your leisure on SEDE.

For the purposes of this discussion, when I say "quality answer" I mean an answer that qualifies a question as "answered" (i.e., not-"unanswered").


For a given topic (tag), how unlikely is it to receive a quality answer?

Data source: March 14, 2012 SEDE data: SEDE query

Tag         Questions  Unanswered  Rank  UnansweredPct  PctRank
android        155514       49813     1           32.0        2
facebook        22584        9029    18           40.0        1
ios             51057       13902    10           27.2        4
iphone         124468       26074     7           21.0       13
web-services    17860        5141    30           28.8        3
asp.net        114440       22832     8           20.0       16
jquery         161533       29186     6           18.1       21
javascript     186792       31103     4           16.7       24
java           219256       32440     3           14.8       30
php            202304       30630     5           15.1       29
wcf             20116        5181    29           25.8        5
c#             277900       35657     2           12.8       38
ajax            34092        8010    20           23.5       10
  • The results were truncated for brevity; the rankings are from 1 (most) to 50 (least); overall, the list is sorted by an arbitrary metric to put the most "problematic" tags at the top of the list.
  • is particularly concerning. This was also the case when I did the original analysis, so it doesn't look like things have changed much since then. It may be that the community lacks enough experts to tackle answering the huge volume of questions. It may also be that the APIs are not yet mature and there are lots of people running into problems that have no solution.
  • The results for the bread & butter tags of , , , etc., are really positive, with both huge volumes of questions and a very low percentage of unanswered questions. If the topic is mainstream and well-understood, it is highly likely to receive a quality answer. While that isn't particularly surprising, it does confirm our intuition.
  • This analysis did not take into account a question's vote score, but I suspect there is a positive correlation between high quality answers and high quality questions, regardless of topic.

For a given subtopic (2-tag combination), how unlikely is it to receive a quality answer?

Data source: February 6, 2012 NDA dump, considering combinations of the top 150 tags

This analysis can be run against the public data dumps, but the query is very very heavy. (Read: best of luck running against Stack Overflow.) The SEDE query is here if you want to run against other sites (you may need to add a minimum threshold for the combinations for the results to be meaningful).

Tag 1        Tag 2        TotalQuestions  Unanswered  UnansweredPct
facebook     php                    3056        1071           35.1
ios          ipad                   4821        1490           30.9
listview     android                4242        1263           29.8
wordpress    php                    4203        1218           29.0
ipad         objective-c            4077        1126           27.6
ios          xcode                  5099        1407           27.6
ajax         asp.net                3632         990           27.3
android      java                  17750        4546           25.6
html5        javascript             3937        1003           25.5
...
database     mysql                  6891        1005           14.6
jquery       html                  14092        2046           14.5
cocoa-touch  iphone                11925        1718           14.4
objective-c  cocoa                  8138        1131           13.9
css          html                  26566        3612           13.6
django       python                11093        1502           13.5
winforms     c#                    15504        1979           12.8
c#           .net                  54198        6591           12.2
c++          c                      9994        1199           12.0
sql          mysql                 14531        1251            8.6
  • The results are ordered by UnansweredPct descending, and the top 50 of those were recorded
  • People apparently have a hard time with list views in Android.
  • These results pretty much mirror the single-tag analysis.

Which are the most frequent subtopics (2-tag combinations)?

(This is a bit tangential to the main topic, but I thought it was interesting to include.)

Data source: February 6, 2012 NDA dump, considering combinations of the top 150 tags

As with the previous query, this can run against the public dumps, but you have to limit the number of combinations for the bigger sites. The SEDE query is here.

Tag 1        Tag 2       Questions
jquery       javascript      55839
.net         c#              54198
objective-c  iphone          36700
asp.net      c#              35233
mysql        php             29666
css          html            26566
html         javascript      23540
ios          iphone          20424
android      java            17750

If you'd like to have a look at all statistics I collected for this question, you can download the Excel spreadsheet here.

share|improve this question
2  
Why did ignore the elephant in the room? I'm talking about facebook. It has double the % of unanswered questions of all the other tags, close to 50%!! IMO it's a failed experiment –  Andreas Bonini May 3 '12 at 22:43
1  
@Koper True, facebook is the worst in terms of unanswered %, but most of the other tags have more unanswered questions in total. android has over 5 times as many. –  Bill the Lizard May 3 '12 at 22:57
    
"March 14, 2012 public data dump" Hey, where's that dump? I don't see it on clearbits or mentioned anywhere else. –  Greg Hewgill May 3 '12 at 23:38
    
Wonderful work. Although, I think comparing .net and C# together is a bit unfair (WPF + C# or perhaps Mono + C#), considering you likely are doing C# on .Net development :) –  user7116 May 4 '12 at 2:03
    
@Bill: yeah, but the highest the % is the less likely users are to ask questions about the subject. If I ask 1, 2, 3, 4 questions about facebook and I don't get a satisfying answer I won't keep trying. And since there is such a huge difference in the % between facebook and the other tags, I think it's the one that deserves the most attention, even if the total is lower –  Andreas Bonini May 4 '12 at 15:28
    
@Greg: Fixed. Thanks. –  Jon Seigel May 5 '12 at 13:30
    
@six: The combinations that I'm showing here weren't hand-picked: all the lists are ordered based on a metric. As you point out, though, some combinations of tags are highly correlated. –  Jon Seigel May 5 '12 at 13:33
add comment

1 Answer 1

android is particularly concerning. This was also the case when I did the original analysis, so it doesn't look like things have changed much since then. It may be that the community lacks enough experts to tackle answering the huge volume of questions. It may also be that the APIs are not yet mature and there are lots of people running into problems that have no solution.

This analysis is flawed. Android, iOS, Facebook...these are "hawt technologies" that everyone and their mother in the development world is trying to get in on. Consequently, it is not unusual to see badly worded questions from people who can barely write english about these technologies.

Additionally, some of the questions asked are so basic and betray such a lack of understanding (from Person off the Street X picking up O'Reilly book Y and thinking they're going to put their resume on the Net as a programmer today) that many experts will read such a question and not bother to waste their time with it.

Especially given that sometimes taking the time to write a well-thought out and reasoned response to question yields limited views and up votes. A reputation based incentive model works as long as the community is well engaged. But when many questions asked are unmarked as answered by the questioner, and there's little activity, there's not as much incentive to spend the time necessary to provide a detailed answer on question that "looks shady."

A more compelling, but obviously harder to answer, question than just looking at the tags is:

Based on these tagged questions, how many of them are substantive?

How can we define substanative? A difficult scientific question to answer. Proposed metrics:

  • How many of them encourage lengthy discussion?
  • What is the reputation of the asker? (are they User19374655845092 with rating 1? Those should probably be filtered out)
  • Do they betray a basic lack of understanding of principles of programming or introductory concepts, such as polymorphism, inheritance, etc etc
  • (this is the hardest question to answer. Involves some sort of text analysis, deriving a semantic network and trying to glean the concepts from it. May help to cluster users based on keywords used and classify question in proximity to others)

Just looking at tags and answered versus unanswered is good and fine. But be careful of the conclusions you're drawing from such a limited scope. Richer analysis would probably involve use of paired t-tests or ANOVA across more metrics. It would be fruitful to write up a blog describing the data collection and analysis process as well.

share|improve this answer
3  
"are they User19374655845092 with rating 1" I knew SO grows fast, but that caught me by surprise. –  Daniel Fischer May 3 '12 at 22:52
    
Well, it is meant to overflow after all :) –  Visionary Software Solutions May 3 '12 at 22:53
1  
You make a good point about the hot technologies attracting poor questions, but this could also mean that you have many newer questions in those tags on a percentage basis. These newer questions might not have had enough time to receive an answer or the answers to them haven't yet been upvoted (marking the question as "answered"). Android in particular has seen a significant rise in the number of questions being asked per day over the last several months. –  Brad Larson May 3 '12 at 23:18
    
For example, look at the difference between [ios] and [iphone], which should be completely overlapping. Only fairly recently have people been tagging iPhone and iPad questions with [ios] (Apple only started calling the OS that two years ago), where they used to be only [iphone], so you have a higher proportion of new questions in [ios]. [ios] has a higher percentage of unanswered questions than [iphone] does, despite the same kinds of questions being asked by the same types of people in both tags. –  Brad Larson May 3 '12 at 23:22
    
I stopped answering android questions because i saw the trend if low votes vs perfectly reasonable answers a while ago. Depressing really. –  wheaties May 4 '12 at 0:34
1  
Also covered here: meta.stackexchange.com/a/125361/144883 –  Uphill Luge May 4 '12 at 1:23
1  
Wouldn't nonsubstantive questions be indicated by low scores? Maybe this part of the analysis could be re-run excluding negative-voted questions. –  Ziv May 4 '12 at 11:01
    
It's news to me that downvoting no longer incurs a penalty. It's worth noting that's a new change. Knowledge of it may not have propagated yet. Consequently, lack of a negative vote doesn't necessary imply a good question. Conversely, one could probably potentially say that multiple up-votes does. –  Visionary Software Solutions May 4 '12 at 17:41
    
Thanks for your input; I agree with your reasoning. I'm not personally familiar with Facebook, Android, etc., programming, so while I can point at it and say that I think it's a problem, I can't be certain (also, I didn't think reading a small random sampling of questions was going to give me a full picture) -- that's why this is a [discussion] question. If you can put together a SQL query to run the analysis you think should have been run, I would be happy to run it on the NDA SO data and post the results here. –  Jon Seigel May 5 '12 at 13:40
add comment

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .