Selecting interesting questions by means of more sophisticated analysis

Question

I’m thinking about possibility to make more sophisticated SE data dumps analysis in order to select most interesting questions. Maybe not a problem for SO because huge number of questions have 3-digits votes counts. So big numbers work very well here and for every adequate tag a novice can find a lot of questions worth reading. But maybe for smaller sites/less popular tags/… the existing very simple voting system could be boosted?

Ideas I have in mind are

• using of reputation of people who participated in the discussion,
• using of average votes for questions/answers posted by each participant,
• using of votes to views ratio (or additions to favorites to views ratio),
• to collect as much characteristics as I can invent and to feed them to a data mining program. In order to find correlation of some combination of them with vote/favorite count,
• some iterative algorithm, e.g. user with high rating give some points to questions they answered, questions give some points to all who participated in them, and again from the beginning.

What do you think, is it worth trying? Which other algorithms could you propose?

Answer 1

I have implemented some parser and made a couple of experiments on the database from Jun 2010 (700000 questions).

TOP-50 questions with sorting by score/views ratio with minor correction for questions with very small number of views. All questions that fell has view count < 1000 so it may be more interesting to make correction stronger:

Score Views Favorites Title
2 3 0 JNI_CreateJavaVM: Buffer overrun if I throw an exception in case of failure
100 800 27 .NET JIT potential error?
5 20 0 Programmatically log on to a site
2 4 0 How can I grab the iTunes-defined name for my iPhone/iPod device?
1 2 0 Word 2007 customisation - template based addins vs. custom dlls written in Visual Studio?
1 2 0 How can I Add Google My Maps API on my website?
74 626 11 Why use try {} finally {} with an empty try block?
10 43 2 in my sql statement I call sum twice for the same argument, is it duplicating the effort?
6 26 0 How to find out the current overflow-checking context?
27 245 4 Why is 'for(var item in list)' with arrays considered bad practice in JavaScript?
5 23 0 How can I define multiple types with the same name and different type parameters using Reflection Emit?
5 23 1 Max Value for MSAccess Date/Time
5 23 0 Can you extract/bring across the rating of a video from youtube to use in your own site?
9 42 0 LINQ to SQL: NOTing a prebuilt expression
9 42 2 Is there an algorithm for determining how much daylight there is?
6 28 0 Config file format
5 24 0 Arithmetic Operations in Arrays with PHP
30 292 7 C# 'is' type check on struct - odd .NET 4.0 x86 optimization behavior
5 25 1 JQuery - Can we capture both the p and div tags at a time in a single sentence using jquery?
27 270 7 Make dialogs compatible with "large fonts".
4 20 0 Partitioning requests in code among several servers
5 25 1 Java Annotations in eclipse Tooltip?
4 20 0 Detect variance on generic type parameters of interfaces
4 20 0 Problem with modifying a page with ajax, and the browser keeping the unmodified page in cache.
4 20 0 Getting access to a custom Master page from a user control
22 224 2 C#: Does Visual Studio 2008 have a tool to show which Exceptions could be raised by a piece of code?
9 46 5 Is there a way to upload videos to youtube from our Website?
24 246 6 How Can Paper Prototyping Help in Development?
5 13 0 jQuery Custom Lightbox issue
20 210 4 What is the usage of global:: keyword in C#?
4 21 1 How do I get the entity framework to work with archive flags?
4 21 0 JQuery: showing elements with a particular title
4 21 1 JQuery datepicker numberOfMonths: display current selection month last
6 32 1 .net, do you get a transaction object?
35 374 6 Refactoring Singleton Overuse
24 257 7 For which surprises do I have to prepare myself if I switch from Sun JDK to OpenJDK?
21 225 2 I found some bizarre code in the linux kernel, could some one explain it to me?
8 43 2 Do more specific css rules load better?
5 27 0 Adding indexes to gerrund tables
5 27 0 Hosting .NET service in IIS (No Interface)
5 27 0 Unbind Events with JQuery: Does it work with events setup in HTML?
41 446 15 What's the opposite of a nbsp?
12 66 4 App Engine SDK DevServer Read-Only Mode?
4 22 0 SourcesSafe 6.0: Search File Comments
4 22 1 Detached Smart Client Syncing
6 33 2 Using SOA principles over OOD in non-service code
4 22 1 How do I make a checkbox in jQuery lock a existing sliding window from moving and then unlock it by unchecking the box?
4 22 0 Corpus/data set of English words with syllabic stress information?
4 22 1 Cannot pull newly-added top-level directory into sparsely-checked out SVN repository
6 33 1 .Net: Do you know a complete downloadable Exception Handling tutorial or Resource ?

More interesting experiment was with users' reputation to post count ratio. I took average ratio between all participants in each question. TOP-50 is following:

AvgUserEst Score Views Favorites Title
122.154 5 846 0 Best approach with dynamic classes using Python globals()
99.610 2 38 0 Probability question: Estimating the number of attempts needed to exhaustively try all possible placements in a word search.
92.322 2 118 0 When is this VB6 member variable destroyed?
80.047 1 33 0 Python help() function and the string.title function
80.211 4 209 0 C++: has not been declared error
79.500 1 368 0 Obtaining client IP Address through ALUI Portal Gateway
67.595 2 148 0 Where do Java Applets live?
67.386 1 64 0 Python: Picking an element without replacement
67.481 4 130 0 Searching for simple variable names like 'c' or 'x' in Emacs.
61.594 2 163 0 Compression algorithms specifically optimized for HTML content?
60.158 0 167 0 Using AXSLT Components in Spring MVC
60.142 2 876 0 Merging MediaWiki contents together
57.827 2 64 0 #include statements that include a slash (header files made up of two parts)
58.244 2 132 2 Markdown: Why are there numerous implementations of the "Markdown" markup language?
56.223 2 462 0 Difference between classpath and endorsed directory
56.987 0 6673 0 How do I create a folder in VB if it doesn't exist?
54.513 2 57 0 Convert a number to its string representation[PHP]
53.250 0 62 0 Tomcat 6.0 doesn't set Cookies in webkit based browsers on localhost
53.250 1 337 0 PHP not able to set cookies through Nginx
51.412 0 28 0 VSLauncher starts wrong version
51.287 2 114 0 Has anyone read Web ReDesign 2.0, and if so, what do you think about it?
49.340 1 58 0 What are some best pratices for managing data in a cloud?
49.000 0 9 0 Getting Apache to close connection after a certain size limit
49.211 0 125 0 training for classification using libsvm
47.422 0 13 0 Performance differences between iframe hiding methods?
47.249 1 95 0 Custom broadcast events in AS3?
45.134 0 72 0 hadoop- determine if a file is being written to
45.081 6 144 0 Copy and pasting code into the Python interpreter
43.250 0 24 0 Microsoft powerpoint Add-In issue
40.346 2 194 0 Disable "Slow script" dialog in Safari while debugging javascript?
40.330 0 23 1 Interfacing my application with existing authentication systems
39.095 2 183 0 Data import wizard library for .Net?
38.276 0 14 0 How many indexes will actually get used?
38.382 3 97 0 C# BinarySearch breaks when inheriting from something that implements IComparable?
38.244 1 59 0 Setting css properties with javascript not working on strict mode?
38.138 3 36 0 Recommended language for modifying a MySQL DB
38.333 0 152 0 Calling SQL Server Reporting Services from PHP - authentication problem
39.182 1 218 1 C#: Multiple type parameters in extensions
38.171 1 336 0 how can you determine location of tag from inside said tag?
36.261 0 51 0 simple preg_replace() that I can't get working (i suck at regex)
35.000 1 405 0 ClickOnce error after delpoying -- has a different computed hash than specified in manifest.
36.043 2 98 2 Enhanced Directory Browsers on Webservers
34.439 1 183 0 Lines of Code in a MS-Access Database
33.175 0 101 0 Best way to display model data in a master layout in an MVC framework
34.967 6 503 2 How do I prevent passwords from being saved?
35.000 2 2097 1 Why is SQL Server 2008 Management Studio Intellisense not working?
32.045 0 72 0 Suggested (simple) approach for drawing large numbers of visual elements in WPF?
31.841 0 10 0 Rails Throws "Int64 does not have a valid constructor" Under IronRuby
31.841 0 29 0 Passing parameters in PowerShell 2.0
31.841 3 114 0 NDepend CQL Count Query