7

Background

The stats on se/sites is the only place on the network where stats across different sites can be compared.

This is a sample of what se/sites looks like

That page has 3 major issues:

  1. Inaccessible for Analysis
  2. Lacking Important Parameters
  3. Lacking Flexibility

These could all be improved very easily on the SE side, while any kludge on the user side will be clunky and lack the same power.

Inaccessible for Analysis

Currently you have to parse through the HTML of the page which is daunting to say the least given the way it is set up, or to go with the heavily rounded numbers by copy-pasting the actual text on the page.

Take a look at this HTML:

<div class="lv-info">
  <h2><a href="http://serverfault.com">Server Fault</a></h2>
  <p class="lv-description">Q&amp;A for professional system and network administrators</p>
</div>
<div class="lv-stats-wrapper">
  <div class="lv-stats-box">
    <span class="number"><span class="thousand" title="159,688 total questions">160<span>k</span></span></span> questions
  </div>
  <div class="lv-stats-box">
    <span class="number"><span class="thousand" title="297,710 total answers">298<span>k</span></span></span> answers
  </div>
  <div class="lv-stats-box">
    <span class="number">82%</span> answered
  </div>
  <div class="lv-stats-box">
    <span class="number"><span class="thousand" title="141,229 registered and unregistered users">141<span>k</span></span></span> users
  </div>
  <div class="lv-stats-box">
    <span class="number">
      <span class="thousand" title="198,491 median visits per day over the last two weeks">198<span>k</span></span>                        </span>
    visits/day
  </div>
  <div class="lv-stats-box">
    <span class="number">

      <span title="98 new questions per day on average over the last two weeks">98</span>                        </span>
    questions/day
  </div>
  <div class="lv-stats-box">
    <span class="number">
      4y6m
    </span>
    site age
  </div>
  <div class="lv-site-resources">
    <a href="http://twitter.com/ServerFault">twitter</a>
    <a href="http://blog.serverfault.com">blog</a>
  </div>
</div>

The information is hidden in the titles and not even formatted as numbers (you'd have to parse the title string to get at them). This is less than user friendly. All stats on that page should be available as CSV and/or XML and/or something equally accessible.

Lacking Important Parameters

I recently did some analysis for Workplace.SE but it was quite a bit of work. In addition to the inaccessibility of the numbers (I just dumped the text data in to excel, and worked with rounded numbers after a lot of massaging), there were very important pieces of information missing:

  1. Site Key (e.g. Mathematics = math.stackexchange.com)
  2. Beta/Full Site (this is only distinguishable by icon)
  3. Any usable date for site creation (it is only listed as an XyYm string)
  4. Date when the site graduated (not available anywhere)

These pieces of information, especially as more sites grow through the beta process and in to full fledged sites, should be available. Additionally, as Bill points out there is information on the API on more sites than available on se.com/sites, specifically closed betas. That information should also be added to se.com/sites.

Lacking Flexibility

gnat pointed out that our question total wouldn't be so bad on workplace.se if we had a lower ratio of closed questions than comparable non-beta sites. Unfortunately, there is no easy way to get that information. I had to go to each site, and do a search with is:question closed:true to get that info. This is horribly inefficient (especially when lacking the site key).

There should be an easy way to search all sites with the same key and get a nice response, since this functionality is not available either on the API* or standard search

* While it would be possible to create some javascript to loop through all available sites using the API, the API is limited in the information it can pick up compared to the standard search -- there is no easy filter, for instance, to look at only closed or migrated questions, etc.

Proposal

Please put revamping the stats page on the to-do list (completion date: soon™), with the following features:

Make Stats Accessible

Allow download in CSV, XML, and/or other useful formats.

Include Important Stats

Add the following stats:

  1. Site key
  2. Beta/Non-Beta
  3. Date of Graduation

Add Multi-Site Search

Allow searching using standard search across all sites, returning only total values. So if I search for the following string:

is:question created:2013-9-15..2013-10-16 closed:true

the results for each site will be given (using the key so I can easily add it to the existing downloadable data). Ideally multiple searches could all be provided and tossed in to a single file, such as:

is:question created:2013-9-15..2013-10-16 closed:true
is:question created:2013-9-15..2013-10-16
is:question created:2013-9-15..2013-10-16 migrated:true
is:question created:2013-9-15..2013-10-16 duplicate:true

Summary

These changes will improve the ability of sites to see where they stand in the network, and allow them to focus their communities on making needed improvements to raise the standard of quality across the SE network.

3

That information is already available through the Stack Exchange API. Specifically, I think you just need a combination of the /sites and /info endpoints. Here's an example for The Workplace.

https://api.stackexchange.com/docs/info#filter=default&site=workplace&run=true

The data is returned in JSON format.

{
  "items": [
    {
      "total_questions": 2319,
      "total_unanswered": 3,
      "total_accepted": 1420,
      "total_answers": 8541,
      "questions_per_minute": 0,
      "answers_per_minute": 0.01,
      "total_comments": 28925,
      "total_votes": 91670,
      "total_badges": 17680,
      "badges_per_minute": 0.02,
      "total_users": 8526,
      "new_active_users": 4,
      "api_revision": "2013.10.9.8200"
    }
  ],
  "quota_remaining": 9981,
  "quota_max": 10000,
  "has_more": false
}
| improve this answer | |
  • 1
    Thanks for the info Bill. As I pointed out in the question, there are many pieces of information that standard search provides, but the API does not (closed questions, migrated questions, etc.). Additionally, you are assuming that the users of seasoned advice (for instance) understand the API well enough to do all of this. Not all SE sites are focused on developers, and I would hope that SE wants to lower the barriers to getting this sort of information. – jmac Oct 16 '13 at 1:50
  • If, on the other hand, you or another SE member would like to put together a site that displays the information I request using API calls so that the rest of us can use it easily, I would be immensely grateful (and SE could use said code to add it to se.com/sites as well). Trying to figure out how to automate API calls is well above my ability (I am not a developer), while analyzing the data is not. – jmac Oct 16 '13 at 1:52
  • 1
    @jmac I was actually just thinking the same thing as I was taking a closer look at the data above. I'm sure we can come up with something. – Bill the Lizard Oct 16 '13 at 1:56
  • 1
    For anyone else interested, "questions_per_minute": 0 is bothering me, so I asked about it on Stack Apps. Can we get better precision on rate fields in the info object? – Bill the Lizard Oct 16 '13 at 2:10
  • 1
    Thanks @Bill! I also wonder why there is more functionality on standard search than on the API, but no idea where to ask (hence this giant beast of a feature-request). Should I be requesting the API to match the standard search over on Stack Apps as a backup plan? – jmac Oct 16 '13 at 2:12
  • @jmac Did you check the advanced search API to see if it has what you need? If it doesn't, then yes, Stack Apps would be the place to request a change to it. – Bill the Lizard Oct 16 '13 at 2:17
  • Wow, thanks Bill! I have been using the API for months and didn't realize there was an advanced search. – jmac Oct 16 '13 at 2:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .