A community is a living organism. As we measure our pulse and blood pressure, we need to be able to monitor the community's health to understand if there is something that we should pay special attention to.
Please take a moment to give it a read and let us know what you think.
I think it makes sense to keep an eye on meta activity to judge the health of a site, as a complement to activity measures on the main site. What is not clear to me from the blog post is whether this is a CM instrument alone, or something the community should be able to view and use?
What's missing from the blog post are details and data. There is a rough description of the method, and a paragraph of results. But the underlying data is not available, so the community can't fully review this indicator.
One of the things that is very hard when looking at any of the activity statistics is to judge what a "normal" value should be for a healthy site. It's very hard to get a good overview over the sites, that information is never directly provided by SE. And it's also hard to get yourself on SEDE (it's possible in general from what I've seen, but cross-site queries are quite far beyond what I could do, and I'm not afraid of SQL). There should be an easy way to compare the activity indicators across all SE sites, so that you can pick sites for comparison that make sense.
One thing that you can judge reasonably well on its own is the trend itself. Declining activity over a long timeframe is concerning in any case, no matter what the absolute value is. So the trends of these indicators should be available as graphs to the community.
So, doctor... if the community's health is failing, what's the medicine?
Are there any thoughts on what to do with the numbers yet? This is apparently a research project, but the blog post doesn't say anything about next steps. How will the gathered data be used, what can we expect?
The blog post seems to suggest this data can be used to reduce social complexity, what I couldn't really figure out is: how does knowing about meta activity, which is only a small part of the whole, help with reducing the social complexity of the entire network of sites?
And finally... I'm kinda worried this may put quantity over quality. Your metrics are mostly focused on numbers, and those can be reached in very 'unhealthy' ways as well. There was a lot of discussion here last year that brought along quite a few new faces, yet I doubt anyone would use those numbers as an indication that MSE was healthy. Is there any approach/safeguard to prevent an approach to community management that's overly focused on metrics and numbers?
In the post, you define a "meta community"1 to be healthy if it is both actively used and used for its intended purpose. I'd like to know how this notion of "meta health" is relevant for the associated site2, or in particular why it would be good for an SE site to invest in the health of their meta.
From the post, it seems you treat meta usage as an end in itself. I don't see it like that, for me meta is not an end by itself, but merely a means to an end. It is a tool to (among others) improve the quality of a site. Additionally, you talk about the intended usage. Whose intention do you mean? Different sites may have different intentions on how meta should be used, or different experiences in how meta has been useful for them. I understand that you want to have tools that are generally applicable to all sites, but I worry that you're going to reduce complex issues to a single number and miss important issues as a result.
For example, you note in your case study on Mathematics.SE that it is a healthy meta community based on usage metrics. However, I was surprised that you did not mention the following in your case study: there is significant disagreement on Mathematics.SE on how to deal with low quality questions. I'm not aware of a single place that has an overview of this issue, but discussions arising from the most recent moderator election questionairre should give you some idea. Whether this disagreement is good or bad is up for debate (and I don't intend to criticize Mathematics.SE here, so please forgive me if I misrepresent this issue), but this issue seems to be significant for that site and I think it is not good that this is issue is missed in an analysis about the "meta health" of a site.
1: As an aside, what community is this referring to? Shouldn't this just be the community formed by the site? "meta community" sounds like a group of people with the primary purpose of "being meta", which is the kind of behaviour that some argue can be a risk for a community if done in excess.
2: To be clear, I will use the word "site" to denote the group of people who visit and post on <sitename>.stackexchange.com and/or the associated meta an chat space. Some people may use the word "community" for this group, but people use this word in a way I don't understand, so I will refrain from using it myself.
A lot of users are active on the meta site
This needs some context. I participate on a smaller site and the community hardly uses meta. It's hard enough writing technically accurate answers (considering average research time per answer 30 minutes upwards) that taking the additional time to engage on "meta-discussions" just feels secondary.
Meta site are also fueled by users raising "issues", and the tiny community of mods adding authoritative answers. In a site where "issues" are rare and the core community is focused on the flow of content "meta-discussions" may simply not arise.
Besides, a lot of the users on smaller sites have their core activity on larger sites like Stack Overflow so there's little need or interest in repeating "meta-discussions" that have been held elsewhere before.
