6

This question is a follow up to How does SE Inc. understand and define meaningful user engagement?, which was asked yesterday and focused specifically how SE Inc. defines and looks at user engagement.

I think it could be worthwhile and beneficial for the community to also clearly specify our point of view here. (At least me for example, I am wondering if the rather "highlevel" input from Yaakov shouldn't be a bit more granular).

  • 2
    possibly related: Replace accept rate with citizenship level – gnat Mar 10 at 8:25
  • 1
    in this particular case finding it was easier than usual because I had an answer there (and I well remember being unhappy about Atwood leaving company too soon to implement his own proposal because I liked it a lot) – gnat Mar 10 at 9:30
  • I guess there is meaningful user engagement and there is non-meaningful user engagement. I wonder if by trying to maximize meaningful engagement like the company tries to, it may also maximize non-meaningful engagement. A trade-off between both might be needed. – Trilarion Mar 10 at 10:46
  • 1
    Sure. Whenever you start to measure human behavior in a somehow "public" way, you also change behavior. – GhostCat salutes Monica C. Mar 10 at 11:33
  • 1
    I think that the biggest difference to what the company seems to see as meaningful engagement and what the community may see as meaningful is quantity vs. quality. The company seems to want to maximize activity, the community may be more selective. It's just my opinion but I think the company may be more interested in as many as possible questions answered with comparably less regard of how useful these questions would be for others. Of course I can be wrong. – Trilarion Mar 11 at 16:43
9

The term active user obviously implies some sort of activity. So one could start with classifying activities using agile/design thinking personas:

  • Quentin, the questioner who regularly writes well received questions.
  • Frauke, who regularly finds and reports vote fraud, plagiarism, and other serious problems.
  • Annita, who often contributes by writing well received answers.
  • Victoria, who really likes voting other people's content
  • Ed, who is extremely good at improving other people's content, by editing
  • Regan, who constantly checks and works review queues

... and so on.

Interesting side note: are there negative personas like Valerie, the help vampire? Probably not, you would rather write up a story that says As Annita, I do want to be bothered with OPs coming back and turning answered questions into moving targets or such.

The point of such personas is that they come with a wide range of helpful aspects:

  • mutual understanding: done right, a few days after introducing a specific persona, anybody on the team knows that archetype, and what it stands for
  • techniques such as empathy maps: it is much easier to identify user needs and problems when you can relate to a specific persona
  • and of course: everything you do, any epic/story that gets defined to implement a new feature ... should relate to a specific persona. If it doesn't, there is a good chance that you are building something that isn't designed for your users.

Also of course: part of such personas is the fact you have to write down the numeric quantities and qualities that define each persona. Is Victoria voting once per week, or does it take 5 votes (rolling average) per day to be considered Victora?! Doing so also tells you what you need to measure.

In our case, it would then be helpful to think about "mixtures" of personas in the real users, and how that matters.

And then finally, one can assess:

  • what is the current situation (what are the needs and pain points of those user personas you care about) to then
  • define "where do we want to go to" (build a "to be scenario") to then
  • define the strategy to make that happen
|improve this answer|||||
  • And folks: please stop editing the names I used there. I do not intend to build the word Q U A V E R here. The point is that those names connect to the roles, like Ed the ... editor. Or Frau ke the Fraud detecting person. So, please: leave things be! – GhostCat salutes Monica C. Mar 12 at 19:36
7

Read-only visitors (that find the desired information quickly)

Visiting the site and learning something new or re-learning something forgotten is surely meaningful engagement with the knowledge base content. It's especially meaningful if you find the things you are searching for quickly, so the pure number of visits may not be the optimal measure there. This is kind of passive engagement, but also one of if not the main activity of a knowledge library.

Account creation

Active participation requires an account. It's meaningful as a prerequisite for active contribution.

Voting (especially on new contributions)

Is important to give feedback about the quality of a contribution and to separate the wheat from the chaff early on. It basically is an indicator of signal and noise, but beyond a certain threshold, it largely correlates with views. For example, questions about Git on SO score relatively high, but that doesn't mean they are great questions.

Asking (good, non-duplicate) questions

That basically requires a certain minimal question asking skill and often also a minimal understanding of the subject. It also requires thorough (re-)search and having thought about the subject for quite some time. If done right, there is a chance to get the problem solved and it's a necessary contribution to the knowledge Q&A. Somebody has to write these questions in the knowledge base.

If however, the question is not received well and ends up with a negative score or gets closed and especially if this happens too often it's a non-meaningful engagement. Something (expectations, guidance, lack of effort) has gone wrong.

Asking duplicates is somewhere in between, depending how much search effort has been invested, but in general finding duplicates is regarded a difficult problem still.

Answering (on-topic) questions

Is surely meaningful, but the meaning may be scaled with the gain that the asker and all future visitors get out of it. On SO there is the idea that many of the easy to answer and popular questions have already been asked and that's why the number of views per question and per time decreases. New questions are more specific and get seen less often. That may be a reason why the ratio of new answers to new questions has fallen on SO from approx. 2:1 to now below 1:1 over the last 10 years.

Commenting (on the content)

That's often the glue that is everywhere and improves the efficiency of the whole Q&A process. Comments on new questions are often the key to improving the questions, and highly upvoted comments on answers often indicate crucial information. Comments are often the first step to edits. The only drawback are the ~1-2% (YMMV) of unfriendly comments, they are not meaningful.

Editing

Whether it is removing superfluous verbiage ("Thank you", "Hope that helps"), correcting spelling, editing in of comments or just generally improving the content, many contributions have already benefited greatly from editing. Editing may even be underestimated still in its importance. The editors polish the content but do only get a minimal share of reputation for it.

One problem is that the higher the quality of the existing content, the more quality could get lost by a bad edit. Not all edits are good unfortunately, even if they were meant to be.

On the other hand, old content ages. Editing it, if only to indicate problems now, is a big improvement.

Moderation and curation (via review queues)

Reviewing is quality control and exception handling. Directing attention to new contributions or flagged contributions, it helps keeping the quality high and removing unsalvageable content.

Problem is that reviewing is hard work and the capacity for it is limited (and may even compete with other meaningful engagement). If people ask too many bad questions or post too many unfriendly comments, then the cleanup team has to work overtime. It may be more meaningful then to tackle the quality problems closer to where they are created.

Summary

It may be difficult to actually quantify and measure this meaningful engagement. It's not the sheer number of visits, questions, answers, votes, edits, comments or review actions, but it's more like a quality weighted number of these minus the number of meaningless visits, questions, answers, edits, comments, time needed for review.

|improve this answer|||||
  • 1
    While reading might be meaningful, it is not really "engaging". Just reading a question would be measured as a "Bounce". en.wikipedia.org/wiki/Bounce_rate – tkruse Mar 10 at 10:06
  • @tkruse Reading a question and learning something is not engagement? Why not? Creating an account and logging in just for the sake of doing it are not yet meaningful by itself I would say but surely they are required by the other actions. Will include them. – Trilarion Mar 10 at 10:38
  • I apply a definition like "participate or become involved in", and reading does not seem to fit. – tkruse Mar 10 at 11:47
  • 3
    @tkruse Okay. I see it more like the primary purpose of a knowledge library. Reading should be one of or the main activity and therefore readers are users of the library and are engaging with the library. It's more like a matter of definition and I called it now "passive engagement" in the answer. I see it as a very important part of the engagement though. – Trilarion Mar 10 at 12:00
  • 1
    @tkruse I gave it a thought a while ago and it looks like a tricky matter. If we define engagement as actions changing state of the system then we have to count reading - because it increases views which in turn trigger some badges and impact expiring of close votes / flags (and maybe something else, I didn't dig deeper). Have to admit, I am not quite comfortable with that but couldn't find a sensible way out so far – gnat Mar 11 at 17:28
  • @gnat, I would not count it that way because the intention of the action is not that system change. Engagement requires visitors to do something for the purpose of changing the system. – tkruse Mar 12 at 0:13
  • 1
    @tkruse yes, intention was just what I considered to differentiate passive views from active engagement back then. That would probably work but it feels somewhat slippery because this way we kind of try to get into user's head and risk drowning into even trickier philosophical swamp, "what if user intentionally clicked with main purpose to increase number of views and trigger system changes" (I recall cases of semi-fraudulent actions of some users willing to gain views-related badges, so it is not purely theoretical) – gnat Mar 12 at 11:36
  • 1
    That's why it's called meaningful engagement. What you describe is engagement, but not meaningful. There is still a grey area if course, but to me it's like having a conversation with someone else who is meaningfully engaged in that conversation. If they just listen, they are not engaged, if they talk nonsense without listening, it's not meaningful. – tkruse Mar 12 at 14:38
  • well, uh, this seems to only make things even more complicated. For example, reading is the primary reason for me to participate in more active ways ("To me, Stack Overflow is a tool. I use this tool in my job and I just need to keep it sharp.") If my reading qualifies as meaningless then how everything else I do because of it gets meaning. You know, this is maybe possible to somehow resolve with more sophisticated reasoning but it really feels like going down the rabbit hole, inventing more and more tricky ways out – gnat Mar 13 at 11:48
  • @gnat That's the same way I see it. I don't polish the content because I like polishing, it's because I want to give something back to the readers. I might even go that far and say that if nobody would read here, then nobody would write either. – Trilarion Mar 13 at 14:23
  • FWIW I recently got idea that we can try splitting actions to those that require account (not necessarily registered but still) and those that don't. This way allows to differentiate viewing from other actions (interesting that even anonymous post-feedback requires account, I just tested). I haven't yet made my mind on whether this can useful, need to chew it a bit more – gnat Mar 14 at 7:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .