9

The reputation value from 5 to 10 on positive votes for questions has recently been changed to better reward users who create questions.

But ... what were the impacts on each community?

Has the number of questions increased due to this new incentive? Has the quality of the questions changed as the reward is better? Has there been a significant change in the amount of votes cast on the questions?

And since reputational correction was done retroactively, many users have gained the privilege of using new site tools. Is it possible to evaluate if these new users started using these tools and, if so, if they used them correctly?

How can we evaluate these points in isolation for a specific community?

1

Has the number of questions increased due to this new incentive?

With a SEDE query you can check if the number of questions (and answers) increased after the change, but of course not if this is due to the new incentive. You'd only discover that if you hold a survey among the users asking the questions. Quality of questions is likewise hard to measure quantitatively.


Has there been a significant change in the amount of votes cast on the questions?

I've answered that here with another SEDE query:

When you compare the week after the change (Nov 14th up to Nov 20th) with the week before (Nov 6th up to Nov 12th) – I've left Nov 13th out since the change was announced during that day – you see a 9.4% increase in upvotes and a 12.8% increase in downvotes.

However, I don't think we can call that significant yet. If I look at the site analytics for the sites where I have that privilege, fluctuations of that level are very common. I guess we need data over a longer period (and compensate for the increased number of users who can vote because of the reputation change).

Feel free to fork the queries to do your own analysis; we have more data right now (6-8 weeks instead of days) so this time it might actually be possible to detect statistically significant changes.

Not the answer you're looking for? Browse other questions tagged .