(How) Can SE sites change their reputation payout?

Question

A recent question on Puzzling Meta reminded me that there is a longstanding meta post at PPCG about awarding 10 rep per question upvote. This is mainly due to the relative difficulty and usefulness of asking/answering being different on the site (or appearing to be, at least).

The highly voted answer on PPCG is "Yes, we should do that," but it's been well over three years since it was posted. The answer there said that (at least at that time) meta.so had this implemented already, so it wouldn't/shouldn't be hard to do.

Given that it's been quite some time, I don't know if that's (still) true. I know that meat has undergone changes since then, but as long as I've been actively participating, meta.se (and meta.so before the split) has given 5 points per question upvote, not 10.

My question is: What would the process to change it look like? It's been discussed on meta.ppcg and agreed upon, so what's the next step? Or is it not feasible at all?

Answer 1

I do agree with the underlying premise here: On many of our smaller sites, good questions are the single most important bottleneck to growing stronger communities. And on a few more specialized sites, like Code Golf, questions actually take more work, and the askers are generating more value for the answerers than vice versa, which is the opposite of the norm.

But the problem is this:

The amount of rep that is generated by each action (question up-votes, etc.) is too integral to the system to be a per-site setting.

The few core basics that are most important to the way our sites work need to be fairly consistent across the network.

We do make a number of per-site settings, when what one site needs is different from others, and where the changes won't generally be surprising/confusing when you travel from one to the next.

Easy examples are embeds like the chess player, or LaTex support - they're there on the sites that need them, and not on the ones that don't, so almost no one is confused by the difference.

Answer 2

Optimizing For Pearls, Not Sand explains the existential reasoning for valuing answers more than questions. Even more discussion can be found from around the time the change was actually made on the blog and on meta. In summary, questions are mostly valuable to us because they create the space for great answers. (If you want a counterpoint, I recommend A Day at the Park.)

I'm quite sympathetic to the idea of changing question upvote value back to 10. Recently, I've focused on encouraging question asking, especially on small sites. It's certainly tempting to change the payout in order to influence user behavior. But after looking into the matter, I'm not sure that will actually work. Thanks to the natural experiment created by the reputation change back in March of 2010, we can try to observe changes in behavior on the three effected sites. Let's look at the proportion of questions to total posts on Server Fault (for sake of variety):

(Click the graph to see the public data. Stack Overflow and Super User have similar graphs.)

Since the first few weeks of the site, there's been a long and steady increase in questions being asked in relation to total posts. Another, less encouraging, way to put it is that questions are getting fewer answers on average than they used to. If you kinda squint at the graph, you might suppose there was a minor drop in question ratio around the time they dropped from 10 to 5 reputation. (The line changes from yellow to blue during the week of 2010-03-19 when word got out about the change.) But the minor blip is swamped by a much larger factor; users overwhelmingly shift from asking to answering questions as they post more often:

The x-axis is the N^th post of all users on Server Fault and the y-axis shows the proportion of posts in that position that are questions. So across all types of users, nearly 7 of 10 first posts are questions. By the 100^th post only 1 in 10 are questions. So the long-term trend is driven as much by the relative population of newbies to veteran users. Pulling out the influence of question reputation value (which seems minor in any case) would require building a statistical model to control for this and other factors.

Another possible effect of the change would be to increase voting on questions. The idea is that people won't mind as much awarding +5 for a middling question as they would +10. To test that hypothesis, I graphed the difference between question and answer scores over time:

Don't get excited about that spike in question voting the week after the blog announced the change; it's a bug in my query. You can move the spike anywhere you please by changing the epoch variable. Not surprisingly, voting tracks pretty well with the ratio of new questions to total posts.

As it turns out, there were two changes that occurred in mid-2011 that had a very noticeable change in user behavior: more votes allocated and removing the penalty for downvoting questions. Here's the difference between question and answer downvoting on Server Fault:

The cliff one year ago is the coming of the Roomba. Otherwise, the major trend is a clear shift from downvoting answers to downvoting questions. It seems a couple of seemingly minor changes to how the system handles question downvotes created a far stronger incentive than changing the payout for getting an upvote on a question. My guess is that the question downvote changes provided incentives that are more readily available when people think about how to act. To put it another way, a prolific voter can easily picture the cost of using up a daily vote quota or being penalized -1 for downvoting, but a prolific poster probably won't immediately be thinking of the relative reputation payout.

Summary

Putting aside the intrinsic worth of questions in relation to answers, changing the reputation payout for upvotes will likely have little, if any, effect on posting or voting. More profitable avenues to explore are incentives that directly encourage upvoting good questions.