What effect do post notices have on answer voting?

Question

Moderators can apply post notices to answers that are substantially lacking in either explanation:

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.

or citation:

This post does not cite any references or sources. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed.

I have occasionally used the former; I have never used the latter, since lack of citation is not a criterion for deletion on the site I moderate. At first, I expected that a "needs improvement" notice would attract more downvotes to an answer than usual—by the strength of authority of the notice, or a sense of having received permission, for example. After some observation, I didn't see any particular evidence that was the case, and considered that diffusion of responsibility might also influence readers not to vote where otherwise they might be inclined to downvote (particularly when a good alternative choice, i.e., an accepted or more highly-voted answer, is present).

Currently we have only 5 of 3,130 non-deleted answers noticed in this manner on Engineering; this is not much of a sample for analysing something as complex as voting patterns and motivations. I know that other sites are larger and some use these notices more frequently, so perhaps there's enough data network-wide to have a shot at answering this:

When I am considering adding a post notice to an answer, how can I expect the notice to influence voting?

Answer 1

Since PostNotices got added to the schema this has become a lot easier to answer.

In this multi database query I'll fetch all PostNotices with PostNoticeTypeId 1 or 3. Those match with the two descriptions you've shown. For each PostId the Votes table is joined so we can sum the Up and Down votes. As we know the creationdate of the PostNotice and the creationdate of the vote (unfortunately at date precision) we can determine how many votes before the PostNotice and how many votes after the postnotice were cast.

The whole query looks like this:

-- result table
create table #result(
     site sysname
   , postid int
   , upv_before int
   , upv_after int
   , dv_before int
   , dv_after int
)
GO
declare @sql nvarchar(max) -- holds sql union over all databases

-- build multi db union query
select @sql=concat(N'insert into #result ', string_agg(concat(N'
select ', quotename(name, ''''), N' site
     , pn.postid
     , sum(case 
           when v.votetypeid = 2 
            and v.creationdate < pn.creationdate  
           then 1 
           else 0
           end ) [upvotes before notice]
     , sum(case 
           when v.votetypeid = 2 
            and v.creationdate >= pn.creationdate  
           then 1 
           else 0
           end ) [upvotes after notice]
     , sum(case 
           when v.votetypeid = 3
            and v.creationdate < pn.creationdate  
           then 1 
           else 0
           end ) [downvotes before notice]
     , sum(case 
           when v.votetypeid = 3 
            and v.creationdate >= pn.creationdate  
           then 1 
           else 0
           end ) [downvotes after notice]

from ', quotename(name), '.dbo.postnotices pn
inner join ', convert(nvarchar(max),quotename(name)), '.dbo.postnoticetypes pnt on pnt.id = pn.postnoticetypeid
left outer join ', convert(nvarchar(max),quotename(name)), '.dbo.votes v on v.postid = pn.postid
where postnoticetypeid in (1,3)
and v.votetypeid in (2,3)
group by pn.postid
'),'union all'))
from sys.databases
where database_id > 5
and (name not like '%.Meta' or Name ='StackExchange.Meta')

exec (@sql)


-- build kind of useful stats based on what is in #result
select 0 grp
     , site [Site]
     , count(*) [total observations]
     , avg([upv_before]) [upvotes before notice]
     , avg([upv_after]) [upvotes after notice]
     , avg([dv_before]) [downvotes before notice]
     , avg([dv_after]) [downvotes after notice]
from #result
where site = db_name()
group by site
union
select 1
     , '[Total over all sites]' 
     , count(*) [total observations]
     , avg([upv_before]) [upvotes before notice]
     , avg([upv_after]) [upvotes after notice]
     , avg([dv_before]) [downvotes before notice]
     , avg([dv_after]) [downvotes after notice]
from #result
union
select 2
     , site
     , count(*) [total observations]
     , avg([upv_before]) [upvotes before notice]
     , avg([upv_after]) [upvotes after notice]
     , avg([dv_before]) [downvotes before notice]
     , avg([dv_after]) [downvotes after notice]
from #result
where site <> db_name()
group by site
order by 1 ,4 desc

When run today this is what the result will look like, with the Engineering site selected:

Notice that you can switch SEDE to any site and re-run the query to get the results for any other site you're interested in. The query honors on which site you are.

I have removed all the Meta sites (but not Meta StackExchange) from the results because as we all know Meta is murder.

Keep in mind SEDE is only updated weekly, on Sunday morning.

Don't forget to give the tutorial a try.