As the number of high rep users increases, so does the number of users able to see the currently flagged posts.
This leads to the issue that if 1 in 10 people think that a post is spam when looking through the list, it probably will get the required 6 votes even though it isn't representative of the community's opinion.
Take for example this question (the link is for 10k users) with at present 3 offensive and 2 spam flags. I would agree that it is off topic a bit, but I don't see how it is offensive or spam. The only reason it is still being flagged as spam is that it is no. 2 on the "Flagged Post" list. (The last flag was about 40 minutes ago, despite the question being asked yesterday.)
There are other times when I have had to resort to writing a comment explaining to people who are thinking flagging it why they shouldn't flag it (with links to meta threads).
My only thought was to allow 10k users to disagree with flags, more akin to how voting works. I am not sure how well it would work and it would also add quite a bit more interface complexity to the 10k flags page which would be quite a bit of work for a minor problem.