I'd say no because I've seen on numerous occasions where a first-time answer is a new user who comes in and has a very good answer to share on an existing question that may have an accepted answer, but one that is still relatively poor. The worst thing we can do is not allow someone to answer a question when they have a good answer to contribute, and this has the potential to do that.
Rather than open ourselves up to the possibility of denying quality answers, I'd say we should stick with relying on the community/moderators to clean up the not-so-quality answers.