Yes, I agree that even downvoted answers should be equally as visible as normal answers on Meta.
There already is a penalty for downvoted answers: they're at the bottom of the list (by default ordered by votes).
Unlike non-meta SE sites where it makes sense to make such answers less visible because they would often be of low quality or misleading, negative answers on Meta simply express disagreement.
The problem is that, even with a -8 threshold, it's quite easy for a reasonable answer to lack complete visibility to anyone who might agree with it and upvote it.
It just reduces the debate to those ideas that were most upvoted first, not necessarily the best ones.
For example, I'm a reasonably high-rep user on SO, I mainly answer questions and vote (mostly upvotes but about 10% of downvotes). I also flag questions once in a while (e.g. for closure). While I'm reasonably active on SO, I must admit I'm not particularly active on Meta. It's partly because I find it less interesting in general, but it's mostly because I simply don't have the time for it.
It's clear that some people are much more active than I am on Meta, and I'm grateful for the enthusiasm and time spent by those who do, in particular moderators.
However, the danger is that a small number of frequent Meta visitors will just be able to make someone's ideas on meta far less visible, simply because this specific sub-group doesn't like it there and then. It's then almost too late for other users who are more focussed on the main sites and who might agree with this idea (generally regarding how to rule the main sites) to notice it and upvote it.
As a less frequent Meta user, I find the fact that such answers at the bottom of the (often long) pile deterrent enough. If I do find the time to read more answers on a topic, I'd like to be able to read a sample of all points of views with an equal degree of readability.