In general, (when) is it better to [...]
Going through one at a time.
list progress in the form of comments
Not recommended, unless it's a very trivial answer to someone else's comment (e.g. "No, X didn't help, but thanks for the suggestion.").
This is preferred. However, edits should not be called out explicitly (e.g. with "EDIT: New information goes here" or similar). Instead, incorporate the information into your existing question as if it were there all along. Anyone who wants to know where it came from can review the post history or timeline.
proposed answers to myself?
Situational. If you genuinely think that a particular technique actually works or a particular explanation is actually correct, self-answering may be appropriate. But if you know that an answer is wrong or doesn't work, posting it as an answer should attract downvotes. The answer box is for answers which you believe are right.
What practices can I use to best achieve the following?
a. avoid cluttering my question and making it annoying/confusing for readers
To some extent, this is a matter of style and rhetorical writing. I usually use a structure similar to the following:
How do I frobnicate a sprocket? (title is a clear and succinct description of what you want to know)
I've been trying to frobnicate a sprocket in order to make my widget accept sprocket handles. (give context for what you are trying to accomplish, or what you want to know, and why)
I know that you can frobnicate cogs by using a cog wrench, but that thing doesn't seem to fit very well on my sprockets, and when I try to twist it, it slips off. (briefly explain what you've tried or what information you already know, and why it was unsatisfactory)
I also read this other question about zarking a sprocket, but zarking and frobnicating are really two different things, so I don't think that helps me here. (if someone has pointed to a "possible duplicate," but you think it's not a duplicate, briefly explain why)
What is the recommended way to frobnicate a sprocket from a widget? (restate the opening question, with a slightly more specific phrasing compared to the title, and preferably in bold, so that answerers don't get confused by all the other text on the page and try to answer a different question from the one you asked)
b. avoid the appearance that I consider my own research to be authoritative, given that the other users are often much more experienced/knowledgeable/competent/insightful
If you phrase it as "Here's what I looked into, but it doesn't work or left me confused about X," then no one will assume that you consider it authoritative. If you did consider it authoritative, you would have written an answer instead, so most people should understand that you're just "showing your work."