This sounds like a very bad reason to downvote and unhealthy to everyone involved, you, the asker, the community as a whole, SE's voting system, and not the least its reputation.
If that's the only reason why you're downvoting the question, especially if you yourself think it's actually a good question, you are downvoting for purely personal reasons, which you shouldn't. We vote for posts here, not users.
And not only that, you might even be wrong in your assessment of the asker. You don't know why they never followed up on accepting. They're most certainly not doing it because they're bad, most likely not even because they're "lazy". Maybe the answer isn't so "obviously correct". Maybe they want to give the question more time and encourage more answers. Maybe they just haven't found the time for fully engaging the answers yet. Or at "worst", maybe they even agree with your assessment but just don't know fully yet how SE works.
In the interest of the communities you participate in and also its users', I would strongly implore you to reconsider this approach. Vote abuse of this kind can really hurt a community, and I don't really believe negativity of this kind helps you either, certainly not on the long run.
What you could do is ask the asker if there are any unclarities you haven't adressed yet. But even if he doesn't answer your requests, please don't take that personal. Be happy about the upvotes you got on your answer, be proud you contributed a good answer to the site, and enjoy the reputation you got for it. Acceptance, however, really doesn't mean all that much. And if you're worried that your time was wasted on possibly helping someone who doesn't appreciate it, think of all the other users who upvoted your answer for helping them (and the future users yet to come).