As Jeff famously quipped, good answers are like pearls, while questions are the sand that produces the pearls. The sand itself has no value; it's only useful because of the pearls that emerge from it.
Yet the sand is a necessary part of the process, too. Without the sand, we'd have no pearls.
So, yes, good questions deserve upvotes, too. Perhaps not as much as good answers — but note that the system already accounts for that, by awarding only half as much rep for upvotes on questions as on answers. And what makes a question good? Why, a good question is obviously one that attracts good answers.
Of course, sometimes bad questions attract good answers, too. It's not so common, but it does occasionally happen that a poorly written question ends up with a great answer, not because of the way it was asked, but despite it. And of course, there are a lot more poor questions than great ones, which skews the statistics.
So, what should you do?
You should certainly vote up the answers that you find helpful. And in my opinion, if you find the answers helpful, you should also at least consider voting up the question, too. After voting on the answers, just scroll back up to the top of the page and take a quick look at the question that prompted them. Does it look clear, and concise, and like the question that you would've asked, if you hadn't already found the answer here? Then yes, give it an upvote. It costs you nothing but a few seconds and a mouse click.
On the other hand, if the question is rambling and poorly written and full of irrelevant garbage, you can also choose not to upvote it. Too long to read? Don't upvote. Hard to make sense of? Don't upvote. Full of stuff that's completely irrelevant to the problem you had? Don't bother upvoting.
You should not feel pressured to upvote a bad question just to get rid of the nag dialog. The dialog is there to remind you that you can vote on questions, too, but it doesn't mean you should do so indiscriminately. There's no punishment for ignoring the notification.
On the other hand, if you find yourself repeatedly upvoting answers on a site, without ever upvoting any questions there, you might want to consider if your standards for question quality might be a bit too high. After all, those questions could not all have been completely worthless, given that they all produced good answers.