You have nothing to worry about, you are using your votes absolutely the way they should be used. You get to decide for yourself when to vote and on what to vote, and using effort as a measure of post quality to help you decide to vote is great!
Voting irregularities are voting patterns aimed at defrauding the site, by trying to gain votes for yourself or someone else in a dishonest manner. Some examples would be:
Creating a separate account just to vote yourself up (sock puppeting).
Working with friends to vote one another up regardless of post quality (a voting ring).
Continuously serial voted someone (vote for one person's post regardless of quality, up and down).
There are automated processes in place to revert the occasional serial vote, but repeatedly voting on someone, especially if there are other connections (IP address, other patterns that connect you with the other account) then that could also be seen as a voting irregularity.
There are automated processes in place to revert the occasional serial vote, but repeatedly voting on someone, especially if there are other connections (IP address, other patterns that connect you with the other account) then that could also be seen as a voting irregularity.