This is by far, the most annoying behaviour of all annoying behaviours of StackOverflow.
Please let me turn it off. Add a checkbox or something.
I would suggest that if the warning is removed, a counter should be available somewhere in the profile page so that you can still see how you're doing. Arguably that would be useful to have whether or not the reminder was removed :)
I would suggest that it could appear in the user profile page next to the total votes - so you'd see how much you've voted overall, the up/down votes you've cast today, and how many you've got left for today, all in the same area. Obviously the "today" stuff would only be shown to the user themselves.