No, it's not possible.

https://meta.stackexchange.com/questions/1170/should-i-be-able-to-cancel-my-up-vote-on-a-comment