I think it is really important to know the reasons for getting down-votes.
The users must be asked for providing a valid reason for downvoting a post which already has 5 or more Up-votes (You set the number. I am sure that spams won't be having more than 5 up-votes)
The message (reason) should be private.
The identity of the person who downvoted (and provided the reason) should not be revealed to the person who got the down-vote.
It could be a one-way communication. i.e, The User who got the message should not be able to reply (to avoid abuses).But should be able to report it, if it is any kind of abusing.
This feature would do a great job to avoid people getting Down-Votes from those who keep Grudges on AskUbuntu.