When exactly does a downvote cost 1 reputaion?

Question

I was trying to find the correlation between downvoting and losing 1 rep for the downvote.

If I understood correctly downvoting a post with a score ≥ 0 always takes away 1 reputation from me.

But I couldn't see a consistent pattern while downvoting a post with a negative score. If I'm not mistaken sometimes I lose 1 rep after downvoting a post with a score of -1 and sometimes I don't.

I used to think the pattern is much clearer with posts with the score of -2 or lower. I never noticed any loss of reputation after downvoting one such post (though I cannot claim that I have checked meticulously every single time).

But to add to the confusion I noticed today I lost 1 reputation after downvoting one ranty (not-an-)answer posted by a new user laced with expletives (this one, deleted already). At the time of downvoting it already had a score of -4 (0 - 4).

What are the exact cases where downvoting leads to reputation loss? (Or rather when it does not do that)

Answer 1

Downvotes as a result of "spam" or "rude/abusive" flags being cast and therefore automatically done don't cost you rep.

Downvotes on questions cost nothing.

Any answer downvote you do should cost you one rep point. Unless the question is deleted in a certain period of time (I believe you regain that rep). (This excludes flags triggering a downvote)

(At least, as I understand the system)