So, the vote score you see is the combination of upvotes and downvotes.
You have two upvotes and three downvotes on your question, for a total of -1 score. I've even screenshot the current vote counts (visible at 1000 reputation points):
If somewhere it says -2 on the screen for you, it's incorrect, and you may have a cached page view or something in your browser cache you have to clear out.
Unless you're talking about your reputation page in which case every downvote you receive is a -2 hit to your reputation. THAT is different from the vote count.