Is it possible (or should it be) to know who down/upvoted my post?
Nope. It's a fairly fundamental part of the Stack Exchange network that voting remains anonymous so that people aren't doing each other favours or revenge voting. That's not going to change any time soon.
There are very limited circumstances where moderators can see vote patterns (we can see who votes for who in bulk) and Stack Exchange staff can track down who has voted for abuse reasons.
Oli's answer covers it... but this is significiant enough that I think it qualifies as an answer and not merely a comment:
When it comes to votes--especially downvotes, as those are the kind that tend to bother post creators--the right question to ask is usually not who, but why.
- Sometimes there is just no good reason. There are silly votes, both up and down. Some result from people clicking by accident and not noticing. There are other reasons too.
- Maybe there is something wrong or dangerous in your post, or something someone thinks is.
- Could the post(s) be unclear?
- Are there comments that may help to explain it? This is not guaranteed, and when you see a critical comment and a downvote it's not safe to assume they came from the same person, but either way, if a downvote calls attention to a critical comment that turns out to be valuable, that's (pardon the pun) a plus!
- If there's nothing of value in the downvote or any accompanying comments, or they represent a disagreement about the quality or accuracy of the post where you've decided you're not going to change it... then you can simply move on.
Currently you have 1,195 reputation. Each downvote removes 2 rep. So you'd have to receive an enormous amount of downvotes for it to be a significant change to your rep. If you think you're a victim of serial downvoting, you may want to talk to a moderator (one way is to flag a post of yours where you think it's happening, with a custom flag). Otherwise, it's really no big deal. And a couple of downvotes in a short time, even from the same person, is not serial downvoting. Furthermore, most serial downvoting (and serial upvoting as well) has the associated reputation changes reversed automatically by the system, at the end of the day.
If you ask why instead of who and encounter a situation where a post of your is downvoted multiple times--or multiple posts are downvoted--and you really don't know why and don't want to ignore it, you can post on meta to get input as to why it might be occurring.
With that said, while it's typically fruitless to worry about who, I recommend not worrying too much about why either, especially for posts you've already re-examined for quality and accuracy.