Here's a fairly good treatment of the question (and related voting) from the Big Meta (meta.stackoverflow) : When Should I Vote? .
It covers the bases pretty well.
That being said, I think as long as you vote in good conscience, things will work themselves out in the system. (It's Stackexchange Karmic balancing!)
You can vote flexibly and playfully, but take it seriously. Your intention should ultimately align with the larger interests of the site or community. Not everyone is made in the same mold, has the same approach, or behaves identically -- but there are many models of good citizen, and many ways to provide a positive contribution.
Personally, (and subject to change) I don't downvote much. If I can improve (through editing) a question or an answer, I'll do that. If it's irremediably bad, I'll flag it. If it's technically wrong, or just bad advice, then I'll down-vote it. (Assuming I have votes left, which I don't.)
To the Great Sea of Askubuntu, we're just statistical motes. One user won't break the system, or bring Evil upon the site or her/his self.
If everyone is a dope, things will break. ("Dope Corollary" of Kantian Categorical Imperative.)
So: Don't be arbitrary, but in upvoting, you add reputation to the Great Reputation Bucket. It's hard to do harm this way.
