In most cases, when we upvote we don't really want a post to have a ridiculously huge payout, and when we downvote we don't want to wipe it out completely. We want something reasonable between those two extremes.
Imagine upvotes and downvotes merged into one entity and working like this:
I want this post to be worth not less than X and not more than Y.
This gives you the option to be careful and reasonable when achieving your aim. This way everybody can see that you are not being extreme, which helps a lot when emotions are concerned.
(Of course, the blockchain needs to be able to execute that intention, which might be a bit complex, but the complexity needs to be hidden away from the user.)
And note that this method is backward-compatible: the current upvote is equivalent to setting both X and Y to infinity, and the current downvote (a.k.a. flagging) is equivalent to setting both X and Y to zero.