Which lens should I get?

Lens choice is personal preference and it depends on what YOU need the lens to do. Like you said, they are two different lenses with two different purposes.

When I am unsure of what glass I want, I rent some. I would suggest that you visit a local camera shop and rent both lenses for a couple of days to see which one has more use to you.

The shop I deal with rents glass at a decent price, and if you buy a lens when you return it, they deduct the rental fee off of the purchase price (either buying the rental lens at the used price or a new lens of the same size).

Matt,

BEI has a good idea. Try using or renting the lenses before you buy. Also consider what you are shooting. The lenses you have listed have very different focal lengths. The 24-70 is not going to be so great for portraits and the 70-200 will not be as useful for scenic work.

depends on a great many things:

1) budget

2) experience

3) camera

4) what you are shooting