I heard a radio story this morning about a new report from the National Trust for Historic Preservation arguing that retrofitting existing buildings is much greener than building new LEED certified super-efficient ones. That's because construction, as such, is pretty non-green. This, to me, is only the beginning of the story and I highly recommend Sarah Laskow's discussion of the issues.
The big thing here is that even if we never tore down another building and just did retrofits of everything, it's not as if construction of new buildings would come to a halt. As long as the population grows, there will be new construction. So the right question to ask about a teardown is probably something like "how tall does the new structure need to be before a new build is greener than retrofit + build elsewhere"? That's going to depend on various features of the local situation. Adding more population to San Diego is much more climate-friendly greener than adding new population to Houston because of differences in the local climate and electricity sources.