That's why Robertson's answer scares his fellow fundamentalists. They see it as a step toward "relative morality." Pretty soon, you're letting your experience of same-sex couples erode your certainty that they shouldn't be allowed to marry. You might even start to think some women have good reasons for abortion.
Robertson has had squishy tendencies all along. In 1992, he said "if a woman gets raped by a syphilitic or something like that, I mean, she maybe should have a right" to terminate the pregnancy. In 2001, he said Chinese officials enforcing that country's one-child policy were "doing what they have to do." In principle, Robertson opposes abortion, the one-child policy, and divorce. But in all three cases, he's open to contrary viewpoints and exceptions.
Maybe he's just a hypocrite. Or maybe, like the rest of us, he's trying to learn from living.