He was very opposed to colonialism. That was a bad word. He believed that any nation or "empire" that had territories should treat all inhabitants, in the far-flung territories as well as near the center, as equal citizens with equal democratic and legislative and governing rights. In other words, he was against "colonialism" but he never used "empire" in a pejorative manner.
Dick Cheney may be fully aware of what Franklin meant when he uttered the word "empire," but even so it can't have escaped his notice that the word's contemporary meaning is much more provocative.