Joe Stiglitz says high levels of inequality hold back economic recovery but Paul Krugman disagrees and I think gets the better of the argument. That said, I think Stiglitz's conclusion may be correct. Krugman says you "you can have full employment based on purchases of yachts, luxury cars, and the services of personal trainers and celebrity chefs." And I agree with him, you can. Or at least you could. But you don't and that seems significant to me.
My conjecture would be that high levels of inequality greatly complicate the political economy of expansionary policy. Expansionary policy isn't zero-sum, but it's not distributionally neutral either. In fact, there's no reason to think expansionary policy will be pareto optimal. In other words, positive-sum expansionary policies that have more winners than losers still create losers. So to the extent that you have a lot of inequality, your politics is naturally going to be more focused on questions of distribution than expansion. I think you saw that in liberal hostility to even temporary expansion of the Bush tax cuts and even more clearly in things like the GOP turn against Making Work Pay and the payroll tax cut.
A potentially related issue is that there's evidence that inappropriately tight monetary policy increases inequality meaning that insofar as economic elites care more about relative status issues than absolute income issues they have an incentive to push for tight money. Normal people, it seems to me, care about both so there's no incentive here for wildly too tight policies but it matters.