Suppose the only people allowed to immigrate to the United States were trained electricians. Increasing the supply of electricians by increasing the number of Electrician Visas would be in the interests of the American people as a whole. But increasing the number of Electrician Visas would be bad for incumbent electricians. Then someone might come along and say "hey, this Electrician Visa program is great—we ought to expand it massively so that plumbers and carpenters and structural engineers and architects can immigrate too." A bunch of conventional wisdom might develop among elites about the desirability of increasing "high-skilled immigration" with a general understanding that "skilled immigration" is a kind of euphemism for "building trades."
And yet sooner or later a native-born electrician is going to show up to spoil the party, talking about how the existing Electrician Visa program is hideously unfair and the whole concept of a building trades shortage is a myth. Just look at the high wages earned by plumbers and architects compared to electricians and you'll see the real issue here. Developers just don't want to pay their workers! We don't need more building trades immigration, we need to end the Electrician Visa program and we'll all be richer.
I think that's basically what's happening in the real world with Robert X. Cringely's critique of the H1-B visa program aptly titled "H1-B Abuse Limits Wages and Steals US Jobs" except in the real world "high-skilled immigration" is a euphemism for "engineers" and instead of an Electrician Visa program we have what amounts to a Computer Programmer Visa program.
See Adam Ozimek's critique as well but the Cringely critique illustrates one of the reasons why I like to make the broad case for higher levels of immigration at all skill levels rather than the narrow case for skills. The basic issue with immigration is that it's all about complementarities. When new immigrants arrive who are very similar to you personally the tendency is for your real wages to fall, but the real wages of everyone else (including the immigrant) to rise. And the important fact about this is that the gains are much larger than the losses. Because it's a positive sum interaction, in principle you could compensate the losers with redistribution. But in the particular case of immigration, there's no need to do that because very few people are very similar to you personally. The important thing is just to not restrict immigration to one particular kind of person. Only allowing computer programmers in is bad for computer programmers and only allowing plumbers in is bad for plumbers. But a broad-based expansion of legal immigration is good for everybody as long as the immigrants are genuinely here to get jobs. You wouldn't want to throw our doors open to criminals or let every foreign 83 year-old start getting Medicare coverage, but more foreign workers in general is good for American workers in general while if you scrutinize any particular potential migrant you're going to find plenty of people with a rational basis for wanting to keep him out.