How so? If we kept our current socialist system as it now stands, I fully agree with you. However, if we moved towards something more moderate and transportable, like a social corporatist or alternatively more libertarian economic system, then:
1. the cost of living in our countries minus guaranteed social services would make our countries not so attractive to most of them anyway.
2. We would save money on unnecessary immigration bureaucracy which could go towards paying off our debts.
3. it would give both us and them access to a larger labour market and consumer market.
4. territorial wards would become meaningless. For instance, any Palestinian who'd want to live on Israeli land would be totaly free to do so, and same for an Israeli on Palestinian land. And since we'd lal have common citizenship, voting might have to be restricted to residents. In other words, we'd be allowed to vote for the local and national governments of where we reside at any given time. This would mean that if a people want to move to a certain land, all they'd have to do is move there.
Sorry. I've thought about it, and I still can't see the negative (again, assuming that we're adaptable enough to restructure our current socialist system towards a more social corporatist or liberarian one).
Now as a temporary transitonal phase, I could see granting governments the right to expect people entering their repective countries to know the local lingo. But otherwise, non-socialist countries ought to adapt just fine.