First of all, Allan Gregg, the head of Harris/Decima polls, was the one who made the comment about letting the office remain vacant...
But when you get to the real issue, GerryH and others are right: if we do away with the office of Governor General we need to replace it with something else. That means a complete constitutional overhaul and honestly, with the regional divisions in this country, I don't think it will happen and I don't know if the country would survive more rounds of polarizing discussion that would result from it. I think there is more appetite for senate reform than an abolishment of the Monarchy but it too is stalled for the same reason.
Also, if you listen to the discussion linked, its not just a wave of the pen: the costs would be huge, from reprinting of things like stamps and currency, replacing emblems on every gov't institution (from the RCMP to the post office to park wardens)and including a re-writing of the entire legal system. We complain about the tax dollars spent on royal visits, etc. but they are miniscule to the costs of eliminating the Crown.