So what? If we say that it's not worth developing trade relations with a country unless it's our main trading partner, then we might as well trade only with the US and close our borders to the rest. If the goal is diversification, then we could lower trade barriers to even our most insignificant trading partners and even potential trading partners.
Looking at it that way, not only should Canada promote closer trading ties with Australia, but also with the 191 countries in the world including even North Korea to the degree that UN sanctions will allow it.
I'm in favour of increasing trade with everyone, but I don't think Australia and New Zealand are going to add much in either imports or exports.