A court has given the town of Publier, in eastern France, three months to remove the work.
If it fails to do so, it will be fined €100 (£80; $105) a day.
Mayor Gaston Lacroix said he will try to relocate the marble statue on private land.
The image of the mother of Jesus has been standing on the shores of Lake Geneva since 2011.
It has been the subject of local controversy for some years after it was paid for with municipal funds.
France has long had strict rules ordering the separation of church and state.
In 2010, it became the first country in Europe to prohibit the wearing of face-covering Muslim veils in public spaces.
French town told park's Virgin Mary statue must go - BBC News
If it fails to do so, it will be fined €100 (£80; $105) a day.
Mayor Gaston Lacroix said he will try to relocate the marble statue on private land.
The image of the mother of Jesus has been standing on the shores of Lake Geneva since 2011.
It has been the subject of local controversy for some years after it was paid for with municipal funds.
France has long had strict rules ordering the separation of church and state.
In 2010, it became the first country in Europe to prohibit the wearing of face-covering Muslim veils in public spaces.
French town told park's Virgin Mary statue must go - BBC News