Lemme ask you.... If the expression was figurative, and it completely understood, then why add-in the disclaimer at the end?
Wouldn't that be redundant?
Is your beef with how he structures his sentence then?
Look, I don't for a second believe he meant it seriously, or literally I should say, and I don't think any sane person would either. I think he was making a joke, not necessarily a good one, but a joke none the less.
Tell ya what, maybe you can answer the question that I posed earlier.... Would it be OK to yell "FIRE - figuratively speaking" in a crowded theater?
It would be stupid to yell that because that makes no sense.