I'd say you are, in a way. By focussing on what he did not do, you are implicitly denying the importance of what he did.
Defamation is not exactly a new thing. It's been part of the law for centuries. Until the 20th century, it was a crime as well as a civil cause of action.