Well, you've seen right here that lots of people do care. Enough to discriminate, illegalize, and murder.
That's why it's an issue. The sexually variant didn't all suddenly get together at a meeting of the Great Big Icky-Sex Conspiraceeee! and decide to impose themselves on the world, seeking...