Maybe what we really need are different terms to distinguish the two. Remember that kink-shaming post last year, and how most of us felt that saying no to something didn't mean OMG you're sick for being into that? Well, not getting turned on from reading explicit slash (or het) doesn't make one a homophobe (or hetrophobe). Neither does simply not seeing a particular character as gay or straight. And right now, there's no way to say that without either risking pissing off your author or jumping through linguistic hoops.