Nowadays anyone to the right of Jeb Bush is smeared by the left as racist, sexist, homophobe, etc. Ignore such nonsense, it usually just means you're being effective.