Buyers are ditching home deals, while some brokers say they feel forced to racially profile people and turn down business.